Posted by: kristinej | August 6, 2013

August 6, 2013

The question “what do you do?” or “what are you studying?” is never an easy one to answer. Here’s a long-winded attempt to answer that question, perhaps fueled by the increasingly frequent question “what will you do when you graduate?”

If I’ve gained anything this summer, it’s an appreciation for how complicated the idea of international development is.  Nothing is black and white. Are we doing the right thing?  Is our project really helping people?  Is it helping the right people?  Are we using our money in the best way possible?  Are there unintended consequences that we never thought of?  Should we even BE here??

I didn’t have to grapple with these questions as much when I was in Mali.  I could see, and still do believe, that Project Muso was doing a lot of things right.  Living in a somewhat isolated area, I was pretty withdrawn from the larger development world of the World Bank, national development agencies, contractors, and large NGO’s (non-governmental agencies).  What I saw was a few dedicated people implementing small-scale programs that were visibly making a difference in the community.  A new ward at the health clinic meant that pregnant women had a place to sit when they went for prenatal consultations.  Mothers could bring their children to my host mother’s house for a rapid malaria test, and she would accompany them to the health clinic if needed. Evaluation results show child disease rates falling in Yirimadjo, and the project has scaled up from 20 Community Health Workers to 75 (http://www.projectmuso.org/node/116).  The project has been incredibly efficient, working within a very small budget and drawing its staff from the community. Still, its scale is small and it’s taken years to get this far.  It’s a drop in the bucket, making its own small difference while at the same time just one of thousands and thousands of development projects in Africa.  What makes some more effective than others, and why, after all this time, do we still not really know what works best?

These questions are the reason I was drawn to evaluation.  Sure, I could go sit in a classroom and watch children learning with new textbooks that have been donated, but are they actually learning more?  Will they spend more time studying at home? Will more of them graduate?  At Social Impact I worked mostly on performance evaluations (sometimes also called process evaluations).  In most cases a donor such as the United States Agency for International Development (USAID) would hire us, as independent evaluators, to determine whether or not their grantees were spending American money well.  Were they doing what they said they would do, and were they doing it well?  Did the program design make sense and was it the most effective way of reaching the intended objective?  Often they would also ask us to determine what impact the program had had, but this was hard.  Sure, we could see if a country’s political situation had improved over the last 3 years, but could we say it was due to a specific project?  Hardly.  There are so many factors.

The business of evaluation is booming, despite, or perhaps because of, difficult financial times.  Donors want to know where their money is going and that it’s being used well.  At the same time, impact evaluation has emerged as a way of using statistical methods to attribute changes to a specific project.  The most rigorous of these methods is the randomized control trial (RCT) – the same method used in medical trials.  You give two identical groups of patients two different treatments (or one treatment and one placebo) and see how they differ at the end of the trial.  Since you randomly select the treatment and control groups from within a mostly homogenous group, the treatment is the only factor that could have led to a difference between the two groups at the end of the trial.

This same practice is now being applied to development projects.  The project I’m working on, for example, will install satellites and computers in 70 schools in 4 districts in Ghana and will remotely teach math and English classes to students there. On the evaluation side, we will randomly choose 70 schools from a sample of 150 schools that have been deemed eligible for the program.  We can therefore expect the group of treatment schools to be, on average, the same as the group of control schools.  We’ll test English and math scores in all schools this fall, and then again in 2 years when the kids in the treatment schools have finished the program. If the scores of the treatment kids are significantly higher than those of the control kids, we will be able to say that the program was effective in raising learning levels.

While the program and the evaluation won’t start until this fall, my role this summer has been to get everything set up.  I’ve worked on our IRB application, which gives us permission to do research with human subjects, collected data on all of the schools in our districts, and designed a questionnaire to give to the kids in the program.  Yesterday I finally had the chance to pilot the questionnaire.  We visited a primary school an hour outside of Accra, and had a local surveyor ask the kids some of the questions to see how long it took and how well the children understood.  We will then make revisions and the team will pilot it again up north near our other target district.

Where am I going with this?  I think evaluation is important, I really do.  We all want to know that our work is making a difference.  But there’s a limit to what it can do.  While some evaluations are done right, and the results are shared and incorporated into policy, others are rushed, poorly done, and ultimately ignored: a colossal waste of money.  In other cases, the focus seems to be more on checking boxes, reporting to donors, and creating jobs in what is now essentially a global business just like any other.

So what will I do when I graduate?  I’m not sure.  But this summer has given me a lot to think about.  While I love Africa, I have always been highly uncomfortable with the idea of being a white person coming in with money and my American degree and trying to “do good.”  I recognize that I’m here mostly for selfish reasons: to progress my career, to learn, and to have some adventures.  But still, I try to live humbly within the context.  There’s nothing I hate more than going to a restaurant and hearing white people talking loudly about things they don’t like about Africa, or discussing Ghana’s problems, as though the waiters or other patrons can’t understand English.  And I cringe whenever I see tourists pull out their camera in a village without so much as a greeting, or walk down the street in short shorts as though they are still in America.  And then there are development agencies whose staff live in gated homes with pools, cooks, and drivers and may not know even a word of the local language.

My goal for this year, then: figure out how to reconcile wanting/needing to have a stable career and some standard of comfortable living with doing work that is not so removed from actually helping others.

DSC00493

Learning math

DSC00490

Primary school in Dangme East district

 

Advertisements

Responses

  1. I love this post! Captures everything so well; and I’m pretty sure I can name many of the bad examples you reference 🙂 Can’t wait to have you back and hear more about everything!

  2. I totally agree with you about the tourists. I once lectured two American couples about their behavior on a bus in Barbados. Same issues,

  3. Beautifully written! Captures many of the issues I’ve had in this field and is largely the reason I chose to switch to domestic work. Good luck figuring it out and I look forward to hearing the outcome of this reflection!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: