Data: Use It or Lose It!
Humanitarian action is tilting toward the big data revolution but we need to abide by a few driving principles to stop us from falling over.
In this world of ubiquitous data, we can sometimes get caught up in hyperbole associated with what data will enable us to do. Don’t get me wrong. We need tried and tested tools for how we work. Yet, we actually have thousands of tools for every aspect of humanitarian action. It is like the biggest DIY-humanitarian action store on the planet.
What we need are tested models. Models we know work because they are based on solid evidence. Tools that we've used over and over again so that we know precisely when and how to use them in different contexts and for different needs. Otherwise, we are stuck using hammers and nails when we should be using lathes and injection moulding. To make this shift, we need data.
Big Data: Until Donors Are Willing to Put Up ‘Big’ Money to Go along with ‘Big’ Data, this Remains a Pipe Dream (And, honestly, why should they?)
Big data assesses and tracks broad variations, be they market preferences, social trends, financial fluctuations, commodity prices, or population movements, only the latter of which is related to humanitarian action. Combining social and news media with push-pull telephony can give us near “real-time” information about population movements and/or sentiments concerning a quickly evolving crisis.
This type of big data analysis is super important and yet we should leave such things to the likes of Google, Facebook, Palantir or Premise. We will never have the computer number crunching power and associated analytics to do the requisite analysis. Nor should we. These players have the core competitive advantages and they are happy to leverage these toward humanitarian action.
What we should be doing is forging partnerships with these big players. They are eager to learn about the complexities of our work and, if we stick with the Googles of the world, they won’t become competitors. Instead we can find a win-win within their philanthropic ventures. We can better leverage their big analytics, their big number crunching power, and their big wallets toward macro-level analytics as crises unfold.
This is great but most of our work focuses on the micro-level. We focus on individuals, their families and their communities. We focus on their needs and how we can work with them to better withstand and recover from a crisis. This requires a range of tools. This is where we find ourselves back in that massive DIY store, searching the aisles for the right tools but with no real experience (data/evidence) to choose amongst all the shiny contraptions on the shelves.
Small Data is Our Friend Too!
We need valid, compelling data at the micro-level. To test existing tools and models, we need to start collecting statically valid data at the single activity level. This will tell us which tools have worked where, under which conditions, and this will provide insights into that particular context.
The trick is to make sure that any mico-level data set uses common standards, is complete, and that it answers key questions about a specific intervention and about this type of intervention more generally. These data sets can then become fundamental building blocks. If we use common standards (as below) and well designed protocols, each data set can be combined to provide much broader trend level analysis.
These micro-level data sets, once combined, will show which tools worked in which conditions and, when combined with cost analysis, the return on investment associated with different interventions. We will get much closer to knowing how many people we can serve with which interventions. We can understand which partners work best in which contexts. We can scale-up or scale down different combinations given the needs. We will actually have some solid science behind our work and rather than walking aimlessly through the maze of a DIY store, we will have a highly refined, tested, toolkit that we can use over and over again no matter the job.
Stop Messing with Standards
To start collecting data at the micro-level, we need to have standard questions/protocols that are always used for each sector/activity.
We have some standard tools, like SMART nutrition surveys, food consumption scores (FCS), the coping strategy index (CSI), household dietary diversity scores (HDDS), etc. These are great.
The issue is that we need to maintain their integrity! I have seen far too many organisations take one of these standard tools and modify the questions, dropping some and adding others, because they think this will make the tool better. Craziness! Would you expect a doctor to change a standard diagnostic tool because she thought she knew better than the medical community? Would you expect engineers to create their own load-bearing ratios and calculations? Would you want the police to just decide what’s wrong and right? Of course not.
We can’t let our overly developed linguistic and analytical skills allow us to be dismissive of accepted models. Use the models. Collect the evidence. Then, it can be compared across activities/contexts.
In fact, we need to go further. We need standard survey protocols for every sector and activity. The Sphere Project has gone some way toward this. (www.sphereproject.org) We need to use these standards—not adapt or change them, but use them, over and over again. Don’t’ go about re-inventing the mouse trap.
Good God, Please! Move beyond outputs!
I am tired of working with organisations who spend loads of money to send out monitors into some of the most dangerous operating environments just to count buildings or infrastructure or to review tally sheets. We shouldn’t need to point this out. We should not ask people the most mundane of output level questions only, e.g. did you go to the clinic, did you receive your plumpy nut, how often do use the latrines, et. al.
I have seen the last one, as ridiculous as it is, on countless surveys. Imagine the response: “Umm, uh, once a day???” The question basically asks how often they poo. Come on!
I know how important latrines and WASH facilities are. I know how important it is to confirm that people get the right amount of food support or that they actually went to the clinic there to support them. We need to validate that organisations have done what they are supposed to, at the right time and for the right people. This is fundamental. However, the cost of data collection is in getting the people out to the intervention sites. If we just ask simple output questions, we are missing a major opportunity.
We need to ask outcome level questions. For instance, if the intervention is a clinic, we can ask about the quality of care and general impression within communities about this type of support. For nearly all humanitarian actions, we can ask about how the support enables individuals, families and communities to better withstand and recover from shocks. We can ask investigative questions about community dynamics (including host communities). We can ask about traditional coping mechanisms and their relation to direct aid. We can assess shifts in gender relations due to the crisis. We can ask how the intervention enables people to focus on other issues, e.g. improving food security, livelihood strategies, etc.
Some of these are complicated and actually collecting valid data can be challenging. Perception questions are influenced by current dynamics, e.g. the state of the person at the time of the interview. This means that these become a “snap shot” at best. If done well, however, even these snap-shots can become baselines for other longitudinal surveys. Every piece of data becomes a brick toward a sturdier evidentiary structure. So long as we are keeping to our standards and using common sense, we can contribute to building something great.
Mobile Phones Are Everywhere. Use Them.
Yes, we should use SMART phones and sexy-groovy on-line user interfaces to manage “live” surveys and to do basic analysis. The barriers to entry for these have become so low that no organisation can sensibly refute using this basic technology.
We should also recognise that most people we serve have mobile phones and some may even have smart phones. SMS push-pull engagement strategies through telephony is a really interesting and a quickly developing arena. UNICEF’s RapidPro was an early mover in this arena and remains compelling as it expands to new areas.
Of course, technology also includes card-based cash disbursements, bio-metric registrations (or even more advanced systems like UNHCR’s IrisScan), and satellite imagery for infrastructure, agriculture, urban contractions/expansions, and waterway changes (droughts and floods). It includes a range of new packaging materials and technologies for everything from clean water to solar power. The issue with these is that they require a cadre of technical experts and they only make financial sense when they can be used at scale. These are big tools that require big budgets, similarly to big data. Don't let the hoopla in the press melt your hard, rational humanitarian heart. get back to what works for the people we are there to serve.
This brings us back to the humble survey. Surveys should be integral to every intervention. They should use standard protocols and move beyond simple output measures, as described above. We should conduct these on phones, tablets or laptops, using geo-referencing and collecting video/audio information as useful.
We use SMART phones in the most sensitive areas in Somalia so are no real security issues. In fact, if anyone anywhere is still using paper and pen, they should be fired and/or their funding should be halted. Pen-paper-Excel is prone to massive errors and it simply isn’t necessary. The ubiquitousness of survey software and mobile-interfaces make using pen and paper seem like a relic of some distant past.
Make Your Graphs Pop!
Once you have the data, don’t waste it on monotone layouts or impenetrable graphs generated from the latest Excel package. The graph is a perfect distillation of data into a user-friendly visual that demonstrates trends and differences and that answers specific questions unequivocally. Don’t mess around with shite! Make your graphs pop!
In brief, they should be simple enough so that a lay person can understand them, use different but complementary colours for variables (although avoid using more than 5 colours), use complete words/phrases for legends and labels, and include expository text in the body of the report that explains the key issues.
In fact, we should really be moving to interactive reports where graphs/charts and maps can actually show progressions over space or time. Hans Rosling’s Gapminder (www.gapminder.org ) is a great example of this.
Get the Data Out of the Hands of M&E People
A common complaint is that data stays in the sphere of M&E teams instead of becoming an operational tool. I hate to say this but this is typically due to two basic reasons. First, the M&E team is being overly sensitive and territorial about its data. This should be remedied by direct managerial action. Data is a tool to be sued; not a tool for reports. Second, the data, its graphs, its tables, and any exposition is simply too poor for operational utility. We will write a specific post on this but, really, if operational people aren’t using your data it is probably the fault of the data and analysis rather than the fault of the operational people.
Share Data Far and Wide
Like an M&E team that hordes its data, any data hoarding should be discouraged. Data is meant to spark conversations, to provide new insights, to prompt project adaptations or changes, to deepen individual and organisational learning. It can only do so if it shared.
Fine. Take the time to QA it but don’t take so long that it becomes dated. Fine. Share it first with those immediately concerned but then broaden it to other divisions/departments, organisations, and god forbid, even to donors.
Share it warts and all. It needs to be shared for us to get better. It needs to be shared so that all of our actions get better. The people we are there to serve deserve this level of analysis, thought, and improvement.
