A Question of Quantity and Quality: How a participatory approach turned citizen-generated data into official statistics in Kenya

06.04.2023

Kenya is making solid investments in education, however its policy making requires a huge amount of complex data across a range of indicators – including social and household data, which the National Statistics Office (NSO) does not have the capacity to produce. Through a recent initiative supported by PARIS21, GIZ, GPSDD and the Government of Flanders, the Kenya National Bureau of Statistics and civil society organisations in Kenya have come together to use the detailed, insightful data produced by CSOs along with the robust quality standards of the NSO. In 2022, for the first time, education data from a CSO organisation was used in official reporting.

Photo of aerial view of Nairobi by night

A commitment to better education revealed data gaps in Kenya

With a young population, education is a major priority for Kenya’s Development Agenda. This is reflected in the fact that the education sector is one of the country’s major budgetary expenditures, at 19%, which has translated into Kenyan children receiving more years of education than the average for countries of the same income level.¹ However, the population is far from homogenous, with communities ranging from highly urbanised to pastoral, different races and ethnicities, as well as communities that may be isolated through geography or because they live in informal housing. Data need to be granular and disaggregated to reflect the diversity of the population, and to understand the impact these factors have on educational attainment and design policies that aim for equitable access to quality education.²

The country’s national statistical office, KNBS and the Ministry of Education are responsible for collecting, analysing and disseminating data on education which are used to make policies that ensure equitable access to quality education. However, Kenya recognises that the availability of timely, disaggregated data is a major issue in monitoring development goals, as highlighted by its recent Voluntary National Review.³

Overcoming data gaps started by bridging a divide

KNBS realised that it needed to take action to fill data gaps, however official data producers and civil society data producers needed to overcome entrenched views of how each other operated. Ben Avusevwa, Director of Statistical Coordination and Methods at KNBS, explains:

“With the introduction of the SDGs, we found that KNBS could not keep up with need for statistics and there were important data gaps. We were aware that civil society were producing data and statistics, but there was an atmosphere of mistrust between us; we didn’t always understand where their data had come from, and we tended to work at arm’s length.”

Florence Syevuo, Executive Director of the SDGs Kenya Forum, an umbrella organisation for CSOs in Kenya, describes the situation as a “tussle” between the NSO and CSOs, who were sometimes frustrated by the timeliness of the NSO’s data, or what they perceived as a lack of clarity around definitions, or even a perception that data was kept hidden deliberately.

“On the one hand, CSOs have been contesting the official statistics on crises affecting vulnerable groups, such as GBV or child mortality. They were saying that their situation has not been very well captured. On the other hand, CSOs have been trying to work with KNBS for a long time, but the NSO has a specific mandate, limited capacities and a more traditional approach. But this time we saw a lot of commitment from the KNBS side.”

Through a new project supported by PARIS21 and GPSDD with funding from GIZ and the government of Flanders among others, KNBS began working with local civil society organisations (CSOs) to complement and augment the country’s official statistics to help address some of these data gaps.

… and turning the page in collaboration between the government and non-state actors in Kenya

KNBS as the national statistics office is the custodian of official statistics, and as such its data is subject to rigorous national and international standards. However, it lacks the bandwidth to engage with the numerous CSOs and examine their data practices. CSOs, on the other hand, bring in-depth knowledge and expertise on specific topics, but their methods of collecting data lack scientific rigour. PARIS21 support to Kenya has built on existing partnerships and taken the form of technical guidance in formulating quality criteria, providing peer-exchange opportunities with other NSOs, and advocating for this novel approach among CSOs and development partners. Consequently, in June 2022, KNBS and the SDGs Kenya Forum invited CSOs to the inauguration of a new set of quality criteria for evaluating the citizen-generated data, introduced as part of Kenya National Quality Assurance Framework.^₄ Ben Avusevwa said during the inauguration:

"KNBS, sees a lot of value in these alternative data sources. We are now methodologically prepared to evaluate these data sets and to look whether they can shed light on important data gaps that we have. We are looking into institutionalising that collaboration by forming a technical working group with civil society organisations".

Following the inauguration, KNBS, the Ministry of Health, State Department for Gender, the NGO Board and ten CSOs, supported by PARIS21 and the government of Flanders, spent a full week looking at the sample CSOs datasets through the quality dimensions and evaluating whether they were fit for the purpose of filling key data gaps, especially in SDG reporting. KNBS invited these CSOs to present their data collections to statisticians and policy makers, and the CSOs in turn learned about the operating model of a statistical office. It was during that workshop that the organisation, Twaweza’s dataset successfully passed newly established criteria.

“Speaking directly to civil society actors, showing them our methods of work, but also learning about their work has helped us to overcome many prejudices and barriers”

said KNBS Statistician Sarah Omache, who took part in the development of the quality criteria and the meetings with civil society actors.

“We were also glad to hear how the CSOs’ perceptions of our work have changed, once they could meet us face-to-face”

More data, but also deeper insights from civil society’s expertise

The work to bring together civil society and KNBS has paid off, not only in more data, but also in benefitting from CSOs’ years of experience and unique depth of understanding in their fields of expertise. James Ciera, a professional statistician who has been working with the CSO Twaweza for over ten years, points out that not only can CSOs provide new evidence, they are able to go deeper into issues and shed more light on problems than could be gathered from official statistics. For example, school records typically used to measure education outcomes, may overlook children’s’ home environment, which can impact their school performance:

“What happens at home determines the educational outcomes of the child. Official administrative records and surveys can provide information about schools and test results; but to understand why children are succeeding at school or not, you need information from the home environment. You can learn a lot from how many books there are at home or the level of sanitation facilities in their house”

Twaweza has been collecting education data since 2011 in the form of education surveys (“Uwezo surveys”), which are modelled on the Demographic and Health Survey, a standardised questionnaire used in many countries around the world. More recently they conducted a study on the “Status of remote-learning among school-going children in Kenya during the Covid-19 crisis”. When undertaking their data collection, they used national sampling from KNBS, and worked closely with the Ministry of Education to evaluate the data collection tool and to interpret the results. Working closely with the government has also helped them to get a greater visibility amongst government and policy makers.

“The education sector takes a lion’s share of the budget. We knew that we cannot simply undertake the study and say that the learning is not happening – it was quite shocking and we expected that the government wouldn’t even listen to us. So, we ensured that the government agencies were closely involved in the design of our work and went with us to the field. This way we built trust and we obtained a strong support for the results”

Twaweza also took additional measures to obtain good survey results. They recruited and trained their survey enumerators from local communities and villages.

“You do not want to import people from other places. Local people understand the dynamics within the community and their presence is breaking the ice. This is extremely important to build trust with interviewees and obtain truthful and genuine responses”

Twaweza’s education data were released as official data by KNBS,⁵ marking a huge milestone not only for Kenya’s education sector, but also in paving the way for other citizen-generated data sets to be included in national data.

“We are very proud. We made a good progress in terms of data management in the past years. The fact that our data were recognised as official statistics has been a huge achievement and motivator for us. But it is also a huge achievement for the government”

How to ensure that citizen-generated data becomes a long-term solution to data shortages?

NSOs and CSOs have a lot to offer each other, but a space for dialogue is essential to reach mutual understanding and a plan to move forward. The goal of reaching the SDGs has provided common ground and a shared language for the different actors in Kenya’s wider data ecosystem, and efforts have been made by state and non-state to ensure that citizen-generated data turns into better policies. PARIS21, as a partnership, has been able to bridge these two groups of actors, bringing understanding of the wider ecosystem in Kenya and providing a forum for NSO and CSOs to dialogue and collaborate. The progress in using citizen-generated data in Kenya shows the potential of investing in the participation of non-state actors in the national statistical system to shed light on issues, which have been previously overlooked in official data. The example from Kenya shows how proactive and agile NSOs can build sustainable solutions to overcome their own capacity shortages. Kenya’s successful pilot is also a story of how governments and citizens can work hand in hand to tackle important societal or environmental issues.