Statistically, our challenges in compiling this index were many, including choosing the most appropriate statistical method to aggregate the data into one composite Index, and at a more basic level, finding the most “suitable” set of indicators that sufficiently capture governance as defined by the Board of the Foundation, its Founder, and its Advisory Council and Technical Committee members.
The 2009 Ibrahim Index of African Governance is a work in progress, and is both a continuation and an improvement on the work of the first two years. We continue to put together a number of official data indicators that capture governance in the way that the Foundation and its partners perceive it. In addition, this year, largely as a result of direct input from our African partners and interlocutors, we have introduced a considerable number of qualitative indicators (expert assessment). We believe that these add substantial value and nuance in capturing aspects of governance, particularly in relation to assessing current governance performance.
This is especially the case given the highly lagged nature of a large number of official data, some of which had to be excluded because it reflected little of a government’s recent performance. However, while the worst of those legacy indicators were excluded, some indicators with a more limited legacy effect were still included.
At the practical level, we found that many official data indicators that we would have liked to include did not have sufficient data coverage and were not released or updated periodically enough, to warrant inclusion. This factor led us to exclude what could arguably be considered the most important indicators on governance: poverty indicators. On another front, and similar to what was done previously, for many indicators that we include the lack of a full dataset for the period 2000-2008 meant that we had to estimate missing values. For this, we used mean substitution and extrapolation. In addition, indicators which had only one year’s data were excluded.
The Ibrahim Index is a composite. Following the gathering of the raw data on all the indicators, a method was chosen to put the raw data on to a common scale, that is to say, to re-scale or normalise the raw data, so that it can be usefully combined to produce an overall score for each country. There are a number of statistical methods and data aggregation techniques to choose from. The IIAG uses the same method as in the past, namely, the “min-max” method. However, this year, a statistical technique was used to deal with outliers, given the high degree of sensitivity of the min-max method to outliers.
Fundamentally, the min-max method involves re-scaling the raw data values to a scale of 0-100, for every indicator, for every country, and for every year. This re-scaling is done using the formula:
[xt – Min(X)]/[Max (X) – Min (X)]*100
Where xt is the raw value for that indicator for a particular country in year t, and the Min(X) and Max (X) are the minimum and maximum values for that indicator over the whole period and for all countries. The final result was subtracted from 100 where necessary, so that a higher number always indicated a better performance.
The method used and the nature of the data mean that it is more instructive to look at scores and ranks in more recent years, rather than in the early years. A key reason for this is that data in the early years are patchy but data availability improves substantially over time. Comparisons of scores across sub-categories and categories are misleading and so should be avoided.Moreover, comparisons across countries (for the same period) should be governed by the non-trivial and considerable margins of error, which are present in any Governance Index or indicator. Those margins of error mean that score or rank comparisons when differences across countries are small should be avoided, since they would reflect a statistical “tie”.
Finally, this year, in order to render the Ibrahim Index more reflective of recent performance, we decided to use the latest available data for every indicator where it was available. This means that for the year marked 2007/2008, for example, 2008 data was used if available, and 2007 if not.
For more information, please read:
On the 2009 Edition of the Ibrahim Index of African Governance: Strengths, Challenges and Applications by Dr Daniel Kaufmann, Brookings Institution.