Tanzanian Water Pumps
Thousands of water pumps stand in Tanzanian villages; some work, some are broken, and a repair team can only reach so many. We asked whether a pump's own record, its age, type, management and location, can tell you which ones will fail.
- Client
- University of Nottingham
- Type
- Academic Project
- Year
- 2023
- Our role
- Joint research project
- Services
- 59,400
- Waterpoints in the dataset
- 80.9%
- Best accuracy, Random Forest
- 99% → 77%
- The overfitting we caught
01
Two independent answers
Two of us ran separate pathways through the same data, choosing our own cleaning, features and models, then cross-checked each other's data with each other's models so every difference could be measured. Our pathway grouped rare funders and installers, log-transformed water amounts, added each pump's age, scaled the geographic features and compared AdaBoost, XGBoost, Random Forest and a neural network.
02
The 99% trap
Dimensionality reduction lifted accuracy sharply, with the neural network reaching 99%. It was too good to be true. Five-fold cross-validation showed the models were memorising: accuracy fell to about 77% across the board, so we did not rely on it. The honest number is nearer 80%.
03
Results
Random Forest and Gradient Boost performed consistently across splits and datasets, with XGBoost close behind.
| Model | Accuracy | F1 | AUC |
|---|---|---|---|
| Random Forest | 80.93% | 80.39% | 90.59% |
| XGBoost | 78.75% | 77.35% | 89.58% |
| Neural network (MLP) | 76.71% | 75.48% | 87.21% |
| AdaBoost | 72.02% | 69.31% | 79.42% |
Our pathway, 80/20 hold-out split.
04
What matters
Waterpoint type, extraction type, management and location drive whether a pump works. Older pumps fail more often, and pumps with a public meeting behind them are more likely to be working, a sign that community engagement matters as much as hardware.