Welcome to agrodem user’s guide!

Achieving universal electricity access by 2030 will require a rapid increase in the rate of new connections and in levels of investment, particularly in countries with low level of access to electricity. The challenge is significant. Designing and selecting the optimal electrification approach requires access to reliable data and information regarding electricity resource availability, demand levels, economic activity and functional infrastructure to name a few.

The paucity of such information may hamper electrification progress. However, this situation is gradually being improved with the increasing availability of new data and analytical tools, especially in the field of geospatial analysis. Geographic Information Systems (GIS) and remote sensing techniques are becoming openly available and can now provide a range of location-specific information that has not been previously accessible. Take for example the field of agriculture, where data is rapidly approaching the scale of ‘big data’. They can provide farm stakeholders with spatial and temporal information about climate and local weather, soil conditions, crop quality, field biodiversity, and crop yields.

Inclusion of agricultural electricity demand can enhance the richness of the geospatial electrification planning process. To date, demand estimation has typically depended on measurement of population density and an assumed average residential demand. On occasion, this is augmented by including some types of productive loads (e.g. large industrial or mining loads). However, existing and potential demand for electricity from activities related to agriculture and agro-business, is missed usually not included.

This document presents an analytical and spatially explicit approach for estimating aspects of electricity requirements related to agricultural activities; namely irrigation and few selected post-harvest activities. The underlying work has been a collaborative effort between Vivid Economics and KTH Royal Institute of Technology and is part of a multi-year ESMAP-funded project - Geospatial Electrification Planning in the Africa Region (P165617) - aiming to facilitate the execution of geospatial electrification planning activities in various sub-Saharan African countries.

The following sections provide a brief overview of this approach with indicative examples for the main processes included and links to the code developed to implement those processes in an open and reproducible way.

Contents

Overview

This document serves as a data/process descriptor for the code available at the project github repo. Note that what is included here is far from exhaustive. The aim, instead, it to highlight the main methodological steps as well as to provide a better understanding of the open source code developed to support the modelling exercise.

Note

Detailed documentation supporting this project is available here.

General info

The methodological approach of this exercise is visually presented below.

_images/overview_diagram.JPG

Methodological flow and key modelling elements of this exercise

The core part of the methodology is the irrigation model, which was developed to provide an estimate of crops’ water and electricity requirements in a designated area of interest (AoI). The ‘agrodem’ model has been constructed as such to allow for its full customization by the user based on available data, information and other modelling constraints.

Downscaling methods have been explored in ordert to overcome data limitations or low resolution crop allocation data. The supporting metholology can help users build up input data in higher resolution using existing tabular data from surveys or statistics.

To evaluate future scenarios, relevant code was developed in order to enable users explore hypothetical alternative scenarios of cropland expansion (“extensification”). This component enables the user to evaluate the impact of expected changes to crops, as well the impact that policy can have on the expansion of crops to new areas over time. Users can interact with the code and create alternative future scenarios that best suit the purpose of the modelling exercise.

Finally, the model has been augmented to allow for assessment of electricity requirements for post-harvest activities (milling, drying, pressing, cold storage etc.). The model was developed as such to allow calibration of input data depending on the type of crop and AoI.

The components described above, create important blocks of work. Each block describes a particular activity and is usually characterized by three elements, namely inputprocessoutput. It also comes with open source code (available in the form of jupyter notebooks) that provides a clear step-by-step description of how to run embeded processes. Sources are linked where needed (e.g. equations, specific values, assumptions etc.) for transparency and open review.

Note

This is a spatial analysis, therefore some processes require either the installation of spatial libraries in python or the use of a GIS (check QGIS) environment. It is recommended that the user uses Python >= 3.5 through anaconda distribution; all required python packages are included in the full_project_environment file. QGIS plugins developed and used in this exercise come with separate documentation on installation requirements.

Irrigation module

Overview

This module estimates the electricity requirements to pump ground- and surface-water for crop irrigation. The model simulates in a simple yet scientifically sound manner the link between crop distribution, water use and electricity demand. The model is sufficiently flexible to incorporate potential changes in harvested area, climate, irrigation technologies and water management constraints. The methodology consists of three main phases, set out in the bullets below and summarised in Figure below.

_images/Irrigation_Fig.jpg

Methodological steps of the of irrigation model

Phase 1 models the agricultural calendar and the corresponding planting, growing and harvesting seasons for crops grown in the study area, as determined by input geospatial data. Temporal and spatial criteria regarding the crop water needs in the various climate zones and land conditions are taken into consideration.

Phase 2 models the reference (ET0) and crop (ETc) evapotranspiration in each of the modelling months. Water requirements are then estimated as the difference between ETc and effective rainfall in a given location.

Phase 3 estimates the electricity (kWh) necessary to supply the required water in each location. The assessment of electricity requirements depend on the morphology of the land, both underground and over ground, and take into account the different operating and application pressure levels required under different irrigation techniques and technologies.

Input data preparation

Input 1: Crop calendar that describes the crop phases (or cycles) within a calendat year. Note that this may vary per crop and region. That is, the modeler can specify different crop cycles per AoI (e.g. admininstrative level) if this information is available. An example of such a file is available in the project’s repository as Sample_Maize_Crop_Calendar for rainfed maize in Mozambique.

_images/Irrigation_Fig3.jpg

Sample crop calendar for rainfed maize in Mozambique

Input 2: Crop layer where each location represents a potential demand node for irrigation (e.g. farm). The resolution (spatial & temporal) may vary and it depends on data availability for the AOI. Once collected, the crop layer shall be transformed into a vector dataset (in csv format) that contains the following attributes:

  • country (name)
  • state (name - admin 1 or 2)
  • lat, lon (deg)
  • crop (name - modelling crop)
  • Fraction (%)
  • harv_area (harvested area in ha)
  • curr_yield (Current yield in kg/ha)
  • max_yield (Maximun yield in kg/ha)
  • gw_depth (Ground water depth in m)
  • sw_dist (Distance to surface water in m)
  • sw_depth (surface water depth in m)
  • elevation (in m)
  • awsc (Water storage capacity of the soil in mm/m)
  • sw_suit_idx (Surface irrigation suitability index: 1= suitable 9999= non suitable)
  • prec_i (Average precipitation in mm/month; i=1-12)
  • srad_i (Average solar irradiation per month in kJ m-2 day-1; i=1-12)
  • wind_i (Average wind speed per month in m s-1; i=1-12)
  • tavg_i, tmax_i, tmin_i(Average, Max, Min temperature per month in C; i=1-12)

An example of such a file is available in the project’s repository as Sample_Moz_Maize_1km.

_images/Irrigation_Fig2.jpg

Sample of crop input data showing the supporting columns with attributes

Note

  1. Features related to surface water (sw_dist, sw_depth, sw_suit_idx) can be extracted with the used of this QGIS plugin. It was developed by the team and together with instructions for installation and use.
  2. Extraction of other features can be done using the open source script Agrodem_Prepping, which is based on spatial packages and Qgis.

Parameterization & model run

Once the crop calendar information is collected and the crop vector layer fully attributed, the irrigation model is ready to run. Note that this is available in the project’s repository as agrodem.

Note that there are a few input parameters that need to be determined in the model per se. These include:

  • Kc factor for init. – dev/mid – late crop cycles (source FAO )
  • Current yield of the crop (in kg/ha)
  • Maximum yield of the crop (in kg/ha)
  • Effective Rooting Depth (in meters)
  • Field Application Efficiency – aeff (%)
  • Distribution Efficiency – deff (%)
  • Pumping hours per day (in hours)
  • Pressure head (in meters)
  • Powered pump efficiency – electric (%)
  • Motor efficiency (%)

Note

These values are available in the literature. However, they are predominantly based on value-laden judgements and assumptions of the modeller, informed - in best case scenario – in consultation with agriculture experts.

Output data

The output of the model indicates electricity requirements for irrigation of the selected crop and AoI. The spatial resolution of the results are defined by the initial vector later and stored in .csv format. Each row indicates a particular location (e.g. farm); and each column indicates a particular attribute for this location. These include all attributes used to derive electricity requirement in the first place and products of the analysis (water and electricity requirements).

Results are available in any GIS compatible, OGC complaint format (e.g .shp, .csv, .gpkg, .tiff). We have selected the .csv format as it can provide information in tabular form but also be visualized in relatively easy and straight-forward manner in a GIS environment.

_images/Irrigation_Fig4.jpg

Indicative results indicating locations of rainfed maize in need for irrigation in the base year (2017-18) in Mozambique

Note

The final result file includes only the locations with non-zero electricity requirement. This is to reduce volume of output data. One might select to extract the full list of locations by minor modifications in the code base, if interested in all products of the analysis.

Special notes

The irrigation model elaborates on three major steps that assess electricity requirements for irrigation (surface or ground) for an AOI. It can receive crop allocation data at varying temporal and spatial resolution and is modular, thus fully customizable as per need.

However,

  • Level of parameterization is high and highly dependent on experts’ value-laden judgement. That is, model input parameters should be decided with caution and under the consultation of local agriculture/energy experts.
  • The model can be used for a quick, screening analysis however one should be aware that many assumptions were set in place. For example, the model assumes that water reservoirs (both surface and underground) have unlimited flow capacity for irrigation purposes. In reality limits do exist – these are usually covered in detailed hydrological models/analyses – yet not part of this analysis.
  • Spatial resolution of input data may have an impact on the results. Low resolution is bound to rough assumptions; whereas higher resolution can leverage spatial information with higher accuracy – and thus the insights one can get out of this exercise. This part is (partially) covered in the next section.

Downscaling module

Overview

High resolution crop allocation datasets are not usually available in open access sources. In principle, ‘downscaling models’ use high-resolution input data (e.g. in our case temperature, precipitation, soil pH etc.) which have predictive power over low-resolution input data (e.g. in our case national or admin level aggregate values of harvested area, production etc.).

In order to do so, we have used the Fine Scale Land Allocation Tool or FLAT model. FLAT is a statistical-based open access tool that combines dependent variable data measured at an aggregate level (e.g., state/provincial or national) with independent variable data measured at the pixel level to produce estimates of the dependent variable at the pixel level.

Input data preparation

In terms of the dependent aggregate values of harvested area, we have used FAO Agro MAPS Global Spatial Database of Agricultural Land use Statistics. The database provides agricultural statistics and data collected and compiled from national resources, surveys, and official in-country agencies, assuring some degree of consistency across different agricultural products and geographic areas. This consistency is considered highly important in this exercise as it allows replicability to other regions or agricultural products.

For the purposes of this exercise, level 2 administrative crop data were used as shown in Figure below.

_images/Downscaling_Fig.jpg

Visualisation of harvested area data of rainfed maize per admin level 2 in Mozambique, as retrieved from FAO Agro-Maps

Note

In case input data is outdated, they will need to be projected to the base year. For example in this case the total harvested area for maize was equal to only 782,608 ha in 2008 (input data), while 2017-18 FAO estimates indicate about 1,962,000 ha in Mozambique. This can be done through a python script available as Preparing Agro Maps.

Then, the above dataset is transformed into a base (vector) layer at the targeted downscaling resolution (e.g. 1km). This is used later on to extract the predictor attributes per location. The steps for this are as follows:

  1. Project crop layer to target crs (in this case EPSG: 32737 – WGS 84/UTM zone 37S )
  2. Calculate “area” in sq.km and “perimeter” in km for listed admin areas
  3. Convert “area” from sq.km to ha
  4. Create the base grid mesh (10km, 5km, 1km, 500m or 250m)
  5. Only keep locations where harvested area for the listed crop is available.
  6. Convert polygons to centroids indicating the location of crops at the selected resolution.
  7. Export output as a .csv file

Note

In order to facilitate the process, we have developed a QGIS plugin that together with instructions for installation and use is available at the project’s repository.

Finally, once base (vector) data layer is ready and available as a .csv file, predictor values are extracted from raster layers. As predictors in the FLAT model, we may use the following variables:

  • Average temperature
  • Average solar irradiation
  • Average precipitation
  • Average wind speed
  • Soil Ph in H20
  • Depth to bedrock
  • Bulk density
  • Clay content
  • Texture class
  • Soil organic carbon
  • Drainage class
  • EVI
  • NDVI
  • Elevation - Slope
  • Cropland extent

The extraction process yields a .csv file that serves as input to the FLAT model. An example of this file is available as FLAT_input_Maize_10km.

_images/Downscaling_Fig2.jpg

Snapshot of the .csv file to be used as input in the FLAT model

Note

All of the above data can be collected either in their raw format or using a Python script (GEE_Imagery_Extraction) that was developed specifically for the extraction of the Images and ImageCollections using the GEE API.

Note

  1. The extraction process can be conducted through code available as FLAT prepping in the project’s repository.
  2. With regards to the imported geospatial layers (both raster and vector files), each one should be re-projected into the same coordinate system e.g. EPSG: 32737 – WGS 84/UTM zone 37S (+proj=utm +zone=37 +south +datum=WGS84 +units=m +no_defs) for Mozambique.

Parameterization & model run

The FLAT model can be executed with the following steps.

Step 1. Go to directory:

>/ FLAT_model/src

Step 2. In the directory run:

>/ Rscript RforFLAT.R flat_input.csv 0.534 1 15

where:

  • flat_input.csv is the output csv of FLAT prepping process
  • 0.534 is the target spatial resolution in arc.min
  • 1 is the number of modelled crops
  • 15 is the number of independent variables

Note

This process requires R to be installed at your working station. Running the above script creates nine derivative .csv files that are needed in Step 3. These include cropnames, pixels, states, data, statelevelcroparea, name, pixelarea, statelevelareainfo and variables.

Step 3. Copy the nine .csv files into the resource directory:

>/ FLAT_model/src/resource

Note

FLAT.gms should be located in the same directory as well - default by installation.

Step 4. Open GAMS Terminal; move to resource directory and run:

>/ gams FLAT.gms

Note

This will run FLAT model. Once complete, the log file (FLAT.ist) will be generated. This can be used to monitor the specifics of the run and track any issue in the debugging process (if needed). The result file is a .dat file usually under the name “finalresults.dat”. Other output files might include “Pixel-level cropland predictions against FAO aggregate values”; “Coefficient estimates”; ”Standard error”; “Covariate matrix for parameter estimation”.

Step 5. Export results (Optional)

>/ FLAT_model/src

run > Rscript dattotiff.R maize finalresults.dat (to generate a raster file (.tiff) of the results) or

run > Result.r (to export results in .csv format)

Note

Transformation of the result (.dat) file into a .csv file that can later on be used for the irrigation model can also be conducted using the script Dat to csv.

Output data

The output file provides a prediction the downscaled crop allocation. Each location (point in the vector layer) is attributed with a fraction of the total harvested area of the admin level it belongs to. Aggregation of the downscaled results per admin layer sum up to the original values.

_images/Downscaling_Fig3.jpg

Example output from the downscaling process for rainfed maize in Mozambique

Special notes

The downscaling process is a good – yet experimental – way to achieve higher granularity of crop distribution, especially in areas where there is data scarcity.

The FLAT model that was selected in this assignment is open source (although GAMS is needed for big datasets) and straightforward to test, use and customize. It predicts cropland allocation using pixel-level biophysical attributes which are openly available at the desired spatial resolution (1 km). The econometric approach provides estimates of the effects of biophysical factors on cropland allocation.

FLAT performance metrics are in alignment with available literature; visual inspection of results does also agree with qualitative findings from sample agricultural survey. However, the selected cross-validation approach highlighted that inconsistencies in the sample dataset are high to achieve any satisfactory results. This highlights the need for standardization of collection, processing and dissemination of survey related datasets.

Note

A first and second order validation process was conducted in this project and is available in the project report. This was implemented through a Python script available as CrossValidation.

Future Scenarios module

Overview

Electricity requirements for irrigation in the future will depend on intensification and/or expansion of selected crops over the area of interest. Here follows in brief a methodology (and supporting scripts) that may be used to answer two key questions:

  • What is the physical suitability of land in terms of crop “extensification”?
  • How much of the potential expansion is economically feasible?

We have developed a spatially explicit approach that provides flexibility to create scenarios of future crop expansion covering both physical and economic suitability over an area of interest. Our approach builds upon a study by Chamberlin. The basic steps are presented in the Figure below.

_images/FutScen_Fig1.jpg

Methodological approach for generating future scenarios of crop expansion

Step 1. Involves the definition of physical suitability for expansion. That is, a layer that indicates areas where crop can be expanded in the future vs areas that are restricted from crop expansion based in bio-physical factors.

Step 2. Involves the economic feasibility of expansion. That is, a layer that indicates hot spots for crop expansion based on high expected net revenue estimates.

Step 3. Involves the combination of physical and economic feasibility layers into a proxy layer for future crop expansion that reflects indicative actions related to productivity gains, environmental resilience or both.

Input data preparation

The preparation of physical suitability (step 1) relies on the collection and processing of the following layers.

  1. ESA land cover map
  2. IUCN protected areas
  3. EU flood maps
  4. WorldPop population distribution
  5. Total new harvested area for the modelled crop at the end year (e.g. 2040)
_images/FutScen_Fig2.jpg

Preparing physical suitability map for future crop expansion

Data sets shall be collected and resampled at the same target spatial resolution (e.g. 500m) and projection system. Then, they shall be re-classified into binary (0-1) suitability layers. In general, for the population layer, cells with more than a threshold population (urban areas) shall be excluded from potential future expansion. For the land cover layer, built up areas, permanent snow and/or ice, open water, bare areas and no data shall be excluded. Similarly, protected areas and regularly flooded areas shall also be restricted from future expansion.

Note

The classification and/or definition of extraction zones may be differentiated based on projects mandates (e.g. environmental policy, agriculture sector plans etc.).


The preparation of economic feasibility (step 2) relies on the collection and processing of the following layers and parameters.

  1. Raster layer indicating the location of markets (binary 0 – 1)
  2. Raster layers indicating attainable yield for modelled crop (GAEZ)
  3. Factor of attainable yield (numerical value provided by user)
  4. Raster layer indicating output prices per ha of modelled crop
  5. Raster layer indicating production cost per ha of modelled crop
  6. Crop prices shock (numerical value provided by user)
  7. Alpha and gamma controls for distance decay equation

Following the approach of Chamberlin et al, the suggested approach used an explicit economic framework to estimate returns to agricultural production in each grid cell, based on land productivity, market access and market prices. Cells that can be profitably cultivated are considered as suitable for cropland expansion. The economic profitability is evaluated at each grid cell location in terms of gross margins. In practice, this means calculating the difference between potential revenue and production costs, on a per hectare basis.

_images/FutScen_Fig3.jpg

Potential revenue at each grid cell may be derived by multiplying potential yield estimates with farm gate prices. Prices can either be assumed constant across space or made spatially explicit. The latter can be modelled using the distance decay function.

_images/FutScen_Fig4.jpg

Methodological diagram for identifying potential future cropland based on physical suitability and economic feasibility

Finally, combination of physical and economic suitability layers was used to create a proxy layer for future crop expansion that reflects indicative actions related to productivity gains, environmental resilience or both. This approach allows for various future scenarios around market prices, production costs and agricultural productivity to be explored.

Model run

The methods presented above can be implemented through code available as Making Future Scenarios in the project’s repository.

Output data

The output of the model indicates potential locations of crop extensification in the future. Results are available in raster (.tiff) and can vary based on crop modelled, productivity gain assumptions and environmental resilience as described above. The Figure below shows extensification for maize in Mozambique under four example scenarios namely Sustainable Growth (SG), Environmental Stewardship (ES), Economic Growth (EG) abd Business As Usual (BAU).

_images/FutScen_Fig5.jpg

Potential areas for maize extensification in Mozambique, under the SG, ES, EG and BAU scenarios respectively (from left to the right)

_images/FutScen_Fig6.jpg

Potential areas for maize extensification in Mozambique, under the SG, ES, EG and BAU scenarios respectively (from left to the right)

Results indicating future distribution of crops can be combined with datasets indicating existing crop allocation. The latter can then be prepared and used in the agrodem model in order to estimate water and electricity requirements for the modelled crop in the future.

Special notes

This document presented a pipeline for generating future scenarios in regards to “extensification”. That is, identifying locations for new crop areas based on policy mandates.The supporting code can be used to generate several variations of the above.

Note however that the methodology presented above is bound to certain limitations. For example, we assume 100 ha (full grid coverage for 1km cells) of harvested area in new locations. Also, these may overlap between different crops as we only model one crop at a time. Future work might want to further explore those assumptions.

Post-harvesting (& post-analysis) module

Overview

In this exercise spatial modelling of post-harvest processes was limited to on-farm primary post-harvest processing. This is because these are most likely to take place in areas without access to the grid. A review of available literature and survey data indicated the following findings:

  • Electricity requirements for post-harvesting activities can be classified into two groups; post harvest and primary processing (drying, milling, chilling etc.) and secondary processing (packaging, bottling etc.). Note that primary processing is more akin to happen close to farms (in rural areas) than secondary processing.
  • Heating and drying is commonly applied to harvested crops to reduce the moisture content and increase shelf life (e.g. grains, fruits and pulses). The electricity demand is approximately 6 kWh per metric tonne of processed crop.
  • Milling is another common value chain component across most agricultural products, used to produce flour. Small mills (electric or diesel motors) have a power demand of ~33 kW per tonne of crop milled. Larger (commercial) mills may consume ~15-26 kW per ton ne milled.
  • Pressing is the process of oil extraction, typically applied to soybean, sunflower, sesame, palm oil and ground nuts. Power demand ranges between 10-23kW per tonne of crop pressed.
  • Cold storage is amongst the most electricity intensive processes in the agriculture value chain, especially in warm climate countries. Electricity requirements depend a lot on size of cooling facility and target temperature. Small evaporation cooling storage room (20 m2) may require ~0.9 kWh/tonne, while refrigerated cold room (80 m2) ~30-50 kWh/tonne.
  • Packaging takes place when preparing harvests for transport between the fields and storage facilities, and from storage facilities to retail facilities. In the context of Sub-Saharan Africa, this is mostly done by hand, without significant electricity requirements.

Note

Note that some agricultural products (cassava, meat processing, dairy products, sugarcane etc.) have specific post-harvest processing requirements, thus additional electricity needs. A more elaborate description of those is available in the project report.

Parameterization & model run

Modelling electricity demand from post-harvest activities in this project consisted of the following steps:

  • Step 1: Identifying the post-harvest technologies in use in the specific country. The type of technology used depends on country specific practices and technological resources. The specific technologies should be identified as the electricity demand profile varies according to technology.
  • Step 2: Estimating the volume (in terms of tonnes) of agricultural products being generated in the area of interest By identifying the types of crops being produces, this will narrow down the potential post-harvesting processes and their corresponding electricity demand.
  • Step 3: Applying the technology usage profile to the crop production profile. The final step brings together insight gathered from steps two and three to generate a country specific profile of electricity demand from post-harvest processes. Note that estimating the proportion (in terms of tonnes) of each crop that is processed as supposed to sold directly after harvest is also possible. This proportion can vary by crop and country, depending on the extent to which products are processed or stored in farm.
_images/Post_Fig1.jpg

Methodological flow of modelling post-harvesting activities

In this exercise, inputs in the modelling of post harvesting activities involved electricity requirements for the different activities identified as shown below.

_images/Post_Fig2.jpg

Common post-harvesting activities for the main crops in Mozambique

Note

Modelling post-harvesting activities is implemented through Python script Result post analysis that was developed as part of an overal post analysis of irrigation model results.

Output data

The output indicates electricity requirements for post-harvesting activities of the selected crop and AoI. Results are available at the resolution of crop data but can also be aggregated by administrative area as shown below.

_images/Post_Fig3.jpg

Common post-harvesting activities for the main crops in Mozambique

Special notes

Introducing electricity to post-harvesting activities can increase efficiency and productivity. We have developed an approach, in which electricity requirements for such activities can be spatially identified and quantified.

However,

  • At this stage, production is estimated based on two yield ratios (current and max potential) that do not vary per location. This could be implemented in our modelling approach, if such data become available in the future.
  • Many parameters in the modelling exercise require expert judgement to set. That may include, technologies used, common practices, policies etc. That is, model input parameters should be decided with caution and under the consultation of local agriculture/energy experts.
  • It is very difficult to decide in which areas post-harvesting activities will take place. Thus, this exercise can only be used for a quick, screening analysis of estimated power requirements for such activities that combined with irrigation requirement can help delineate the spatial condors of electrification for agro-productive uses.

Credit and Contact

This work has been the result of collaboration between KTH Royal Institute of Technology and Vivid Economics and has been funded by the World Bank.

Contributions are described below:

For any questions, feedback or general inquiries please to not hesitate to contact the development team.