Using containers to analyse the impact of climate change and soil on New Zealand crops
Predicting crop productivity and environmental outcomes of agricultural systems requires modelling quantitative data around soil characteristics (e.g.: nutrients and soil water dynamics) and climate information. Traditionally, the scientific research community runs these simulations using standard desktops. When the simulations need to be done at scale (ie: wide-area, or landscape scale), detailed quantitative soil information is not always available. This leads to generalisations and simplifications, which may affect the resulting estimates in ways unknown. However, when there is detailed information, the compute capacity required to analyse that data grows beyond what a standard desktop can handle within the average human lifespan. In this talk, I will share our experiences creating a new data processing pipeline using software containers (Docker and Singularity), that allows processing of vast amounts of high-resolution soil and weather data in a scalable way. This system has already been used in a case study to simulate productivity and environmental aspects of a multi-year crop rotation system across a catchment in the Hawke’s Bay region of New Zealand, at a 5 arc-min grid-cell resolution (~5 km grid size).