Using containers to analyse the impact of climate change
C2 | Thu 24 Jan | 11:35 a.m.–12:20 p.m.
Hello there! I am an IT professional specialising in GNU/Linux and Open Source. I also have a Law degree, but computers are my true passion. I have approximate knowledge of many things. I am a science enthusiast and an aspiring polyglot (in both human and computer lingos). I use Oxford commas and indent my code with spaces 😄. When I take a break from being a geek, I enjoy discovering the world and other cultures, particularly if there's food, wine, or beer involved.
Predicting crop productivity and environmental outcomes of agricultural systems requires modelling quantitative data around soil characteristics (e.g.: nutrients and soil water dynamics) and climate information. Traditionally, the scientific research community runs these simulations using standard desktops. When the simulations need to be done at scale (ie: wide-area, or landscape scale), detailed quantitative soil information is not always available. This leads to generalisations and simplifications, which may affect the resulting estimates in ways unknown. However, when there is detailed information, the compute capacity required to analyse that data grows beyond what a standard desktop can handle within the average human lifespan. In this talk, I will share our experiences creating a new data processing pipeline using software containers (Docker and Singularity), that allows processing of vast amounts of high-resolution soil and weather data in a scalable way. This system has already been used in a case study to simulate productivity and environmental aspects of a multi-year crop rotation system across a catchment in the Hawke’s Bay region of New Zealand, at a 5 arc-min grid-cell resolution (~5 km grid size). Linux Australia: http://mirror.linux.org.au/pub/linux.conf.au/2019/c2/Thursday/Using_containers_to_analyse_the_impact_of_climate_change_and_soil_on_New_Zealand_crops.webm YouTube: https://www.youtube.com/watch?v=R9EWTddl4KM