CROP is a Research Observations Platform designed and created by the Research Engineering Group at the Alan Turing Institute in collaboration with Dr Ruchi Choudhary's research group and Growing Underground.
The aim of CROP is to prototype a digital twin of the Growing Underground underground farm.
CROP is an on cloud-based application which utilizes the flexibility, customization and evolution that a cloud-native system provides, to better refine, simplify and improve the processes and architecture of the system with regards to our research needs.
The digital twin:
The Unity 3D model is found in this repo
CROP is implemented using a well established software stack (given below) and exploits four different services on the Azure cloud computing platform.
Platform | Service | Software |
---|---|---|
Function | ||
WebApp | ||
Storage | ||
PostgreSQL |
The directories are structured as follows:
core
has all the code for interacting with the database.
This includes defining the database schema in structure.py
, some utilities, and modules for each of the ingress functions, for reading and writing the data from various sensors and other data sources.tests
has the test suite for core
.webapp
has the Flask webapp.ingress_functions
has the configurations and code for the Azure functions for importing data into the database.
The code itself for each function is nearly trivial:
It just calls the corresponding function in core
.
This folder mainly exists to hold the host.json
and function.json
files for each function.models
has the code for the forecasting models, ARIMA and GES.media
has illustrations, logos, etc.util_scripts
has various utilities that don't fit the other categories..secrets
has shell scripts for setting environment variables with secrets, such as database passwords, to facilitate running a local copy of the webapp for development use.
The version-controlled files are merely templates, to be filled in with the actual values for the secrets.All of tests
, webapp
, ingress_functions
, util_scripts
, and models
import and use various parts of core
. None of them import from or talk to each other.
The repository root also has three Docker files:
Dockerfile_ingress_functions
builds a container that holds all the functions in ingress_functions
, in an environment to run them on an Azure function app.Dockerfile_models_functions
builds a similar container that holds the functions in models
, for running the predictive models on an Azure function app.Dockerfile_webapp
builds the webapp, ready to be deployed as an Azure app service. It builds on webappbase
.Dockerfile_webappbase
is a container that installs some of the dependencies needed by the webapp.
It's separated from webapp
to improve build times, and shouldn't need to be rebuilt except in rare circumstances.Some of the subfolders have their own READMEs. See them for more details of each part.
Gentelella - a free to use (MIT license) Bootstrap admin template on which the webapp is built.
We employ a continuous delivery toolchain using Github Actions which deploys the latest version of CROP when a push or a PR is made to the main
or develop
branches.
The Github Actions
The Azure services for the webapp and the ingress functions listen to updates on Docker Hub, and deploy the latest container once it has been pushed to the Hub.
The main
and develop
branches are deployed to production and testing versions of the platform, respectively. The former is relatively stable, the latter may be broken in the course of development.
A complete set of infrastructure for a CROP deployment on Microsoft Azure cloud can bet setup using Pulumi.
For instructions on how to do this, see here
When developing the code for the CROP platform, it may be useful/necessary to run the webapp locally. To do this:
pip install .
. This will install the contents of the core/
directory (which contains database-related schema and code, and many of the queries) as a python package called cropcore
. It will also install all the dependencies listed in requirements.txt
..secrets/crop.sh
. There is a template file called .secrets/template_crop.sh
where you can fill in the info as you have it (then copy to crop.sh
), or you can ask for a copy of this file from an existing developer.source .secrets/crop.sh
.npm
(node package manager) installed. To install this on MacOS, you can do brew install npm
.webapp/
directory, and run the command ./run.sh
. You should then be able to navigate to the webapp by pointing your browser to http://localhost:5000
.If you found a bug or need support, please submit an issue here.
We welcome contributions! If you are willing to propose new features or have bug fixes to contribute, please submit a pull request here if you have permissions, or fork the repository, make your changes, and submit a pull request from your fork.