adds the updated deployment - new, secrets and pvc
adds caching - to record events as they are received
database storage... atm sqllite but using sqlalchemy orm so can be swapped relatively easily
logic to use the grib processing config to identify what files constitute a complete data set and thus the need to trigger the downstream action of the pipeline