precision-sustainable-ag / SemiF-DataReporting

This repository provides detailed reports on Semifield image data, including data contents, species distribution, temporal and spatial distribution, missing data analysis, and status of unprocessed or backlog data.
MIT License
0 stars 0 forks source link

Refactoring and Reimplementation of semif data reporting #23

Open mkutu opened 1 week ago

mkutu commented 1 week ago

What should it be able to do?

  1. Provide understanding of what we have in terms of semifield data
  2. Look at multiple versions of benchbot data (V2, V3.0, and V3.1)
  3. weekly output to slack

What capabilities must be in place to enable the desired output?

  1. read and output contents of azure blob containers and mounted NFS storage
  2. provide statistics about preprocessed and processed data volumne, storage size, and distribution across various metadata fields

Flow chart of data movement from collection across bbot platforms, to preprocessing, and to processing, froma and to various data locations Image