External Crop Calendar (ECC) table

[gnodnooh]

The FDW's crop calendar (start_date, period_date, season_date, etc.) only relies on "season_name" and does not differentiate between fnid, crop_production_system, product, etc., which is unrealistic. The current HarvestStat processing only expands it with "planting_month" and "harvest_month" manually.

The external crop calendar feature allows you to load/apply data from other crop calendar sources, such as Gary's Global Crop Calendar Compilation or the FAO's crop calendar.

• AF example; USDA
• YE expample; FAO.

Consideration:

• Should deal with year-off issue
• Gary's GCCC also has multiple sources and variables. We need to narrow our approach.

[gnodnooh]

We adopted an external crop calendar: https://docs.google.com/spreadsheets/d/1ZBk6KKJtgaVIMktVJibPfmK79m8bSAEvgl7nlY8fDmc/edit#gid=0

• In this spreadsheet, we manually specify planting_month, harvest_month, planting_year, and harvest_year per [country, season_name, product, crop_production_system].
• The planting_year and harvest_year are extracted from season_date and start_date in FDW, respectively. Those FDW dates have year-off issues and can be updated in the future. Here, our "planting_year" and "harvest_year" mean an offset value, so 0 means the year is correct, and -1 means shift the one year earlier.
• Once the spreadsheet is updated, please locate it to /data/crop_calendar/external_crop_calendar.csv.

Once we make "stack", the below code will create draft of ECC:

stack[['country','season_name','product','crop_production_system','planting_month','harvest_month']].drop_duplicates().sort_values(['country','season_name','product','crop_production_system']).reset_index(drop=True)

Here is a procedure:

1. Extract a crop calendar table from the "stack" dataframe.
2. Add/Update (1) to the Global ECC (Google Sheets).
3. Download (2) and overwrite the local ECC CSV file.
4. If a year-off issue is found in validation, we correct/download the Global ECC again (steps 2-3).