While working on my data validation script, the script picked up that just_harvest.csv does not have some columns which are specified in the Schema. Specifically, missing "WIC|FMNP | fresh_produce | food_bucks | free_distribution". My data validation script is assuming that if a file located in the "Cleaned_data_files" directory is missing columns listed in the schema, that file is in an improper format and the Github Action will stop.
If that's an assumption we want to make, then just_harvest.csv and/or its prep script needs to be amended to include those columns. If we don't want to make that assumption, then we'll just need to talk about what the requirements are, specifically.
While working on my data validation script, the script picked up that just_harvest.csv does not have some columns which are specified in the Schema. Specifically, missing "WIC|FMNP | fresh_produce | food_bucks | free_distribution". My data validation script is assuming that if a file located in the "Cleaned_data_files" directory is missing columns listed in the schema, that file is in an improper format and the Github Action will stop.
If that's an assumption we want to make, then just_harvest.csv and/or its prep script needs to be amended to include those columns. If we don't want to make that assumption, then we'll just need to talk about what the requirements are, specifically.