nasaharvest / crop-mask

End-to-end workflow for generating high resolution cropland maps
Apache License 2.0
95 stars 28 forks source link

Added Malawi corrective labels 2020 #270

Closed MsPixels closed 1 year ago

ivanzvonkov commented 1 year ago

Oh one other thing, the class needs to be added to the datasets list here: https://github.com/nasaharvest/crop-mask/blob/77b581b2ec2a4a4b366f4648be57488761bda84b/datasets.py#L885

ivanzvonkov commented 1 year ago

@MsPixels if you look at the test Github action, you'll see:

FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/crop-mask/crop-mask/data/raw/Malawi_corrective_labels_2020/Malawi_corrective_labels_2020.csv'

Do you know how to fix this?

MsPixels commented 1 year ago

Kinda, so I tried dvc add, commit and push and I updated the PR. And then, I got this error... https://github.com/nasaharvest/crop-mask/actions/runs/4028016506/jobs/6924418829#step:7:332

ivanzvonkov commented 1 year ago

Alright let's see if this runs now...

ivanzvonkov commented 1 year ago
image

This failure makes me think that dvc commit was run but dvc push was not run or failed. Could you run dvc push again locally and then we can rerun the pipeline?

MsPixels commented 1 year ago

I tried dvc push and I got this error again image

ivanzvonkov commented 1 year ago

I tried dvc push and I got this error again image

What if you run gcloud auth application-default login again to relogin, you should have the additional permissions now.

MsPixels commented 1 year ago

I got this after running gcloud auth image

ivanzvonkov commented 1 year ago

Earth Engine export working now!

ivanzvonkov commented 1 year ago

Ok I took a second look at this PR, my explanation earlier this morning was not fully correct. There's a duplicate but it's across two datasets Malawi_FAO_corrected and MalawiCorrectiveLabels2020. The points are not identical but they get matched up to the same point:

image

I am thinking we can merge this PR and figure out how to deal with this in the future.

MsPixels commented 1 year ago

Alright Ivan. Thank you!