search

microsoft / farmvibes-ai

FarmVibes.AI: Multi-Modal GeoSpatial ML Models for Agriculture and Sustainability
https://microsoft.github.io/farmvibes-ai/
MIT License
651 stars 108 forks source link

Facing errors in Land degradation notebook run with different geometry. #154

Closed click2cloud-SanchitG closed 2 months ago

click2cloud-SanchitG commented 4 months ago

In which step did you encounter the bug?

Notebook execution

Are you using a local or a remote (AKS) FarmVibes.AI cluster?

Local cluster

Bug description

Bug: Scenario 1: Running land degradation with default geometry given in the notebook is working fine. Scenario 2: Running land degradation with cutomize geometry, its producing the error with the 9th cell of notebook Below is the 9th cell inputs: 

    request = cimgt.OSM()
    fig, ax = plt.subplots(figsize=(12,10),
                   subplot_kw=dict(projection=request.crs))

    ax.add_image(request, 7)
    clipped[0].plot(ax=ax, alpha=0.9, transform=request.crs, cmap="RdBu")

Output has been provided as an attachment.
[Link of the land degradation notebook](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/land_degradation/land_degradation.ipynb)

Notebook_9th_cell_error.txt

Steps to reproduce the problem

1) Run the notebook with below customize geometry and timerange: POLYGON ((51.26022142627923 25.40282910148663, 51.26612416152082 25.397875753434136, 51.26315666620698 25.39382551187464, 51.26083427857088 25.393184453989292, 51.25770550633936 25.396447985950743, 51.25906023246108 25.401721900157057, 51.26022142627923 25.40282910148663)) time_range = (datetime(2017, 1, 1), datetime(2018, 12, 31))

It would be great if we can able to run the notebook with customized geometry.

rafaspadilha commented 3 months ago

Hello, @click2cloud-SanchitG. Thank you for raising this issue and for the patience in my reply. I was able to run the notebook with the geometry and time range that you provided and found the same issue.

The problem is not with the workflow or notebook, but with the quality of the output Landsat rasters for this geometry and time range. Below are a few random dates and their landsat and NDVI rasters (I've increased the geometry a bit, to improve the visualization):

Landsat: image

NDVI: image

The black/white pixels in Landsat/NDVI rasters show pixels with missing data. This seems to happen in the majority of images. Because of this, the workflow is unable to train the pixel-wise linear trend model. The error that you see happens during plotting the linear trend raster, due to an overflow in the axis definition (because the values in the rasters are very large due to the bad prediction).

To deal with cases like this, a possible workaround is filtering out rasters that have a high percentage of missing data, before feeding it to the NDVI + linear trend ops. You would need to break the landsat_ndvi_trend workflow in two parts: one that downloads Landsat, and another that receives the filtered list of Landsat rasters and perform the remaining tasks of the workflow (NDVI computation + linear trend). In between them, you would filter the bad rasters.

Let me know if you have any doubts or if I can help you with anything else.

rafaspadilha commented 2 months ago

Closing this issue for now. Let us know if you have any doubts.