Inconstistent response from /statistics endpoint leads to analysis charts not showing up

[danielfdsilva]

The app expects the statistics information used in the analysis to be present under a specific key (1), but some datatsets (eg: Wetlands Ch4 Emissions, CH4 plumes emissions) use a different key.

• Can this key be standardized?
• Is there only ever 1 key and we can use that?
• Is there a way to know what key should be used?

cc @slesaad @j08lue @smohiudd

[anayeaye]

@danielfdsilva re #571

the /statistics response returns data under a b1 key, and we're expecting a 1 key (This is so in VEDA). Is this possible to standardize?

Do you mean the difference between the array of full raster band statistics in the STAC item metadata vs the keys returned in the statics result? I found some multi band data to look at how bands are referenced the statistics and item metadata:

STAC Item asset raster:bands (i.e. band 1 = raster_bands[0]) This is stored in the database and is generated by rasterio create_item. example: https://dev-stac.delta-backend.com/collections/blue-tarp-planetscope/items/70130_raw_2022-02-12

Versus the band name band 1 = b1 in the statistics endpoint result? I think this is the default band name from the dataset reader and b1, b2, b3... is expected. example: https://dev-raster.delta-backend.com/cog/statistics?url=s3://veda-data-store-staging/blue-tarp-planetscope/70130_raw_2022-02-12.tif

If I understand you right, this is the expected behavior--so for a raster with one band you will always have b1, in some stray cases you may see the other band names. Failed to submit this yesterday afternoon on the discussion in #571 but I think it fits here now.

[danielfdsilva]

@anayeaye Thank you for the explanation. In the case of no2 for example (https://staging-stac.delta-backend.com/collections/no2-monthly/items/OMI_trno2_0.10x0.10_202210_Col3_V4.nc) we only have 1 band, so I'd expect the statistics to reply with b1 but we're getting a 1 (https://staging-raster.delta-backend.com/cog/statistics?url=s3://veda-data-store-staging/no2-monthly/OMI_trno2_0.10x0.10_202210_Col3_V4.nc.tif)

Any idea what is going on? Could this also become b1 so we standardize?

Btw, this also happens with other datasets

[anayeaye]

@danielfdsilva thanks for the example! That is not what I expected. So this may be something in the tif. We can look into that--sorry to have assumed that the b<number> band naming was universal!

[vincentsarago]

by default band names in statistics should be b1... but in older rio-tiler/titiler it may have been 1

This was changed in rio-tiler 4.0 https://github.com/cogeotiff/rio-tiler/blob/main/CHANGES.md#400a0-2022-10-20

[anayeaye]

Thanks @vincentsarago! @danielfdsilva we've already upgraded the develop backend and the band names are consistent there. We're planning on deploying that update to staging on Monday.

https://dev-raster.delta-backend.com/cog/statistics?url=s3://veda-data-store-staging/no2-monthly/OMI_trno2_0.10x0.10_202210_Col3_V4.nc.tif

[slesaad]

yep, we're using the upgraded titiler in ghgc, so we're getting b1 there.

once VEDA upgrades too (planned for monday), we'll consistently get b1 there too. we should plan for the upgrade in the frontend and backend simultaneously so it doesn't affect the users using the dashboard analysis.

@danielfdsilva we're planning to do it on monday, do you think ui push to production with this change can happen simultaneously? otherwise, we should hold off on the backend upgrade until the frontend is ready too. @anayeaye

[anayeaye]

There is now a work around to support both b1 and 1: https://github.com/NASA-IMPACT/veda-ui/commit/904bf0e092e8a8f491640068edf124a10ade46b8 so we are safe to release the backend change