NASA-IMPACT / veda-config

Configuration template for the Dashboard Evolution Project
https://visex.netlify.app/
Other
18 stars 37 forks source link

feat: migrate to production veda stac catalog and titiler #414

Closed anayeaye closed 4 months ago

anayeaye commented 4 months ago

Why are you creating this Pull Request?

To migrate the Dashboard to new openveda.cloud VEDA instance.

How tested

Steps taken to validate the new production VEDA instance are documented here.

netlify[bot] commented 4 months ago

Deploy Preview for visex ready!

Name Link
Latest commit aaa167281beeec7cbb4699b046ef478de800a218
Latest deploy log https://app.netlify.com/sites/visex/deploys/66830ac24c9c330008f12dc1
Deploy Preview https://deploy-preview-414--visex.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

hanbyul-here commented 4 months ago

Thanks for the PR and all the work put into this. @anayeaye 🙇 - If possible, I would like to know what the service URL change means for the dashboard. Is this the exact same service from a different domain? Did openveda.cloud instance introduce any change that the dashboard needs to be aware of - if there are, can you point me those?

I think I am currently unsure what should be reviewed in the PR. Should it mainly check that all the datasets get rendered and get analysis results as before? I would appreciate it if you can give me the pointers 🙇 🙇

sandrahoang686 commented 4 months ago

The endpoint updates are really just a change of pointing to where the data lives now so we should make sure any datasets that may have recently been merged are in the prod DB/endpoints as well. @anayeaye am I missing anything?

For config instance, lets make sure E&A pages works the same as before this change. Our benchmark should be the same behavior as before these endpoint changes.

anayeaye commented 4 months ago

@hanbyul-here @sandrahoang686 sorry for the delay in reply but I think it is all cleared up already

mainly check that all the datasets get rendered and get analysis results as before

Exactly--we think that the dashboard should not experience any change when transitioned to the new stable production URLs. Behind the scenes we copied the data from veda-data-store-staging to veda-data-store and generated new STAC records.

hanbyul-here commented 4 months ago

I did not test every dataset - Based on some radom selections, I think the app is behaving in the same way with the new serviceUrl. (To note, this doesn't. mean that all things are working

ex. https://deploy-preview-414--visex.netlify.app/analysis/results?start=2018-01-01T00%3A00%3A00.000Z&end=2022-12-31T00%3A00%3A00.000Z&datasetsLayers=grdi-filled-missing-values-count&aoi=gsia%40mizlCsthgIqx%60yA%3Fdmf~Dv_hxHzoqeB

This analysis fails, but so does on the old serviceUrl :

https://www.earthdata.nasa.gov/dashboard/analysis/results?start=2018-01-01T00%3A00%3A00.000Z&end=2022-12-31T00%3A00%3A00.000Z&datasetsLayers=grdi-filled-missing-values-count&aoi=gsia%40mizlCsthgIqx%60yA%3Fdmf~Dv_hxHzoqeB)

I will leave a bit more thorough QA to @aboydnw 🙇

aboydnw commented 4 months ago

I went through most of the datasets and didn't notice any issues. Facebook population density doesn't load but it also doesn't load for me in production, so I'm guessing it is a separate issue.

I did notice something else weird where in the Data Analysis tab there are duplicates for the MTBS Burn Severity layer. Clicking on one of these layers checks/unchecks all 4 here. In reality, there is only one layer for the Disalexi ET Suppression dataset shown here, but there are multiple MTBS Burn Severity layers, across different datasets. I wonder if there is something with the statistices endpoint that doesn't handle layers with identical names very well? It might not matter for this page since we are replacing it with the new E&A page very soon, but I wanted to call it out in case it is in face something with the statistics endpoint itself that could lead to problems down the road. image

anayeaye commented 4 months ago

Thanks for the QC, @aboydnw! It looks like the facebook population density dataset is working but perhaps another colormap would be preferable and/or perhaps a min zoom? After seeing no errors I zoomed in and found that it is just hard to see the layer. Here's a working tile https://openveda.cloud/api/raster/mosaic/ba15348a0b850d46b5dc8b3838b8c45a/tiles/WebMercatorQuad/8/141/89?assets=cog_default&resampling=bilinear&bidx=1&colormap_name=ylorrd&rescale=0%2C69

aboydnw commented 4 months ago

oh wow, you're totally right 😆 thanks for double checking!