search

qAIRa / qAIRaMap-OpenSource

Website for air quality monitoring
https://qairamap-test-opensource.qairadrones.com
GNU General Public License v2.0
4 stars 23 forks source link

Link Copernicus Sentinel-5p data to the website #115

Open SabCampos opened 2 years ago

SabCampos commented 2 years ago

Access to the data of: https://scihub.copernicus.eu/ in order to compare the information on google maps.

Vivihung commented 2 years ago

@SabCampos , I expect to work on qAIRaMap as part of vGHC 21 tomorrow. Is this task to bring data from https://scihub.copernicus.eu/ and rendered it in the Google map of qAIRaMap website? If so, should it overlap with existing map (QHAWAX, Drone, Movil) or we would like to create a 4th map for Copernicus Sentinel-5p? Do we expect a live request or there should be cache in between?

SabCampos commented 2 years ago

Hello @Vivihung, very nice to meet you! So, the main idea is to get to show the air quality data from the Sentinel-5p satellite as layers on the qHAWAX map. There is no need of cache in between.

Vivihung commented 2 years ago

Thanks for the quick response. I would like to work on this task over the weekend. Will give an update early next week if it’s okay.

developer22-university commented 2 years ago

@SabCampos Sorry to ask here. But i couldn't reach out anywhwere. I need slack invite link. Please share with me.

SabCampos commented 2 years ago

https://join.slack.com/t/opensourceday-ghc2021/shared_invite/zt-wdyub2eb-9K_j3ajUAhdkYgWwX5twkw

Vivihung commented 2 years ago

Couple things to consider. According to copernicus's documents, to access the api we'll have to do self-registration. Then the api request must provide a credential. There is quota limitation: two parallel downloads per user.

  1. If it's with credential, it's better to hide it in the api layer (which I believe isn't in current project)
  2. If we use the credential as service principle. Your users might be throttle since every qAIRaMap users shared the same username.

Not sure how we store secret in api. One common approach is to save it in service/system config on the web host. Depends on what host you used, if it's from cloud providers (Amazon, Google, Microsoft), there might be other options to manage secrets.

Vivihung commented 2 years ago

Another question... Which one of the Sentinel-5P projects have the air quality information we want to display? image

SabCampos commented 2 years ago

image This information :)

Vivihung commented 2 years ago

Share some findings, it would be helpful if any of the contributors could cross check S-5P to validate them, in case I missed anything.

  1. I cannot find valid data points in S-5P website. There are projects, but mainly metadata with no metrics that we are interested in.
  2. Here is another example https://s5phub.copernicus.eu/dhus/odata/v1/Products('663bfd01-65f3-41e9-84cb-546105bc7725')?$format=json
  3. Any suggested tool to view NetCDF files? The default download format is .nc, in case my ODATA query has something wrong, would like to view what's inside the .nc file. Their web tool is hard to explore data content... BTW, the official recommendation for NetCDF is crazily long. Tried QGIS but it's a big software that seems overkill...
SabCampos commented 2 years ago

This https://github.com/16EAGLE/getSpatialData seems to help. But that would be like a middle step.