Fossil Fuel CO₂ Emissions analysis causes 500 on /statistics

[j08lue]

For example in this analysis:

https://earth.gov/ghgcenter/analysis/results?start=2021-01-01T00%3A00%3A00.000Z&end=2022-12-31T00%3A00%3A00.000Z&datasetsLayers=co2-emissions&aoi=~fsia%40%3F%3F_ye~O_ol%7BU%3F%3F~xe~O

an example request causing a 500 when called from the frontend app is a POST request to https://ghg.center/api/raster/cog/statistics?url=s3://ghgc-data-store/odiac-ffco2-monthgrid-v2022/odiac2022_1km_excl_intl_202105.tif

Please note: understanding and documenting this issue is the most critical part. Fixing it can come after the release, if it shows that we do not have the time before.

cc @anayeaye @hanbyul-here

Acceptance criteria

• [x] Figured out what is causing this
• [x] Opened a ticket to fix it

[anayeaye]

Quick checks in AWS

• No refused database connections
• In raster lambda: Memory Errors like this Unable to allocate 14.3 GiB for an array with shape (106800, 144000) and data type uint8

[anayeaye]

Reproducible error request https://ghg.center/api/raster/cog/statistics?url=s3://ghgc-data-store/odiac-ffco2-monthgrid-v2022/odiac2022_1km_excl_intl_202108.tif payload

{"type":"Feature","properties":{},"geometry":{"type":"MultiPolygon","coordinates":[[[[-180,0],[-180,89],[-60,89],[-60,0],[-180,0]]]]}}

failing request to reproduce memory errors observed in raster-api log above

curl -X 'POST' \
  'https://ghg.center/api/raster/cog/statistics?url=s3://ghgc-data-store/odiac-ffco2-monthgrid-v2022/odiac2022_1km_excl_intl_202108.tif' \
  -H 'accept: application/geo+json' \
  -H 'Content-Type: application/json' \
  -d '{"type":"Feature","properties":{},"geometry":{"type":"MultiPolygon","coordinates":[[[[-180,0],[-180,89],[-60,89],[-60,0],[-180,0]]]]}}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168  100    34  100   134      2      8  0:00:17  0:00:15  0:00:02     8
{
  "detail": "Internal Server Error"
}

[anayeaye]

The search geometry is fine. The dataset is VERY high resolution.

The request works with a much smaller geometry.

curl -X 'POST' \
  'https://ghg.center/api/raster/cog/statistics?url=s3://ghgc-data-store/odiac-ffco2-monthgrid-v2022/odiac2022_1km_excl_intl_202112.tif' \
  -H 'accept: application/geo+json' \  
  -H 'Content-Type: application/json' \
  -d '{"type":"Feature","properties":{},"geometry":{"type":"MultiPolygon","coordinates":[[[[-118.98917,44.80303],[-108.08194,43.97778],[-109.75011,37.16468],[-119.37413,36.85728],[-118.98917,44.80303]]]]}}' | jq

[anayeaye]

Solution

max_size=1024 path parameter (thanks @vincentsarago!)

curl -X 'POST' \
  'https://ghg.center/api/raster/cog/statistics?url=s3://ghgc-data-store/odiac-ffco2-monthgrid-v2022/odiac2022_1km_excl_intl_202108.tif&max_size=1024' \
  -H 'accept: application/geo+json' \  
  -H 'Content-Type: application/json' \
  -d '{"type":"Feature","properties":{},"geometry":{"type":"MultiPolygon","coordinates":[[[[-180,0],[-180,89],[-60,89],[-60,0],[-180,0]]]]}}'

{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[-180.0,0.0],[-180.0,89.0],[-60.0,89.0],[-60.0,0.0],[-180.0,0.0]]]]},"properties":{"statistics":{"b1":{"min":-127.76295471191406,"max":1422.452392578125,"mean":0.9611924020867598 <SNIP>

@hanbyul-here can this parameter be added to the analysis request? I think it could be OK to use for all datasets.

[j08lue]

Opened a ticket to fix this:

• https://github.com/NASA-IMPACT/veda-ui/issues/765

[dzole0311]

To follow-up on this as it looks related, I noticed that choosing Delaware, Arizona, Indiana and a few other presets and running the analysis, the Air-Sea CO₂ Flux dataset errors out with a 500 on the statistics endpoint. It works though if I choose California as a preset.

It can be reproduced here.

cc @anayeaye