natcap / urban-online-workflow

This repository hosts the beta implementation of the Urban Online ES Workflow. The project is intended to give urban planners the ability to create and assess scenarios using InVEST Urban models.
1 stars 5 forks source link

displaying small geotiffs in OpenLayers #66

Closed davemfish closed 1 year ago

davemfish commented 1 year ago

Thanks to #65 , we now have rasters created by wallpapering & parcel fill that we can display on the openlayers map. But I'm encountering this problem when loading these data sources in the same way we do the nationwide LULC -- that is as a ol/source/GeoTIFF as the source of a ol/layer/WebGLTile.

Here's how the failed fetch looks in devtools.

General
---------
Request URL: http://localhost:3000/scenarios/1/1_parcel_fill.tif
Request Method: GET
Status Code: 416 Range Not Satisfiable
Remote Address: [::1]:3000
Referrer Policy: strict-origin-when-cross-origin

Response Headers
--------------------
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Range: bytes */3356
Date: Wed, 21 Sep 2022 21:38:19 GMT
Keep-Alive: timeout=5

Request Headers
-------------------
Accept: */*
Accept-Encoding: identity
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost:3000
Pragma: no-cache
Range: bytes=0-65536
Referer: http://localhost:3000/
sec-ch-ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36

I don't understand why we get a 416 here, since it seems like the range requested (Range: bytes=0-65536) is wider than the full size of the file (Content-Range: bytes */3356). And what I've read suggests the intersection of the byte range should be returned. https://httpwg.org/specs/rfc9110.html#byte.ranges

A client can limit the number of bytes requested without knowing the size of the selected representation. If the last-pos value is absent, or if the value is greater than or equal to the current length of the representation data, the byte range is interpreted as the remainder of the representation (i.e., the server replaces the value of last-pos with a value that is one less than the current length of the selected representation).

However we can modify the Headers to request a specific range, and when I do that with a range <= to the length of the file (e.g. Range: bytes=0-3355) the fetch completes successfully. So technically a solution could be to inform the client of the length of the file beforehand, but it really seems that shouldn't be necessary.

Another option: Since these rasters tend to be fairly small, it could make sense to always fetch the complete file and load it as an ol/layer/Image rather than a TileLayer with GeoTIFF source. But I think that would require serving an RGB image rather than the singleband LULC.

dcdenu4 commented 1 year ago

But I think that would require serving an RGB image rather than the singleband LULC.

This is what I had to do for the global web viewer, though because of a different issue with Mapbox limitations. I found it a bit tedious to have to generate the RGB images and track those as well. Hopefully we can work this out so we don't have to do something similar.

davemfish commented 1 year ago

The server returning the 416 here is the vite dev server. And it may not be in compliance with the byte-range spec. So this may not be an issue in production, but it's still an issue now nonetheless.

phargogh commented 1 year ago

I added an nginx server to serve static files (and a bonus browseable index) at localhost:9000 in 338732c, which appears to work fine for me locally! But we will have to adjust URLs to point to a new port for those files.

davemfish commented 1 year ago

The new server is working great. I think we can close this issue.