search

developmentseed / titiler

Build your own Raster dynamic map tile services
https://developmentseed.org/titiler/
MIT License
774 stars 160 forks source link

Too much data for declared Content-Length #947

Closed RS-Y closed 3 months ago

RS-Y commented 3 months ago

When I use mosaicjson to request a service, the request is very slow when minzoom is small, and When mosaicjson bounds are exceeded it results in a 204 error. Below is the detailed error information

Traceback (most recent call last):
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/etc/miniconda3/envs/titiler/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/responses.py", line 261, in wrap
    await func()
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/responses.py", line 253, in stream_response
    await send({"type": "http.response.body", "body": chunk, "more_body": True})
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 493, in send
    output = self.conn.send(event=h11.Data(data=data))
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/h11/_connection.py", line 512, in send
    data_list = self.send_with_data_passthrough(event)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/h11/_connection.py", line 545, in send_with_data_passthrough
    writer(event, data_list.append)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/h11/_writers.py", line 65, in __call__
    self.send_data(event.data, write)
  File "/etc/miniconda3/envs/titiler/lib/python3.10/site-packages/h11/_writers.py", line 91, in send_data
    raise LocalProtocolError("Too much data for declared Content-Length")
h11._util.LocalProtocolError: Too much data for declared Content-Length

The mosaicjson used is as follows: {"mosaicjson":"0.0.3","version":"1.0.0","minzoom":17,"maxzoom":22,"quadkey_zoom":17,"bounds":[113.26320648193358,22.845014155559195,113.28612327575685,22.86099101151058],"center":[113.27466487884521,22.853002583534888,17],"tiles":{"13212222120030321":["E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_0_0_cog.tif"],"13212222120030330":["E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_0_0_cog.tif"],"13212222120030331":["E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_0_0_cog.tif"],"13212222120031220":["E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_0_0_cog.tif"],"13212222120031221":["E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_0_0_cog.tif","E:\workspace\1\cog\1\0\1_0_b1_b2_b3_tile_1_0_cog.tif"]...........................}}

RS-Y commented 3 months ago

Especially when minzoom is < 10，The service response is quite slow，Then my entire service process will be occupied and stuck

vincentsarago commented 3 months ago

Dear @RS-Y, While we encourage people raising issues, we ask for user to provide more information in order to be able to help you as much as we can.

You're talking about 2 different issues:

When mosaicjson bounds are exceeded it results in a 204 error

This was solved in 0.18.1, https://github.com/developmentseed/titiler/blob/main/CHANGES.md#0181-2024-04-12

When I use mosaicjson to request a service, the request is very slow when minzoom is small

We do need to make a better documentation for this. You can read https://github.com/developmentseed/titiler/issues/291 first.

The issue, is that when requesting low zoom level, the tiler will have to open/read numerous files. Some files might represent maybe only 1pixel on the final tile... meaning that it's almost useless.

We usually recommend to set fetch tiles starting for the mosaic minzoom. We also have a settings to return 400 is the zoom requested is not between min/max zoom

https://github.com/developmentseed/titiler/blob/a03de921c8950dc6b36e57f1fdd59db29b45b475/src/titiler/mosaic/titiler/mosaic/factory.py#L299-L318