search

AdaCore / aws

AWS is a complete framework to develop Web based applications in Ada.
Other
127 stars 37 forks source link

Storage_Error when using [AWS.Client.Get return Response.Data] on musl Linux #365

Open andrewathalye opened 1 year ago

andrewathalye commented 1 year ago

As the title indicates, when I try to download relatively large files on musl Linux a Storage_Error is propagated. I believe this is due to musl having a limited stack size (by default, 80KB), however even when this is increased the error still occurs, just after more time. After looking at aws-client.adb, it appears that the full response body is being cached in a stack variable, which could be part of the issue.

I tried to use AWS.Client.HTTP_Utils instead, and this did not cause a Storage_Error, however I get "Socket closed by peer" instead, which is also unexpected, since I’m able to download the file in a web browser and via curl. This could also be a symptom of the reduced stack size, so I will do further testing with a manually increased stack size.

andrewathalye commented 1 year ago

I can’t provide a useful backtrace at the moment because GCC / GNAT on musl doesn’t support backtraces, however if necessary I can try to get libbacktrace working. This should reproduce in an Alpine container with the latest Alire. Building AWS for musl requires changing the alire.toml to point to "x86_64-alpine-linux-musl" instead of "x86_64-pc-linux-gnu", however it otherwise works fine.

andrewathalye commented 1 year ago

Increasing stack size to 8MB does not solve the "Socket closed by peer" which occurs with HTTP_Utils.

Edit: for clarification "small" files up to ~10MB seem to work fine, but my target file, which is a 400MB JSON manifest, causes crashes. The actual file content shouldn’t matter too much, and I can test with a different file if you’d like, but it should be a universal issue.

andrewathalye commented 1 year ago

Update: I was able to get Client.HTTP_Utils.Send_Request to work for the same file, it looks like my usage of the other subprogram was incorrect. Client.Get still errors with the same exception.