[Python][C++] S3FileSystem with proxy_options is very slow on Windows

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

https://arrow.apache.org/

Apache License 2.0

14.59k stars 3.54k forks source link

[Python][C++] S3FileSystem with proxy_options is very slow on Windows #28317

Open asfimport opened 3 years ago

asfimport commented 3 years ago

When running pytest with pyarrow 4.0.0 on a Windows 10 VM, I noticed that the test_s3_proxy_options test takes an extraordinarily long time to finish. This is mostly because each call to S3FileSystem with proxy_options takes about a minute to finish.

FYI @sahil1105

Reporter: Ian Cook / @ianmcook

_{Note: This issue was originally created as ARROW-12557. Please see the migration documentation for further details.}

asfimport commented 3 years ago

David Li / @lidavidm: FWIW, I think this might be the same as ARROW-12810 - so it may be faster now. (test_s3_proxy_options IIRC stood out to me as a very slow test in that PR as well, but setting the env var described there fixed that.)