search

irods / irods_capability_automated_ingest

Other
12 stars 16 forks source link

`PUT`-like operations create extra connections which never clean up #276

Closed alanking closed 3 weeks ago

alanking commented 1 month ago

We believe this is related to https://github.com/irods/python-irodsclient/issues/619.

This issue can be mitigated by specifying allow_redirect=False in the **options map passed to the call to PRC put(). However, this may negatively affect performance for file uploads as a direct connection with (i.e. a redirect to) the data's host will not be established as a direct result of that option.

This issue was created in order to implement the mitigation technique should we decide not to increase the minimum PRC version required for use with this application.

alanking commented 1 month ago

I wonder if the performance hit would only really affect large file uploads. When uploading many small files (arguably the main use case of this tool), establishing a new connection from the client side seems like it might be worse than just hopping through the server redirects, especially when you consider that the server-to-server connections are maintained inside the agent and may be reused.

alanking commented 3 weeks ago

allow_redirect is now explicitly set to False for all data-uploading operations.