[SYNPY-1356] Refactor sync to synapse

[BryanFauble]

Problem:

1. The syncToSynapse function provided in the utility package had it's own multi-threaded implementation to allow the upload of files in parallel. Using this it had a shared thread pool executor with the file part upload as well.

Solution:

1. This PR contains the changes that were independently reviewed in these 2 PRs:
2. https://github.com/Sage-Bionetworks/synapsePythonClient/pull/1084
3. https://github.com/Sage-Bionetworks/synapsePythonClient/pull/1089
4. Conversion of the multi-threaded portion of the syncToSynapse code to instead rely on asyncIO tasks/futures - Meaning all of the code is going to be executed in a single thread and only down the line will parts of files be uploaded in parallel on different threads.

Testing:

1. Planning to use the benchmark upload script in addition to integration testing around this logic.

[pep8speaks]

Hello @BryanFauble! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file synapseutils/monitor.py:

Line 149:89: E501 line too long (89 > 88 characters)

• In the file synapseutils/sync.py:

Line 710:89: E501 line too long (89 > 88 characters) Line 1193:89: E501 line too long (91 > 88 characters)

• In the file tests/integration/synapseutils/test_synapseutils_sync.py:

Line 206:89: E501 line too long (117 > 88 characters)

Comment last updated at 2024-05-14 02:00:36 UTC