Failure while fetching extensions.

[annieherram]

Hey, I'm running the default docker-compose vscsync service (using SYNCARGS=--syncall --skip-binaries) and there's a failure while fetching the cpp tools extension that crashes the whole fetch.

docker-compose run vscsync
[I 220705 13:10:32 sync:534] Syncing VS Code Specified Extensions
[I 220705 13:10:32 sync:543] Searching for VS Code Extension: *
[I 220705 13:11:12 sync:545] Found 37215 extensions
[I 220705 13:11:12 sync:558] Syncing VS Code Recommended Extensions
[I 220705 13:11:40 sync:569] Checking and Downloading Updates for 37215 Extensions
[I 220705 13:11:40 sync:574] Progress 0/37215 (0.0%)
[W 220705 13:11:41 sync:204] _get_asset_types(). More than one version returned for ms-vscode.cpptools. Unhandled.
Traceback (most recent call last):
  File "/opt/vscoffline/sync.py", line 575, in <module>
    extensions[identity].download_assets(config.artifactdir_extensions)
  File "/opt/vscoffline/sync.py", line 148, in download_assets
    for availableasset in availableassets:
TypeError: 'NoneType' object is not iterable

Update me if you need any more info about this. Thank you!

[Ebsan]

I'm also starting to see connection errors when trying to syncall. The docker image fails and exits at different percentage completions. Is Microsoft now throttling the sync more?

urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[Ebsan]

This could be due to how sessions are used. A session from requests is stored and used throughout the run. I changed the logic to use request.get() or requests.post() directly and seem to have gotten rid of this issue. I'm just not sure this is the right thing to do. Does it create a new session every time? And if so will it cause Microsoft to throttle connections when we try to sync extensions?

[Ebsan]

Sorry, I misread the original issue. However, @annieherram this issue should have been fixed by the latest version (1.0.20). Could you try deleting your local docker image and then pull the latest image and see if it's fixed?

docker images
# copy down the IMAGE ID of lolinternet/vsync:latest

docker rmi <IMAGE_ID_FROM_ABOVE>

# now run again to pull in the latest
docker-compose run vscsync

[tomer953]

Removing the session while making requests to the same host can lead to a low performance, did you check that? The easiest way is probably look at the time for N requests and compare the results.

https://www.geeksforgeeks.org/session-objects-python-requests/

also, maybe a basic retry mechanism could help (and then make a fallback to a new request without the session?).

In the end, its all about performance and stability.