Blob not found / ConnectionError while fetching package

[tomer953]

Hi, I started a fresh sync yesterday I spotted (with some compare logic) - that a version of extension contains the following msg inside the file itself:

<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobNotFound</Code><Message>The specified blob does not exist. RequestId:234ba74f-d01e-00af-25be-19712e000000 Time:2022-12-27T06:44:51.3609120Z</Message></Error>

this is the content of the Vsix file, and all the other icons files as well.

it looks like this was a temp version or something that got removed while the script is running, since i got two versions in that folder (and that was a fresh sync)

I suggest to somehow verify the content before write it, since if its an error we want to avoid this version

[tomer953]

It happened again in my daily sync:

this is the extension:

Vue.Volar
0.33.13

all the files inside are contains the above msg...

[tomer953]

and now the whole service crashed because of this extension:

log:

[I 230104 07:53:06 sync:777] Progress 100/1569 (6.4%)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 182, in _new_conn
    % (self.host, self.timeout),
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f860abbbc10>, 'Connection to vue.gallerycdn.vsassets.io timed out. (connect timeout=12)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 499, in send
    timeout=timeout,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 788, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='vue.gallerycdn.vsassets.io', port=443): Max retries exceeded with url: /extensions/vue/volar/1.0.20/1672788967852/Microsoft.VisualStudio.Code.Manifest (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f860abbbc10>, 'Connection to vue.gallerycdn.vsassets.io timed out. (connect timeout=12)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/vscoffline/sync.py", line 779, in <module>
    config.artifactdir_extensions)
  File "/opt/vscoffline/sync.py", line 248, in download_assets
    url, allow_redirects=True, timeout=vsc.TIMEOUT)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 553, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='vue.gallerycdn.vsassets.io', port=443): Max retries exceeded with url: /extensions/vue/volar/1.0.20/1672788967852/Microsoft.VisualStudio.Code.Manifest (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f860abbbc10>, 'Connection to vue.gallerycdn.vsassets.io timed out. (connect timeout=12)'))

[LOLINTERNETZ]

Has this stabalised now? It's super odd that only one extension is impacted. I wonder if it was yanked?

[LOLINTERNETZ]

Thanks @tomer953 . I ended up looking into this more. I think the DNS queries/HTTP connections were being rate limited somewhere as the http calls were re-created for each fetch. But this doesn't make a lot of sense, as why does it get to Vue and stop. Maybe Vue is the perfect spot after x requests within a period of time. Were you running syncall? or defaults?

Anyway, I've switched the implementation to use a session and added retry behaviours. This error has disappeared. And released 1.0.24.

Thanks