[2022] Build is broken on Arch for EU/enGB

[andrfgs]

This project doesn't seem to be working anymore, at least it doesn't work under Arch Linux with all the dependencies installed when choosing the EU region and enGB language.

Seems like a problem in the ngdp protocol implementation. Maybe it changed? Might otherwise be the python version. Looking at the keg repository it has't been updated for quite some years so it might be the most likely reason.

Here's the log:

Downloading: 45216b8ef8f4b503051f41f17f530d84: 429 [01:10,  4.63s/]Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 443, in _error_catcher
    yield
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 566, in read
    data = self._fp_read(amt) if not fp_closed else b""
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 532, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
  File "/usr/lib/python3.10/http/client.py", line 481, in read
    s = self._safe_read(self.length)
  File "/usr/lib/python3.10/http/client.py", line 632, in _safe_read
    raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(102400000 bytes read, 12489276 more expected)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/andre/hearthstone-linux/keg/bin/ngdp", line 1048, in <module>
    main()
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/andre/hearthstone-linux/keg/bin/ngdp", line 309, in fetch
    ctx.obj.fetch_versions(versions, blobs, remote_cdn, metadata_only, tags)
  File "/home/andre/hearthstone-linux/keg/bin/ngdp", line 263, in fetch_versions
    item.fetch()
  File "/home/andre/.local/lib/python3.10/site-packages/keg/core/fetcher.py", line 54, in fetch
    temp_path = self.fetcher.local_cdn.write_temp_file(item.read())
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 565, in read
    with self._error_catcher():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 460, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(102400000 bytes read, 12489276 more expected)', IncompleteRead(102400000 bytes read, 12489276 more expected))

[0xf4b1]

Hm this might be a connection error while downloading. Could you try it again? For me at least, I was able to download/update and the game also runs afterwards.

[enricocovili]

Having this problem too. Arch linux, EU regione with enUS locale. Log is almost identical

[0xf4b1]

Sorry, I can not reproduce the problem on my system (Ubuntu 22.04), a fresh install of the game just works fine

[ApatheticSheep]

UPDATE: Seems to have been a problem with one of the Blizzard distribution servers - it works now. But it would have been nice to test against another CDN region when the problem was happening.

I am having a similar issue on endeavourOS (Arch derivative) is there any way to force the US CDN? EDIT: This was working for a long time until I tried to update hearthstone today.

$ ngdp fetch hsb --tags OSX --tags enUS --tags Production
Fetching http://us.patch.battle.net:1119/hsb
Using CDN: eu (available: eu, kr, cn, us, tw)
Downloading: d8c61f5d3583d6cee1b016371f70f4f9: 0 [00:09, ?/s]Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 443, in _error_catcher
    yield09acc905d1e5a03cda8228f4851cfeb: Waiting for data...
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 566, in read
    data = self._fp_read(amt) if not fp_closed else b""
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 532, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
  File "/usr/lib/python3.10/http/client.py", line 481, in read
    s = self._safe_read(self.length)
  File "/usr/lib/python3.10/http/client.py", line 632, in _safe_read
    raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(126877696 bytes read, 75244163 more expected)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ray/git/github/hearthstone-linux/keg/bin/ngdp", line 1048, in <module>
    main()
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/ray/git/github/hearthstone-linux/keg/bin/ngdp", line 309, in fetch
    ctx.obj.fetch_versions(versions, blobs, remote_cdn, metadata_only, tags)
  File "/home/ray/git/github/hearthstone-linux/keg/bin/ngdp", line 263, in fetch_versions
    item.fetch()
  File "/home/ray/git/github/hearthstone-linux/keg/keg/core/fetcher.py", line 54, in fetch
    temp_path = self.fetcher.local_cdn.write_temp_file(item.read())
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 565, in read
    with self._error_catcher():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 460, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(126877696 bytes read, 75244163 more expected)', IncompleteRead(126877696 bytes read, 75244163 more expected))

I tried in a fresh venv and had the same result - appears to be timing out for some reason. here is my pip freeze

$ pip freeze
bitarray==2.6.0
certifi==2022.9.24
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
crypto==1.4.1
cryptography==38.0.3
humanize==4.4.0
idna==3.4
Naked==0.1.32
parsimonious==0.10.0
pycparser==2.21
pycryptodome==3.15.0
PyYAML==6.0
regex==2022.10.31
requests==2.28.1
shellescape==3.8.1
tabulate==0.9.0
toml==0.10.2
tqdm==4.64.1
urllib3==1.26.12
$ python --version
Python 3.10.8

[0xf4b1]

Also in venv it does not occur for me

$ pip freeze
bitarray==2.6.0
certifi==2022.9.24
charset-normalizer==2.1.1
click==8.1.3
humanize==4.4.0
idna==3.4
keg==0.0.0
parsimonious==0.10.0
pycryptodome==3.15.0
regex==2022.10.31
requests==2.28.1
tabulate==0.9.0
toml==0.10.2
tqdm==4.64.1
urllib3==1.26.12
$ python --version
Python 3.10.6

Have you tried to simply run the command again to resume the download? Or does it fail again for the same file?

[0xf4b1]

Closing this as it seems to be a temporary problem with the servers.