search

pacstall / pacup.py

Help maintainers update pacscripts
GNU General Public License v3.0
12 stars 3 forks source link

[Bug]: KeyError: 'Content-Length' #32

Closed Zahrun closed 2 years ago

Zahrun commented 2 years ago

What happened?

image

What do you expect to happen?

pacup works

Which pacscripts are affected?

  1. bitwig-studio-deb (zahrun/pacstall-programs)

Steps to reproduce

  1. pacup bitwig-studio-deb

Version

Latest Release

Relevant log output

[04/28/22 21:49:56] INFO     PacUp 0.1.2 Thyone (dev)                                   __main__.py:271
                    INFO     Parsing 1 pacscripts                                       __main__.py:282
                    INFO     Parsing bitwig-studio-deb.pacscript...                       parser.py:215
                    INFO     Sourcing bitwig-studio-deb.pacscript...                      parser.py:235
                    INFO     Found repology: repology=("project: bitwig-studio")          parser.py:290
                    DEBUG    repology_filters = {'project': 'bitwig-studio'}              parser.py:303
                    INFO     Found version: version="4.2"                                 parser.py:251
                    INFO     Found url: url="https://downloads.bitwig.com/${version}/${gi parser.py:265
                             ves}-${version}.deb"                                                      
                    INFO     Found hash: hash="848204563e6c33f35d8418b07ab16fc885ea8f447c parser.py:278
                             12f731c0ce94674084e36f"                                                   
                    INFO     Found maintainer: maintainer="Zahrun <Zahrun@github.com>"    parser.py:282
                    INFO     Getting project info from repology...                       version.py:128
[04/28/22 21:49:58] INFO     Filtering...                                                version.py:161
                    INFO     Mapping the versions to their filtered packages...          version.py:171
                    DEBUG    filtered = [{'repo': 'aur', 'srcname': 'bitwig-studio',     version.py:177
                             'binname': 'bitwig-studio', 'visiblename': 'bitwig-studio',               
                             'version': '4.2.3', 'maintainers': ['stylemistake@aur'],                  
                             'licenses': ['custom'], 'summary': 'Digital audio                         
                             workstation for music production, remixing and live                       
                             performance', 'status': 'newest', 'origversion':                          
                             '4.2.3-1'}, {'repo': 'nix_unstable', 'name':                              
                             'bitwig-studio', 'visiblename': 'bitwig-studio', 'version':               
                             '4.2.3', 'maintainers': ['bernard.fortz@gmail.com',                       
                             'm@michalrus.com', 'contact@ingolf-wagner.de'], 'licenses':               
                             ['Unfree'], 'summary': 'A digital audio workstation',                     
                             'status': 'newest', 'origversion': None}]                                 
                    DEBUG    versions = ['4.2.3', '4.2.3']                               version.py:178
                    INFO     Selecting most common version...                            version.py:180
                    DEBUG    selected_version = '4.2.3'                                  version.py:182
                    INFO     Fetching release notes...                                    parser.py:313
                    DEBUG    parsed_pacscripts =                                        __main__.py:292
                             [Pacscript(name=bitwig-studio-deb.pacscript,                              
                             pkgname=bitwig-studio, version=Version(line_number=3,                     
                             current=4.2, latest=4.2.3,                                                
                             status=VersionStatuses.OUTDATED), url=Url(line_number=4, v                
                             alue=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.de                
                             b), hash_line=6, maintainer='Zahrun <Zahrun@github.com>'                  
                             repology_filters={'status': 'newest'})]                                   

                    INFO     Sorting parsed pacscripts by version statsuses...          __main__.py:296
                    DEBUG    oudated pacscripts = ['bitwig-studio-deb']                 __main__.py:314
                    DEBUG    up-to-date pacscripts = []                                 __main__.py:317
                    DEBUG    newer pacscripts = []                                      __main__.py:320
                    DEBUG    unknown pacscripts = []                                    __main__.py:323
                    INFO     Adding outdated pacscripts to version statuses...          __main__.py:328
╭────────────────────────── Version statuses ──────────────────────────╮
│ ╭──────────────────────────── Outdated ────────────────────────────╮ │
│ │      Pacscript      Current  Latest          Maintainer          │ │
│ │  bitwig-studio-deb      4.2   4.2.3  Zahrun <Zahrun@github.com>  │ │
│ ╰──────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────╯
                    INFO     Updating pacscripts...                                     __main__.py:424
=> Updating bitwig-studio-deb pacscript (4.2 => 4.2.3)
    ❌ Could not find release notes
                    INFO     Downloading new package...                                 __main__.py:459
    ▐  ⡀     ▌ Downloading package ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0% • 0/100 bytes • -:--:-- • ?
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/aroun/.local/bin/pacup:8 in <module>                                                       │
│                                                                                                  │
│   5 from pacup.__main__ import main                                                              │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ ╭───────────────────────────────────────── locals ─────────────────────────────────────────╮     │
│ │ __annotations__ = {}                                                                     │     │
│ │    __builtins__ = <module 'builtins' (built-in)>                                         │     │
│ │      __cached__ = None                                                                   │     │
│ │         __doc__ = None                                                                   │     │
│ │        __file__ = '/home/aroun/.local/bin/pacup'                                         │     │
│ │      __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7f05cdb35750> │     │
│ │        __name__ = '__main__'                                                             │     │
│ │     __package__ = None                                                                   │     │
│ │        __spec__ = None                                                                   │     │
│ │            main = <function main at 0x7f05cc137130>                                      │     │
│ │              re = <module 're' from '/usr/lib/python3.10/re.py'>                         │     │
│ │             sys = <module 'sys' (built-in)>                                              │     │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────╯     │
│                                                                                                  │
│ /home/aroun/.local/lib/python3.10/site-packages/pacup/__main__.py:652 in main                    │
│                                                                                                  │
│   649 def main() -> None:                                                                        │
│   650 │   """The main function."""                                                               │
│   651 │   traceback.install(show_locals=True)                                                    │
│ ❱ 652 │   app()                                                                                  │
│   653                                                                                            │
│   654                                                                                            │
│   655 if __name__ == "__main__":                                                                 │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/typer/main.py:214 in __call__                                     │
│                                                                                                  │
│   211 │   │   )                                                                                  │
│   212 │                                                                                          │
│   213 │   def __call__(self, *args: Any, **kwargs: Any) -> Any:                                  │
│ ❱ 214 │   │   return get_command(self)(*args, **kwargs)                                          │
│   215                                                                                            │
│   216                                                                                            │
│   217 def get_group(typer_instance: Typer) -> click.Command:                                     │
│                                                                                                  │
│ ╭─────────────────────── locals ───────────────────────╮                                         │
│ │   args = ()                                          │                                         │
│ │ kwargs = {}                                          │                                         │
│ │   self = <typer.main.Typer object at 0x7f05cda27220> │                                         │
│ ╰──────────────────────────────────────────────────────╯                                         │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/click/core.py:1128 in __call__                                    │
│                                                                                                  │
│   1125 │                                                                                         │
│   1126 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1127 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1128 │   │   return self.main(*args, **kwargs)                                                 │
│   1129                                                                                           │
│   1130                                                                                           │
│   1131 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ ╭──────────── locals ────────────╮                                                               │
│ │   args = ()                    │                                                               │
│ │ kwargs = {}                    │                                                               │
│ │   self = <TyperCommand update> │                                                               │
│ ╰────────────────────────────────╯                                                               │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/click/core.py:1053 in main                                        │
│                                                                                                  │
│   1050 │   │   try:                                                                              │
│   1051 │   │   │   try:                                                                          │
│   1052 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1053 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1054 │   │   │   │   │   if not standalone_mode:                                               │
│   1055 │   │   │   │   │   │   return rv                                                         │
│   1056 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ ╭────────────────────────────── locals ───────────────────────────────╮                          │
│ │                args = []                                            │                          │
│ │        complete_var = None                                          │                          │
│ │                 ctx = <click.core.Context object at 0x7f05cc1497e0> │                          │
│ │               extra = {}                                            │                          │
│ │           prog_name = 'pacup'                                       │                          │
│ │                self = <TyperCommand update>                         │                          │
│ │     standalone_mode = True                                          │                          │
│ │ windows_expand_args = True                                          │                          │
│ ╰─────────────────────────────────────────────────────────────────────╯                          │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/click/core.py:1395 in invoke                                      │
│                                                                                                  │
│   1392 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1393 │   │                                                                                     │
│   1394 │   │   if self.callback is not None:                                                     │
│ ❱ 1395 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1396 │                                                                                         │
│   1397 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]   │
│   1398 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ ╭─────────────────────── locals ───────────────────────╮                                         │
│ │  ctx = <click.core.Context object at 0x7f05cc1497e0> │                                         │
│ │ self = <TyperCommand update>                         │                                         │
│ ╰──────────────────────────────────────────────────────╯                                         │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/click/core.py:754 in invoke                                       │
│                                                                                                  │
│    751 │   │                                                                                     │
│    752 │   │   with augment_usage_errors(__self):                                                │
│    753 │   │   │   with ctx:                                                                     │
│ ❱  754 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    755 │                                                                                         │
│    756 │   def forward(                                                                          │
│    757 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ ╭────────────────────────────────────── locals ──────────────────────────────────────╮           │
│ │ _Context__callback = <function update at 0x7f05cc1552d0>                           │           │
│ │     _Context__self = <click.core.Context object at 0x7f05cc1497e0>                 │           │
│ │               args = ()                                                            │           │
│ │                ctx = <click.core.Context object at 0x7f05cc1497e0>                 │           │
│ │             kwargs = {                                                             │           │
│ │                      │   '_': None,                                                │           │
│ │                      │   'debug': True,                                            │           │
│ │                      │   'pacscripts': [                                           │           │
│ │                      │   │   PosixPath('bitwig-studio-deb.pacscript')              │           │
│ │                      │   ],                                                        │           │
│ │                      │   'show_repology': None                                     │           │
│ │                      }                                                             │           │
│ ╰────────────────────────────────────────────────────────────────────────────────────╯           │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/typer/main.py:500 in wrapper                                      │
│                                                                                                  │
│   497 │   │   │   │   use_params[k] = v                                                          │
│   498 │   │   if context_param_name:                                                             │
│   499 │   │   │   use_params[context_param_name] = click.get_current_context()                   │
│ ❱ 500 │   │   return callback(**use_params)  # type: ignore                                      │
│   501 │                                                                                          │
│   502 │   update_wrapper(wrapper, callback)                                                      │
│   503 │   return wrapper                                                                         │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │           callback = <function update at 0x7f05cc1371c0>                                     │ │
│ │ context_param_name = None                                                                    │ │
│ │         convertors = {                                                                       │ │
│ │                      │   'pacscripts': <function                                             │ │
│ │                      generate_iter_convertor.<locals>.internal_convertor at 0x7f05cc155120>  │ │
│ │                      }                                                                       │ │
│ │                  k = 'show_repology'                                                         │ │
│ │             kwargs = {                                                                       │ │
│ │                      │   '_': None,                                                          │ │
│ │                      │   'debug': True,                                                      │ │
│ │                      │   'pacscripts': [PosixPath('bitwig-studio-deb.pacscript')],           │ │
│ │                      │   'show_repology': None                                               │ │
│ │                      }                                                                       │ │
│ │         use_params = {                                                                       │ │
│ │                      │   'show_repology': None,                                              │ │
│ │                      │   'debug': True,                                                      │ │
│ │                      │   '_': None,                                                          │ │
│ │                      │   'pacscripts': [PosixPath('bitwig-studio-deb.pacscript')]            │ │
│ │                      }                                                                       │ │
│ │                  v = None                                                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/aroun/.local/lib/python3.10/site-packages/pacup/__main__.py:479 in update                  │
│                                                                                                  │
│   476 │   │   │   │   "Downloading package", start=False                                         │
│   477 │   │   │   )                                                                              │
│   478 │   │   │   try:                                                                           │
│ ❱ 479 │   │   │   │   latest_hash = loop.run_until_complete(                                     │
│   480 │   │   │   │   │   download(                                                              │
│   481 │   │   │   │   │   │   url.value.replace(version.current, version.latest),                │
│   482 │   │   │   │   │   │   downloading_package_progress,                                      │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │                               _ = None                                                       │ │
│ │                           debug = True                                                       │ │
│ │    downloading_package_progress = <rich.progress.Progress object at 0x7f05cc14aaa0>          │ │
│ │     failed_to_update_pacscripts = {}                                                         │ │
│ │                       hash_line = 6                                                          │ │
│ │                           lines = [                                                          │ │
│ │                                   │   'name="bitwig-studio-deb"\n',                          │ │
│ │                                   │   'repology=("project: bitwig-studio")\n',               │ │
│ │                                   │   'gives="bitwig-studio"\n',                             │ │
│ │                                   │   'version="4.2"\n',                                     │ │
│ │                                   │                                                          │ │
│ │                                   'url="https://downloads.bitwig.com/${version}/${gives}-${… │ │
│ │                                   │   'description="Digital audio workstation for music      │ │
│ │                                   production, remixing and live p'+12,                       │ │
│ │                                   │                                                          │ │
│ │                                   'hash="848204563e6c33f35d8418b07ab16fc885ea8f447c12f731c0… │ │
│ │                                   │   'maintainer="Zahrun <Zahrun@github.com>"\n'            │ │
│ │                                   ]                                                          │ │
│ │                             log = <Logger rich (DEBUG)>                                      │ │
│ │                            loop = <_UnixSelectorEventLoop running=False closed=False         │ │
│ │                                   debug=False>                                               │ │
│ │                newer_pacscripts = []                                                         │ │
│ │              outdated_pacscript = Pacscript(name=bitwig-studio-deb.pacscript,                │ │
│ │                                   pkgname=bitwig-studio, version=Version(line_number=3,      │ │
│ │                                   current=4.2, latest=4.2.3,                                 │ │
│ │                                   status=VersionStatuses.OUTDATED), url=Url(line_number=4,   │ │
│ │                                   value=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.… │ │
│ │                                   hash_line=6, maintainer='Zahrun <Zahrun@github.com>'       │ │
│ │                                   repology_filters={'status': 'newest'})                     │ │
│ │             outdated_pacscripts = [                                                          │ │
│ │                                   │   Pacscript(name=bitwig-studio-deb.pacscript,            │ │
│ │                                   pkgname=bitwig-studio, version=Version(line_number=3,      │ │
│ │                                   current=4.2, latest=4.2.3,                                 │ │
│ │                                   status=VersionStatuses.OUTDATED), url=Url(line_number=4,   │ │
│ │                                   value=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.… │ │
│ │                                   hash_line=6, maintainer='Zahrun <Zahrun@github.com>'       │ │
│ │                                   repology_filters={'status': 'newest'})                     │ │
│ │                                   ]                                                          │ │
│ │       outdated_pacscripts_table = <rich.table.Table object at 0x7f05cc149de0>                │ │
│ │                       pacscript = Pacscript(name=bitwig-studio-deb.pacscript,                │ │
│ │                                   pkgname=bitwig-studio, version=Version(line_number=3,      │ │
│ │                                   current=4.2, latest=4.2.3,                                 │ │
│ │                                   status=VersionStatuses.OUTDATED), url=Url(line_number=4,   │ │
│ │                                   value=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.… │ │
│ │                                   hash_line=6, maintainer='Zahrun <Zahrun@github.com>'       │ │
│ │                                   repology_filters={'status': 'newest'})                     │ │
│ │                      pacscripts = [PosixPath('bitwig-studio-deb.pacscript')]                 │ │
│ │               parsed_pacscripts = [                                                          │ │
│ │                                   │   Pacscript(name=bitwig-studio-deb.pacscript,            │ │
│ │                                   pkgname=bitwig-studio, version=Version(line_number=3,      │ │
│ │                                   current=4.2, latest=4.2.3,                                 │ │
│ │                                   status=VersionStatuses.OUTDATED), url=Url(line_number=4,   │ │
│ │                                   value=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.… │ │
│ │                                   hash_line=6, maintainer='Zahrun <Zahrun@github.com>'       │ │
│ │                                   repology_filters={'status': 'newest'})                     │ │
│ │                                   ]                                                          │ │
│ │     parsing_pacscripts_progress = <rich.progress.Progress object at 0x7f05cc149d80>          │ │
│ │                            path = PosixPath('bitwig-studio-deb.pacscript')                   │ │
│ │                         pkgname = 'bitwig-studio'                                            │ │
│ │                   release_notes = {}                                                         │ │
│ │                   show_repology = None                                                       │ │
│ │ successfully_updated_pacscripts = []                                                         │ │
│ │                            task = 0                                                          │ │
│ │              unknown_pacscripts = []                                                         │ │
│ │              updated_pacscripts = []                                                         │ │
│ │                             url = Url(line_number=4,                                         │ │
│ │                                   value=https://downloads.bitwig.com/4.2/bitwig-studio-4.2.… │ │
│ │                         version = Version(line_number=3, current=4.2, latest=4.2.3,          │ │
│ │                                   status=VersionStatuses.OUTDATED)                           │ │
│ │          version_statuses_table = <rich.table.Table object at 0x7f05cc14a740>                │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /usr/lib/python3.10/asyncio/base_events.py:646 in run_until_complete                             │
│                                                                                                  │
│    643 │   │   if not future.done():                                                             │
│    644 │   │   │   raise RuntimeError('Event loop stopped before Future completed.')             │
│    645 │   │                                                                                     │
│ ❱  646 │   │   return future.result()                                                            │
│    647 │                                                                                         │
│    648 │   def stop(self):                                                                       │
│    649 │   │   """Stop running the event loop.                                                   │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │   future = <Task finished name='Task-6' coro=<download() done, defined at                    │ │
│ │            /home/aroun/.local/lib/python3.10/site-packages/pacup/__main__.py:72>             │ │
│ │            exception=KeyError('Content-Length')>                                             │ │
│ │ new_task = True                                                                              │ │
│ │     self = <_UnixSelectorEventLoop running=False closed=False debug=False>                   │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/aroun/.local/lib/python3.10/site-packages/pacup/__main__.py:96 in download                 │
│                                                                                                  │
│    93 │   async with AsyncClient(follow_redirects=True).stream("GET", url) as response:          │
│    94 │   │   response.raise_for_status()                                                        │
│    95 │   │   makedirs("/tmp/pacup", exist_ok=True)                                              │
│ ❱  96 │   │   progress.update(task, total=int(response.headers["Content-Length"]))               │
│    97 │   │                                                                                      │
│    98 │   │   with open("/tmp/pacup/" + url.split("/")[-1], "wb") as file:                       │
│    99 │   │   │   progress.start_task(task)                                                      │
│                                                                                                  │
│ ╭─────────────────────────────────── locals ───────────────────────────────────╮                 │
│ │ download_hash = <sha256 _hashlib.HASH object @ 0x7f05cb054370>               │                 │
│ │      progress = <rich.progress.Progress object at 0x7f05cc14aaa0>            │                 │
│ │      response = <Response [200 OK]>                                          │                 │
│ │          task = 0                                                            │                 │
│ │           url = 'https://downloads.bitwig.com/4.2.3/bitwig-studio-4.2.3.deb' │                 │
│ ╰──────────────────────────────────────────────────────────────────────────────╯                 │
│                                                                                                  │
│ /usr/lib/python3/dist-packages/httpx/_models.py:993 in __getitem__                               │
│                                                                                                  │
│    990 │   │   if items:                                                                         │
│    991 │   │   │   return ", ".join(items)                                                       │
│    992 │   │                                                                                     │
│ ❱  993 │   │   raise KeyError(key)                                                               │
│    994 │                                                                                         │
│    995 │   def __setitem__(self, key: str, value: str) -> None:                                  │
│    996 │   │   """                                                                               │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │          items = []                                                                          │ │
│ │            key = 'Content-Length'                                                            │ │
│ │ normalized_key = b'content-length'                                                           │ │
│ │           self = Headers([('accept-ranges', 'bytes'), ('etag',                               │ │
│ │                  '"88ab8a45681e1d86e8347d6faf8d121e:1649410299.413639"'), ('last-modified',  │ │
│ │                  'Fri, 08 Apr 2022 09:33:07 GMT'), ('server', 'AkamaiNetStorage'), ('vary',  │ │
│ │                  'Accept-Encoding'), ('content-encoding', 'gzip'), ('date', 'Thu, 28 Apr     │ │
│ │                  2022 16:19:58 GMT'), ('transfer-encoding', 'chunked'), ('connection',       │ │
│ │                  'keep-alive'), ('connection', 'Transfer-Encoding'), ('content-type',        │ │
│ │                  'application/octet-stream')])                                               │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'Content-Length'
wizard-28 commented 2 years ago

I'll get to it tomorrow, thanks for reporting this issue!

wizard-28 commented 2 years ago

Turns out that httpx (the HTTP client used in pacup), doesn't return the Content-Length response header for that link (works for other packages). Causing pacup to crash.

The Content-Length header is returned when I check the response headers though a online tool. So it's definitely present but httpx doesn't seem to be able to read it for that link.

I've contacted them on their matrix channel, haven't received a response yet. I'll create a issue on their GitHub tomorrow if I do not receive a reply by tonight.

wizard-28 commented 2 years ago

My exams are on-going so I wasn't able to follow up on that for 3 days. Sorry for that. I've made a github discussion on the encode/httpx repository after receiving no response in their Matrix room.

Zahrun commented 2 years ago

Sure, that is totally fine. Thank you.