flathub-infra / flatpak-external-data-checker

A tool for checking if the external data used in Flatpak manifests is still up to date
GNU General Public License v2.0
116 stars 34 forks source link

More helpful error message if checksum of source is missing #421

Closed Alexander-Wilms closed 1 month ago

Alexander-Wilms commented 1 month ago

When switching a source from git to archive, I forgot to add a sha256. The fedc error message is pretty cryptic, though:

app-id: org.octave.Octave
runtime: org.kde.Sdk
runtime-version: 5.15-23.08
sdk: org.kde.Sdk
modules:

  - name: ghostscript
    sources:
      - type: archive
        url: https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs10030/ghostpdl-10.03.0.tar.xz
        # sha256: 854fd1958711b9b5108c052a6d552b906f1e3ebf3262763febf347d77618639d
        x-checker-data:
          type: json
          url: https://api.github.com/repos/ArtifexSoftware/ghostpdl-downloads/releases/latest
          version-query: '.name | split(" ") | .[1]'
          url-query: '.assets[] | select(.name|test("^ghostscript-(.*).tar.xz$")) | .browser_download_url'
alexander@PC:~/Technik/Development/flathub/org.octave.Octave> flatpak run org.flathub.flatpak-external-data-checker org.octave.Octave.yaml
Traceback (most recent call last):
  File "/app/bin/flatpak-external-data-checker", line 30, in <module>
    main()
  File "/app/lib/fedc/src/main.py", line 504, in main
    outdated_num, errors_num, updated = asyncio.run(run_with_args(args))
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/main.py", line 463, in run_with_args
    manifest_checker = manifest.ManifestChecker(args.manifest, options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/manifest.py", line 129, in __init__
    self._collect_external_data()
  File "/app/lib/fedc/src/manifest.py", line 179, in _collect_external_data
    self._collect_module_data(self._root_manifest_path, module)
  File "/app/lib/fedc/src/manifest.py", line 239, in _collect_module_data
    self._collect_source_data(module_path, module.get("sources", []), module_obj)
  File "/app/lib/fedc/src/manifest.py", line 251, in _collect_source_data
    self._collect_source_data(
  File "/app/lib/fedc/src/manifest.py", line 295, in _collect_source_data
    data = ExternalBase.from_source(source_path, source, module)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/lib/externaldata.py", line 253, in from_source
    return super().from_source(source_path, source, module)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/lib/externaldata.py", line 172, in from_source
    return data_cls.from_source_impl(source_path, source, module)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/lib/externaldata.py", line 346, in from_source_impl
    checksum = MultiDigest.from_source(source)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/fedc/src/lib/checksums.py", line 23, in from_source
    assert digests, source
           ^^^^^^^
AssertionError: ordereddict([('type', 'archive'), ('url', 'https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs10030/ghostpdl-10.03.0.tar.xz'), ('x-checker-data', ordereddict([('type', 'json'), ('url', 'https://api.github.com/repos/ArtifexSoftware/ghostpdl-downloads/releases/latest'), ('version-query', '.name | split(" ") | .[1]'), ('url-query', '.assets[] | select(.name|test("^ghostscript-(.*).tar.xz$")) | .browser_download_url')]))])

With the sha256 specified:

alexander@PC:~/Technik/Development/flathub/org.octave.Octave> flatpak run org.flathub.flatpak-external-data-checker org.octave.Octave.yaml
INFO    src.manifest: Checking 1 external data items
INFO    src.manifest: Started check [1/1] archive ghostscript/ghostpdl-10.03.0.tar.xz (from org.octave.Octave.yaml)
INFO    src.lib.externaldata: Source ghostpdl-10.03.0.tar.xz: got new version 10.03.1
INFO    src.manifest: Finished check [1/1] archive ghostscript/ghostpdl-10.03.0.tar.xz (from org.octave.Octave.yaml)
OUTDATED: ghostpdl-10.03.0.tar.xz
 Has a new version:
  URL:       https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs10031/ghostscript-10.03.1.tar.xz
  MD5:       248294abc5aee3ca8161012853d73b52
  SHA1:      caa78ad32521b6aace1843e8590fa9c6340de25b
  SHA256:    157212edc96b8ccc409475dce2e49833fb4427f150c455258ded9632c106abee
  SHA512:    59e071cd23aed246b51317d3cccd9ccf8e1e84c4ecf9b22fb938e561eff42cec61915b0bdd96a313a3ea27750579c54ca0d0862b65edd306a621731c0063ffd3
  Size:      68048612
  Version:   10.03.1
  Timestamp: 2024-05-17 11:30:52

INFO    src.main: Check finished with 0 error(s)
bbhtt commented 1 month ago

Linting the manifest is not x-checker's job though, you can use flatpak-builder-lint manifest for that which does proper jsonschema validation of most properties.

Added a basic handling since it reaches the ugly assertion which should not be reached.

wjt commented 1 month ago

Should be solved by #423. Thanks @bbhtt!