thoth-station / init-job

Thoth's initialization job that automatically initializes a fresh deployment
GNU General Public License v3.0
1 stars 7 forks source link

sync of solver rules fails due to invalid specifier #427

Closed harshad16 closed 3 years ago

harshad16 commented 3 years ago

Describe the bug

During the sync of the solver rules, it fails to sync all as few of the rules fails due to invalid specifier, we need to have update the sync to have this rules added

{"name": "thoth.init_job", "levelname": "ERROR", "module": "app", "lineno": 320, "funcname": "cli", "created": 1625844031.976106, "asctime": "2021-07-09 15:20:31,976", "msecs": 976.1059284210205, "relative_created": 6891127.303361893, "process": 1, "message": "Failed to register Python rule {'description': 'Package released before December 2016', 'index_url': 'https://pypi.org/simple', 'package_name': 'rsswidget', 'version_specifier': '<=0.2.1 beta'}"}
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/packaging/specifiers.py", line 678, in __init__
    parsed.add(Specifier(specifier))
  File "/opt/app-root/lib64/python3.8/site-packages/packaging/specifiers.py", line 103, in __init__
    raise InvalidSpecifier("Invalid specifier: '{0}'".format(spec))
packaging.specifiers.InvalidSpecifier: Invalid specifier: '<=0.2.1 beta'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "app.py", line 379, in <module>
    cli()
  File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "app.py", line 318, in cli
    graph.create_python_rule(**rule)
  File "/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/postgres.py", line 6452, in create_python_rule
    version_range=str(SpecifierSet(version_specifier or "")),
  File "/opt/app-root/lib64/python3.8/site-packages/packaging/specifiers.py", line 680, in __init__
    parsed.add(LegacySpecifier(specifier))
  File "/opt/app-root/lib64/python3.8/site-packages/packaging/specifiers.py", line 281, in __init__
    super(LegacySpecifier, self).__init__(spec, prereleases)
  File "/opt/app-root/lib64/python3.8/site-packages/packaging/specifiers.py", line 103, in __init__
    raise InvalidSpecifier("Invalid specifier: '{0}'".format(spec))
packaging.specifiers.InvalidSpecifier: Invalid specifier: '<=0.2.1 beta'

Expected behavior successfully sync.

harshad16 commented 3 years ago

More rules failed due to this specification:

    "package_name": "sampleadd",
    "version_specifier": "<=1.0,0"
    "package_name": "pygdchart",
    "version_specifier": "<=2.0 Beta"
fridex commented 3 years ago

Thanks for debugging this. 👍🏻

Checking the affected packages - sampleadd, pygdchart, rsswidget - all are old and the version specifier is indeed wrong. Based on the exception, no current official Python version parsing can parse these, which means they cannot be installed:

$ pip install pygdchart
ERROR: Could not find a version that satisfies the requirement pygdchart (from versions: 2.0 Beta)
ERROR: No matching distribution found for pygdchart

I think we can drop version_specifier field completely and let the rule apply for all versions (these packages are old anyway).

fridex commented 3 years ago

Spotted one more caosz with version_specifier <=1,1,0. Now all pass correctly 👍🏻

goern commented 3 years ago

/kind bug /priority important-soon