sonatype-nexus-community / jake

Check your Python environments for vulnerable Open Source packages with OSS Index or Sonatype Nexus Lifecycle.
https://jake.readthedocs.io/
Apache License 2.0
111 stars 24 forks source link

[BUG] jake ddt failure: unexpected keyword argument 'sonatype_ossi_score' #118

Closed colinxfleming closed 2 years ago

colinxfleming commented 2 years ago

Describe the bug Running jake ddt in ci and local envs on 2.0.0 is failing with an unexpected kwarg. From a quick scan of the error looks like this might be a snake case vs camelcase problem on sonatypeOssiScore? idk.

To Reproduce Steps to reproduce the behavior:

  1. Run jake ddt
  2. Observe this error:
colin$ jake ddt
                   ___           ___           ___     
       ___        /  /\         /  /\         /  /\    
      /__/\      /  /::\       /  /:/        /  /::\   
      \__\:\    /  /:/\:\     /  /:/        /  /:/\:\  
  ___ /  /::\  /  /::\ \:\   /  /::\____   /  /::\ \:\ 
 /__/\  /:/\/ /__/:/\:\_\:\ /__/:/\:::::\ /__/:/\:\ \:\
 \  \:\/:/~~  \__\/  \:\/:/ \__\/~|:|~~~~ \  \:\ \:\_\/
  \  \::/          \__\::/     |  |:|      \  \:\ \:\  
   \__\/           /  /:/      |  |:|       \  \:\_\/  
                  /__/:/       |__|:|        \  \:\    
                  \__\/         \__\|         \__\/    

            /)                     /)             
        _/_(/    _     _  __   _  (/_   _         
 o   o  (__/ )__(/_   /_)_/ (_(_(_/(___(/_ o   o  

Jake Version: 2.0.0
Put your Python dependencies in a chokehold

Error was on {'coordinates': 'pkg:pypi/regex@2022.3.15', 'description': 'Alternative regular expression module, to replace re.', 'reference': 
'https://ossindex.sonatype.org/component/pkg:pypi/regex@2022.3.15?utm_source=python-oss-index-lib%401.0.0&utm_medium=integration', 'vulnerabilities': [], 'sonatypeOssiScore': 
0.0}
🐍 Collected 165 packages from your environment ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Querying OSS Index for details on your packages ━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━  30% -:--:--
Sanity checking...                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
Traceback (most recent call last):
  File "<REDACTED>/venv/bin/jake", line 8, in <module>
    sys.exit(main())
  File "<REDACTED>/venv/lib/python3.10/site-packages/jake/app.py", line 106, in main
    JakeCmd(args).execute()
  File "<REDACTED>/venv/lib/python3.10/site-packages/jake/app.py", line 80, in execute
    exit_code: int = command.execute(arguments=self._arguments)
  File "<REDACTED>/venv/lib/python3.10/site-packages/jake/command/__init__.py", line 43, in execute
    return self.handle_args()
  File "<REDACTED>/venv/lib/python3.10/site-packages/jake/command/oss.py", line 79, in handle_args
    oss_index_results = oss.get_component_report(
  File "<REDACTED>/venv/lib/python3.10/site-packages/ossindex/ossindex.py", line 84, in get_component_report
    return self._get_results(packages=packages)
  File "<REDACTED>/venv/lib/python3.10/site-packages/ossindex/ossindex.py", line 171, in _get_results
    results = results + self._make_oss_index_component_report_call(packages=chunk)
  File "<REDACTED>/venv/lib/python3.10/site-packages/ossindex/ossindex.py", line 190, in _make_oss_index_component_report_call
    for oic in response.json(object_hook=json_decoder):
  File "<REDACTED>/venv/lib/python3.10/site-packages/requests/models.py", line 899, in json
    return complexjson.loads(
  File "/Users/colin/.pyenv/versions/3.10.3/lib/python3.10/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "/Users/colin/.pyenv/versions/3.10.3/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/colin/.pyenv/versions/3.10.3/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "<REDACTED>/venv/lib/python3.10/site-packages/ossindex/serializer.py", line 50, in json_decoder
    raise e
  File "<REDACTED>/venv/lib/python3.10/site-packages/ossindex/serializer.py", line 45, in json_decoder
    return OssIndexComponent(**named_o)
TypeError: OssIndexComponent.__init__() got an unexpected keyword argument 'sonatype_ossi_score'

Expected behavior Successful run

Screenshots I don't have a screenshot but hopefully this helps  image

Desktop (please complete the following information):

Additional context hi @DarthHater

mbrei commented 2 years ago

I am facing the same issue.

davidrudlstorfer commented 2 years ago

Currently I'm facing the same bug. A fresh install and other attempts also failed.

Strangely I do get #118 and #119 depending on the hardware.

hectorcanto commented 2 years ago

Same error, with jake 1.4.5 and 2.0.0, over python 3.8.9, with or without clearing the cache Command: jake -X ddt --clear-cache

TonyBjerstedtTR commented 2 years ago

Same error with jake 2.0.0, python 3.9.9 installed using poetry both as dev requirement and normal requirement.

daviskirk commented 2 years ago

I assume this is related to https://ossindex.sonatype.org/updates-notice?

ken-duck commented 2 years ago

This has been brought to my attention. The OSS Index team is working on a fix for this issue now. Sorry for the inconvenience.

colinxfleming commented 2 years ago

thanks much @ken-duck , glad to hear the OSS Index team is going to hit this bug with a flying elbow drop. ooh yeah

ken-duck commented 2 years ago

A new update has been pushed to our servers that should resolve the issue.

TonyBjerstedtTR commented 2 years ago

Downloaded latest version and jake ddt witouot additional parameters works. However when I try adding --output-file or even doing a redirect (jake ddt > some-file) I get an error message:

Traceback (most recent call last):
  File "C:\Users\C276261\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\C276261\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\projects\repos\tr\truccr\truccr_azure-manual-transmission\.venv\Scripts\jake.exe\__main__.py", line 7, in <module>
  File "C:\projects\repos\tr\truccr\truccr_azure-manual-transmission\.venv\lib\site-packages\jake\app.py", line 106, in main
    JakeCmd(args).execute()
  File "C:\projects\repos\tr\truccr\truccr_azure-manual-transmission\.venv\lib\site-packages\jake\app.py", line 80, in execute
    exit_code: int = command.execute(arguments=self._arguments)
  File "C:\projects\repos\tr\truccr\truccr_azure-manual-transmission\.venv\lib\site-packages\jake\command\__init__.py", line 43, in execute
    return self.handle_args()
  File "C:\projects\repos\tr\truccr\truccr_azure-manual-transmission\.venv\lib\site-packages\jake\command\oss.py", line 183, in handle_args
    schema_version=SchemaVersion['V{}'.format(
  File "C:\Users\C276261\AppData\Local\Programs\Python\Python39\lib\enum.py", line 432, in __getitem__
    return cls._member_map_[name]
KeyError: 'V1_4)'

BTW -- running python 3.9.9 on Windows 10

colinxfleming commented 2 years ago

@ken-duck yep, can confirm that you all have made this return results instead of erroring again - builds are once again the cream of the crop. (Some fresh stuff too, so the results are a little different, but I imagine that's to be expected since I imagine some new stuff got released into the pond, and it looks like OSS Index free is using a new pipeline.)

ken-duck commented 2 years ago

@colinxfleming great, thanks for letting us know.

OSS Index is definitely using a new pipeline which should provide many more vulnerabilities, with more frequent updates, and with MUCH lower false positives and negatives. It has a significantly larger research team behind it now as well.

Unfortunately we have caused some problems with the switch, since it was a massive change which took well over a year to complete. I am hoping we can get these teething problems dealt with quickly, and your feedback has certainly helped.

colinxfleming commented 2 years ago

That makes sense! I think for us that means we'll keep it off a little while longer while it settles/while we figure out what to do about some of the stuff without an established cure yet. (for ex, the warning on pip that is marked as disputed, which may or may not have a fix right now.) That said, that's definitely a me problem and not a sonatype problem, so ball's back in my court! Thanks much, you all are a bunch of intercontinental champions.

madpah commented 2 years ago

Thanks for resolving this @ken-duck !

@ken-duck confirmed this was a change at ossindex.sonatype.org, and not a but per se in jake.