Create purlcli option to run d2d given a single PURL input

nexB / purldb

Tools to create and expose a database of purls (Package URLs). This project is sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase/ and nexB for https://www.aboutcode.org/ Chat is at https://gitter.im/aboutcode-org/discuss

https://purldb.readthedocs.io/

29 stars 21 forks source link

Create purlcli option to run d2d given a single PURL input #420

Closed pombredanne closed 1 month ago

pombredanne commented 2 months ago

I would like to have this feature with this design (which needs refinement)

Call purlcli using a PURL as an input (typically a binary PURL)
This would call and get back the list of collected using the collect/ API endpoint ensuring that purl2vcs or the corresponding source archive collection is taking place per
- https://github.com/nexB/purldb/issues/418
Given the PURLs for the binay and source, call the matchcode API endpoint to run the d2d:
- https://github.com/nexB/purldb/issues/373
Once the d2d is complete, fetch the results and present them back
The actual results to report is TBD. Some directions:
- un-mapped file resources with would be great candidates for details (See also nexb/scancode.io#1213 )
- we may include the whole JSON results as an option?
- The results would be plain JSON

These are good to use as an example:

pombredanne commented 2 months ago

Here is another example:

pkg:npm/unicorn-magic@0.1.0 yield this download from collect/ https://registry.npmjs.org/unicorn-magic/-/unicorn-magic-0.1.0.tgz#to
Once https://github.com/nexB/purldb/issues/418 is available, collect/ should also ensure that this source repo PURL pkg:github/sindresorhus/unicorn-magic@67bb2fd1da1eecb252ba14284df14c0d234a8cfa and this download URL is created https://github.com/sindresorhus/unicorn-magic/archive/67bb2fd1da1eecb252ba14284df14c0d234a8cfa.zip#from

Therefore I think we should have enough data in this simple case to get the tow and from URLs for a d2d run even if the full indexing would not be completed by the time the collect/ call returns

TG1999 commented 1 month ago

This is done now.

Option d2d and d2d-purl-set are now available in purlcli

This PR has been merged

https://github.com/nexB/purldb/pull/429

To test this feature:

Install PurlDB and matchcode.io from the main branch
Run
- make dev
- make run_matchcodeio in a separate terminal
- make run in a separate terminal

Run this command in purldb after running make dev in another terminal

For running d2d on a single PURL

purlcli d2d-purl-set --purl pkg:github/expressjs/express@4.19.0 --output - --purldb-api-url http://127.0.0.1:8001/api/ --matchcode-api-url http://127.0.0.1:8002/api/

For running d2d on pair of 2 PURLs


 --purldb-api-url http://127.0.0.1:8001/api/ --matchcode-api-url http://127.0.0.1:8002/api/ --output -```

Output with d2d results will appear on screen