phylum-dev / cli

Command line interface for the Phylum API
https://phylum.io
GNU General Public License v3.0
103 stars 11 forks source link

Publishing phylum_lockfile to crates.io #1504

Open jayvdb opened 4 days ago

jayvdb commented 4 days ago

Would it be possible to publish phylum_lockfile to crates.io, as it is incredibly useful for other projects that want to do analysis of project dependencies.

This would require also publishing https://github.com/phylum-dev/phylum-types , and also lockfile_generator , and the latter also looks very useful for other projects.

kylewillmon commented 3 days ago

We have considered publishing it, but we weren't sure it would be worth the effort. If you know of projects that would like to use this crate, please let us know.

I don't think it makes sense to publish phylum_types. But we only depend on it for PackageType, and we could probably refactor the crate to use purl::PackageType instead.

And then there are a few other minor considerations:

  1. phylum_lockfile isn't a great name for a published crate, so we would want to consider renaming it.
  2. It is GPLv3 licensed. We have published crates under this license before (specifically, birdcage), but we would want to consider switching to MIT for this one (to match our purl crate).

But these don't prevent publishing since we can change the name or license at any time.

jayvdb commented 3 days ago

Hi Kyle, tl;dr - good cross-language lock file parsing is the foundation of building project management tools.

While I havent discussed it with the maintainer, https://github.com/simonbuchan/pnpm-extra could benefit from merging its pnpm lock file loader (which loads importers & snapshots) with the one in this repo (which loads packages). At the moment neither has the entire lock file data model. I'd put my hand up to do the work to merge them if desired.

The other Rust pnpm lock file parsers that I know of are https://github.com/moonrepo/moon/blob/d69e3e4952f6e146e07374aa5464ae3bc7caa64f/legacy/node/lang/src/pnpm/mod.rs#L50-L57 , which omits snapshots, and https://github.com/milesj/rust-js-ecosystem/blob/f744bfce7a9eb4a0758cde3bd30e5a85458522f9/crates/package-managers/src/pnpm.rs#L29-L33 which also only loads packages.

Also https://github.com/d4rkr00t/rnpm/issues/1 is "blocked" on needing lock parsers for pnpm and yarn. See also https://github.com/orgs/pnpm/discussions/3419 and https://github.com/pnpm/pn for other pnpm efforts that could benefit from having a good pnpm lock file parser. https://github.com/dimensionhq/volt might get more traction if it could support other node lock files as well as its own lock file format. And there are several tools which convert pnpm lock files to package-lock.json so that pnpm projects can utilise tools that only support the more common package-lock.json - I can see that being rewritten in Rust if there are good data models for these lockfile formats.

All the above is focused mostly on benefits that could come of publishing a pnpm lock file parser as that is the problem I have been focused on, where there is not existing complete Rust parsers available.

My interest is I am currently building a tool for regulatory compliance that needs to load project dependency data and must know (1) the locked version of each dependency and (2) inspect the source for that version of the dependency out of the local cache , targeting three project types atm: cargo, pnpm and pyproject.toml+"uv pip compile" requirements.txt . I would prefer to be using one interface to gather the per-project information, so that I can easily support additional project types. First version is complete for cargo and pnpm; now I am trying to get my logic tidied up by re-locating reusable components into OSS projects.

And due to finding https://github.com/jslicense/licensee.js being the best there is, and yet it is severely limited compared to cargo-deny (c.f. https://github.com/jslicense/licensee.js/issues/92 ), I am quite keen to re-use the tool I've built to add frontend license checking. Seems very easy now that I have got recursive dep tree navigation working.