Invalid type error when parsing metadata for packages

[NorfairKing]

I just wanted to try this out but got this:

$ nix run "https://flakehub.com/f/louib/nix2sbom/0.1.66.tar.gz" -
[00:00:00.000] (7f113f94a7c0) INFO   Getting the derivations from the current system.
[00:00:02.061] (7f113f94a7c0) INFO   Found 14271 derivations
[00:00:02.061] (7f113f94a7c0) INFO   Getting the metadata for packages in the Nix store
Error: "invalid type: map, expected a string at line 493167 column 10"

[louib]

@NorfairKing Thanks for reporting the issue!

This step is fetching the metadata for the packages in your store, which explains why I wasn't able to reproduce locally. You can generate the metadata information for your Nix store with the following command:

nix-env -q -a --meta --json '.*' > packages-metadata.json

If you could send that over to me by email, I could investigate further. There shouldn't be anything sensitive in there, but I'm guessing email might make you feel more at ease than attaching it in this thread.

[NorfairKing]

@louib I ran both again and found that this is the offending part;

  "nixos.javacc": {
    "meta": {
      "available": true,
      "broken": false,
      "description": "A parser generator for building parsers from grammars",
      "homepage": "https://javacc.github.io/javacc",
      "insecure": false,
      "license": {
        "deprecated": false,
        "free": true,
        "fullName": "BSD 2-clause \"Simplified\" License",
        "redistributable": true,
        "shortName": "bsd2",
        "spdxId": "BSD-2-Clause",
        "url": "https://spdx.org/licenses/BSD-2-Clause.html"
      },
      "maintainers": [
        [
          {
            "email": "limeytexan@gmail.com",
            "github": "limeytexan",
            "githubId": 36448130,
            "name": "Michael Brantley"
          }
        ]
      ],
      "name": "javacc-7.0.10",
      "outputsToInstall": [
        "out"
      ],
      "position": "/home/syd/.nix-defexpr/channels_root/nixos/pkgs/development/tools/parsing/javacc/default.nix:28",
      "unfree": false,
      "unsupported": false
    },
    "name": "javacc-7.0.10",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "javacc",
    "system": "x86_64-linux",
    "version": "7.0.10"
  },

The issue according to this error is that the list of maintainers isn't a list of strings:

"maintainers": [
        [
          { <- Error points to here
            "email": "limeytexan@gmail.com",
            "github": "limeytexan",
            "githubId": 36448130,
            "name": "Michael Brantley"
          }
        ]
      ],

[louib]

@NorfairKing thanks for the additional details, I opened https://github.com/louib/nix2sbom/pull/11 with a failing unit test. I should be able to provide a fix soon. Looks like the issue is that the maintainers field is a list of lists :open_mouth:

[NorfairKing]

@louib Excited to try it whenever you get there :D

[louib]

It looks like this package meta in particular is malformed, but was fixed in https://github.com/NixOS/nixpkgs/commit/f14b6f553a7721b963cf10048adf35d08d5d0253

[louib]

@NorfairKing I'm still pondering about how to best handle this derivation. In the meantime, I added a --metadata-path option so that you can prune this specific derivation from the pre-generated metadata file. Assuming that the metadata file is called packages-metadata.json, you can now run the following command:

nix2sbom -f .# --metadata-path packages-metadata.json

[NorfairKing]

so that you can prune this specific derivation from the pre-generated metadata file.

Almost every derivation in nixpkgs looks like this. It's a 400k line file and most of the derivations in it have maintainer lists..

[louib]

@NorfairKing lists of maintainers are supported by the parser. In the case of javacc, it was a list of lists of maintainers, which was fixed in https://github.com/NixOS/nixpkgs/commit/f14b6f553a7721b963cf10048adf35d08d5d0253.

[louib]

@NorfairKing I merged #11, which should fix this issue. Please give it another shot.