SwiftURL ecosystem uses inconsistent format for package identifiers

[fviernau]

Looking at the vulnerabilities linked to SwiftURL packages [1], it seems that there are multiple variants how the canonical package name is constructed:

1. SwiftURL/https://github.com/grpc/grpc-swift.git
2. SwiftURL/https://github.com/apple/swift-nio-http2.git
3. SwiftURL/github.com/vapor/leaf-kit

In SwiftPM a canocial name is derived using some normalization which includes amongst others:

1. Dropping .git suffix
2. Dropping the protocol
3. Lowercasing
4. Dropping port, and user info

...see also [2]. Should osv.dev normalize the IDs of the packages and specify the normalization, so that it is straight forward to craft a query to obtain vulnerabilities for a specific swift package?

[1] https://osv.dev/list?ecosystem=SwiftURL&q= [2] https://github.com/apple/swift-package-manager/blob/24bfdd180afdf78160e7a2f6f6deb2c8249d40d3/Sources/PackageModel/PackageIdentity.swift#L345

[oliverchang]

Thanks @fviernau ! would you also be able to open an issue on https://github.com/github/advisory-database for this?

It would be great to get consensus on this from the (currently only) database exporting SwiftURL vulns.

[cuixq]

@fviernau do you know if there is any official documentation by Swift stating the normalization rules?

[sschuberth]

@fviernau do you know if there is any official documentation by Swift stating the normalization rules?

Maybe @MaxDesiatov can chime in here?

[fviernau]

@cuixq I haven't found any, but maybe I missed it.

[oliverchang]

@darakian thoughts on this?

[darakian]

@darakian thoughts on this?

On swift generally or with respect to the current inconsistency? For the latter, we've normalized our data to omit the transport (http:// or https:// are all I've seen in the wild) as well as to omit the trailing .git. https://github.com/github/advisory-database/issues/3333

In the more general case; swift should really document this. Given the presence of the transport and the .git, I assume other transports and version control schemes are allowed or planned or maybe just possible in the future, but I haven't been able to find any discussion of it.

[andrewpollock]

I've just looked at the current state of things, and I think the state of the data is consistent? Please reopen with specifics if that's not the case.

$ cat * | jq '[{"id": .id, "package": .affected | map(select(.package.ecosystem == "SwiftURL"))[] | .package.name}]'
[
  {
    "id": "GHSA-239c-6cv2-wwx8",
    "package": "github.com/apple/swift-corelibs-foundation"
  }
]
[
  {
    "id": "GHSA-2jx2-qcm4-rf9h",
    "package": "github.com/grpc/grpc-swift"
  }
]
[
  {
    "id": "GHSA-3mwq-h3g6-ffhm",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-4rhq-vq24-88gw",
    "package": "github.com/grpc/grpc-swift"
  }
]
[
  {
    "id": "GHSA-5844-q3fc-56rh",
    "package": "github.com/pubnub/swift"
  }
]
[
  {
    "id": "GHSA-5c9c-6x87-f9vm",
    "package": "github.com/facebook/zstd"
  }
]
[
  {
    "id": "GHSA-773g-x274-8qmf",
    "package": "github.com/apple/swift-nio-extras"
  },
  {
    "id": "GHSA-773g-x274-8qmf",
    "package": "github.com/apple/swift-nio-extras"
  },
  {
    "id": "GHSA-773g-x274-8qmf",
    "package": "github.com/apple/swift-nio-extras"
  }
]
[
  {
    "id": "GHSA-7fj7-39wj-c64f",
    "package": "github.com/apple/swift-nio"
  },
  {
    "id": "GHSA-7fj7-39wj-c64f",
    "package": "github.com/apple/swift-nio"
  },
  {
    "id": "GHSA-7fj7-39wj-c64f",
    "package": "github.com/apple/swift-nio"
  }
]
[
  {
    "id": "GHSA-9cfh-vx93-84vv",
    "package": "github.com/vapor/postgres-nio"
  }
]
[
  {
    "id": "GHSA-c2cc-3569-6jh2",
    "package": "github.com/weichsel/ZIPFoundation"
  }
]
[
  {
    "id": "GHSA-ccw9-q5h2-8c2w",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-frg3-gpcx-968f",
    "package": "github.com/apple/swift-nio-ssl"
  }
]
[
  {
    "id": "GHSA-g454-wj9r-jpg4",
    "package": "github.com/marmelroy/Zip"
  }
]
[
  {
    "id": "GHSA-gcj9-jj38-hwmc",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-gpgx-whwh-r297",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-jq43-q8mx-r7mq",
    "package": "github.com/migueldeicaza/SwiftTerm"
  }
]
[
  {
    "id": "GHSA-mgc4-wqv7-4pxm",
    "package": "github.com/apple/swift-nio"
  },
  {
    "id": "GHSA-mgc4-wqv7-4pxm",
    "package": "github.com/apple/swift-nio"
  }
]
[
  {
    "id": "GHSA-pgfx-g6rc-8cjv",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-pqwh-c2f3-vxmq",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-pv7r-9vjg-g3f9",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-q36x-r5x4-h4q6",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-qppj-fm5r-hxr3",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-qvxg-wjxc-r4gg",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-r6r4-5pr8-gjcp",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-r6ww-5963-7r95",
    "package": "github.com/grpc/grpc-swift"
  }
]
[
  {
    "id": "GHSA-rv3x-xq3r-8j9h",
    "package": "github.com/vapor/leaf-kit"
  }
]
[
  {
    "id": "GHSA-rxmj-hg9v-vp3p",
    "package": "github.com/grpc/grpc-swift"
  }
]
[
  {
    "id": "GHSA-v3r5-pjpm-mwgq",
    "package": "github.com/swift-server/async-http-client"
  },
  {
    "id": "GHSA-v3r5-pjpm-mwgq",
    "package": "github.com/swift-server/async-http-client"
  },
  {
    "id": "GHSA-v3r5-pjpm-mwgq",
    "package": "github.com/swift-server/async-http-client"
  },
  {
    "id": "GHSA-v3r5-pjpm-mwgq",
    "package": "github.com/swift-server/async-http-client"
  }
]
[
  {
    "id": "GHSA-vcvg-xgr8-p5gq",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-vj2m-9f5j-mpr5",
    "package": "github.com/vapor/vapor"
  }
]
[
  {
    "id": "GHSA-vxvm-qww3-2fh7",
    "package": "github.com/mongodb/mongo-swift-driver"
  }
]
[
  {
    "id": "GHSA-w3f6-pc54-gfw7",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-wfvq-p7qf-vv64",
    "package": "github.com/apple/swift-nio-http2"
  }
]
[
  {
    "id": "GHSA-x768-cvr2-345r",
    "package": "github.com/swift-server/swift-prometheus"
  }
]