search

oss-review-toolkit / ort

A suite of tools to automate software compliance checks.
https://oss-review-toolkit.org
Apache License 2.0
1.6k stars 309 forks source link

Configurable URL replacements for the analyzer / downloader #6698

Open dgutson opened 1 year ago

dgutson commented 1 year ago

We need to set some connection configuration of the vcs in general and git in particular, for example whether to use https instead of ssh. Our current workaround is to place this in the docker entry point: IMG-20230317-WA0004

sschuberth commented 1 year ago

Similar feature requests occasionally came up in the past already, I remember @heliocastro having similar requirements. A configurable analyzer mechanism to do regex-based search & replace on URLs could also be used to replace hard-coded substitutions like not using git:// anymore for GitHub repos, see

https://github.com/oss-review-toolkit/ort/blob/2e78d0f4f756b39afb61c8f1d542e5080849ef90/utils/ort/src/main/kotlin/Utils.kt#L180-L183

fviernau commented 1 year ago

A configurable analyzer mechanism to do regex-based search & replace on URLs could also be used to replace hard-coded substitutions like not using git:// anymore for GitHub repos, see

Note that this needs to work for submodules as well. (As long as we rely on recursive cloning)

dgutson commented 1 year ago

@fviernau s/not/note/ ?

If so, indeed I think a substitution wouldn't work for automatic submodules cloning, where ORT does not control the URL.

dgutson commented 1 year ago

I found more configuration rules that we had to set, which I'm not sure that can be specified as regexes:

git config --global url."https://github.com/".insteadOf ssh://git@github.com/
git config --global url."https://github.com/".insteadOf git@github.com:
sschuberth commented 1 year ago

Thanks to https://github.com/oss-review-toolkit/ort/pull/6387 this could probably quite elegantly implemented as a special curation providers that does regex search & replaces on URLs from package metadata.

sschuberth commented 6 months ago

Note that this needs to work for submodules as well.

True, so any such URL replacement would also need to be applied by the downloader, not only the analyzer. A repo to test this with could be https://github.com/TritonDataCenter/node-http-signature/blob/v1.2.0/.gitmodules as it uses SSH / SCP URLs like url = git@github.com:davepacheco/javascriptlint for submodules.