search

sourcegraph / scip

SCIP Code Intelligence Protocol
Apache License 2.0
262 stars 31 forks source link

chore: Remove convert subcommand & dep on SG monorepo #275

Closed varungandhi-src closed 2 months ago

varungandhi-src commented 2 months ago

We no longer need SCIP to LSIF conversion because the Sourcegraph backend has been able to accept SCIP natively since several major releases, and we do not have any use for LSIF.

Fixes https://linear.app/sourcegraph/issue/GRAPH-824

Test plan

Covered by CI

phikai commented 2 months ago

@varungandhi-src @antonsviridov-src I understand that Sourcegraph may no longer need the convert functionality, but other providers (like GitLab) don't natively support SCIP. The convert functionality provides a bridge for users to continue to be able to index their code.

Is it possible to have this functionality restored?

varungandhi-src commented 2 months ago

@phikai, for that functionality, you can either:

  1. Continue using the old scip CLI release which supports SCIP to LSIF conversion. Since we don't make backwards-incompatible changes to the SCIP schema, existing releases of the SCIP CLI are forwards-compatible with future indexes.

  2. Write a small library which handles this conversion yourself, based on the old code here (which is Apache 2.0 licensed, and mostly contains the core logic involved in the conversion). The only external dep that might potentially be problematic was on the Sourcegraph monorepo's types such as reader.Element, but those are very simple (you can still see the code in the public snapshot of the Sourcegraph codebase -- they're directly derived from the LSIF spec):

    type Element struct {
    ID      int
    Type    string
    Label   string
    Payload any
}

When adding new features to SCIP, we have not been making any attempt at trying to map those to LSIF, so I don't think it makes sense for us to act as-if we're still maintaining that code, since we're actually not. We have limited bandwidth on the team, and if I'm being honest, the LSIF related legacy code in Sourcegraph has been slowing us down, so I hope you understand why we might want to simplify things.

alsmnn commented 1 month ago

@varungandhi-src, you can't use the old scip CLI release anymore. If you try to run go build ./cmd/scip it tries to download something from github.com/sourcegraph/sourcegraph/lib, which can't be found anymore. My guess is that this is now a privat repo?

bindings/go/scip/convert.go:19:2: github.com/sourcegraph/sourcegraph/lib@v0.0.0-20220511160847-5a43d3ea24eb: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:20:2: github.com/sourcegraph/sourcegraph/lib@v0.0.0-20220511160847-5a43d3ea24eb: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:21:2: github.com/sourcegraph/sourcegraph/lib@v0.0.0-20220511160847-5a43d3ea24eb: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:22:2: github.com/sourcegraph/sourcegraph/lib@v0.0.0-20220511160847-5a43d3ea24eb: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found

Any other thoughts on how to use the old scip CLI release?

varungandhi-src commented 1 month ago

@alsmnn for older versions, you can download the prebuilt binary releases from https://github.com/sourcegraph/scip/releases

My guess is that this is now a privat repo?

Yes. If you need to build from source, you could fork the scip repo and point the scip commit of interest to the matching commit in https://github.com/sourcegraph/sourcegraph-public-snapshot