Improving dev workflow around GH check indicators for failed LSIF jobs

sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody

https://sourcegraph.com

Other

10.12k stars 1.29k forks source link

Improving dev workflow around GH check indicators for failed LSIF jobs #10451

Closed rvantonder closed 3 years ago

rvantonder commented 4 years ago

When an LSIF job fails for whatever reason, the PR build at a glance fails as a whole (not all checks passed, red X). The overhead of checking whether the failure is due to a broken build (merge blocking) versus LSIF (not merge blocking) can be a pain point. Resolving this in part comes down to not having full control over the nature of GH's check logic or visual indicators.

@gbrik says (paraphrased):

are we frequently getting red Xs on SG builds? when i added the indexing to 40 of our repos the other day i didn't get a single transient error so my impression is that it's fairly robust. i'd rather suffer the red Xs on our projects as motivation to maintain reliability, but if they're really frequent we could reconsider

Let's use this issue to track cases of LSIF failures to inform whether this is a recurrent thing worth thinking about.

rvantonder commented 4 years ago

Issue cropped up here: https://github.com/sourcegraph/sourcegraph/pull/10572/checks?check_run_id=664502349

Run src lsif upload -github-token=***
Repository: github.com/sourcegraph/sourcegraph
Commit: 83a5e262b21cfd7acdcc4ec82678035dc6484f0f
File: ./dump.lsif
Root: cmd/precise-code-intel
error: 500 Internal Server Error

no precise-code-intel-api-server endpoints could be found (this may indicate more precise-code-intel-api-server replicas are needed, contact support@sourcegraph.com for assistance)
##[error]Process completed with exit code 1.

rvantonder commented 4 years ago

Separately, this issue cropped up with users forking a repo and Sourcegraph.com returning a 500 code because it was busy cloning the fork.

shrouxm commented 4 years ago

Separately, this issue cropped up with users forking a repo and Sourcegraph.com returning a 500 code because it was busy cloning the fork.

@rvantonder implementing #9700 this week which will fix that specific case, also filed #10576: re changing default behaviour of indexing on forks (for forks of public repos, don't do it by default)

rvantonder commented 4 years ago

https://github.com/sourcegraph/sourcegraph/pull/10914/checks?check_run_id=700718622

Repository: github.com/sourcegraph/sourcegraph Commit: 2ae4cf6e37b152e7c9c6b0f652f79a6c2c0da2d1 File: ./dump.lsif Root: browser error: 401 Unauthorized

2 errors occurred:

given repository sourcegraph/sourcegraph not listed in installed repositories

you do not have write permission to the repository

Related to GH being down maybe?

shrouxm commented 4 years ago

'twas, https://sourcegraph.slack.com/archives/C07KZF47K/p1590175330444900

rvantonder commented 4 years ago

big sad. putting up https://github.com/sourcegraph/lsif-upload-action/pull/14 as option for external projects (don't think we should enable that on our repos, good to know about what affects uploads)

rvantonder commented 4 years ago

Another one, not sure why

Repository: github.com/sourcegraph/sourcegraph
Commit: 7552f28ed35a5bc2c9839e78e7f2d51dc3ce3916
File: ./dump.lsif
Root: 
Post "https://sourcegraph.com/.api/lsif/upload?commit=7552f28ed35a5bc2c9839e78e7f2d51dc3ce3916&github_token=***&repository=github.com%2Fsourcegraph%2Fsourcegraph": read tcp 172.18.0.2:60140->104.26.9.187:443: read: connection reset by peer
##[error]Process completed with exit code 1.

Job

rvantonder commented 4 years ago

Another

Run src lsif upload -github-token=***
Repository: github.com/sourcegraph/sourcegraph
Commit: 0ab59c2994d191be02e251ee43bc260aa9c4af8a
File: ./dump.lsif
Root: web
error: 500 Internal Server Error

git command [git rev-parse 0ab59c2994d191be02e251ee43bc260aa9c4af8a^0] failed (stderr: ""): Post "http://gitserver-4.gitserver:3178/exec": dial tcp: lookup gitserver-4.gitserver on 10.55.240.10:53: no such host

job

rvantonder commented 4 years ago

 Upload LSIF data29s
##[error]Process completed with exit code 1.
Run src lsif upload -github-token=***
Repository: github.com/sourcegraph/sourcegraph
Commit: 4a68001a2d73d3aaea1ccd0c9501ca75a40dfb93
File: ./dump.lsif
Root: 
error: 502 Bad Gateway

<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

job Probably when pods bounce

macraig commented 3 years ago

Closing due to inactivity.