cassamajor commented 1 year ago

Similar to #1, but in my case it is remote repositories. I connected Sourcegraph to my GitHub account and it only picked up two repositories (one private, one public) when I selected the option "Add all repositories from selected organizations or users".
For the "Add all repositories from selected organizations or users" option, in the text field, the organization name is not actually saved when entered. To have the organization name persist, I had to switch to the JSONC Editor and add it manually.
The "Add selected repositories" option clearly shows other repositories in the Organization that were not picked up by the "Add all repositories from selected organizations or users" option.
I cannot seem to remove a connection.
- If I try to remove a connection from the Setup Wizard. after a certain period of time, I received an error: "Context deadline exceeded".
- If I try to remove a connection from Manage code hosts in the advanced settings, they are visually removed from the list but still exist. They can be seen when the page is refreshed or click "Show more".
When I clicked "stop" from the Sourcegraph UI, the app then gets stuck in showing "Starting Sourcegraph..."

Additional Information

Type of access token used: "Fine-grained personal access tokens"
App version: 2023.03.23+209542.7216ba
Installation method: https://storage.googleapis.com/sourcegraph-app-releases/latest/Sourcegraph%20App.dmg

cassamajor commented 1 year ago

From the logs:

maximum number of private repositories included in license (1) reached"}

I guess that answers a few questions..

cassamajor commented 1 year ago

Another error from the logs:

ERROR   service run/util.go:47  github.com/sourcegraph/sourcegraph/enterprise/cmd/executor/internal/run.newQueueTelemetryOptions    Failed to get docker version    {"Resource": {"service.name": "sourcegraph", "service.version": "2023.03.23+209542.7216ba", "service.instance.id": "REDACTED"}, "Attributes": {"error": "'docker version -f {{.Server.Version}}': : exec: \"docker\": executable file not found in $PATH"}}

When I run docker version in my terminal, the executable is definitely found.

cassamajor commented 1 year ago

And looking around the at files:

site-config.json has the following data

{
    "auth.providers": [
        { "type": "builtin" }
    ],
    "externalURL": "http://127.0.0.1:3080",

    "codeIntelAutoIndexing.enabled": true,
    "codeIntelAutoIndexing.allowGlobalPolicies": true,
    "executors.frontendURL": "http://127.0.0.1:3080",
}

Output from jq when trying to read the file:

jq . /Users/REDACTED/Library/Application\ Support/sourcegraph-sp/site-config.json
parse error: Expected another key-value pair at line 10, column 1

Same with Firefox's built-in JSON reader: SyntaxError: JSON.parse: expected double-quoted property name at line 10 column 1 of the JSON data

slimsag commented 1 year ago

Thank you for the detailed bug report @cassamajor! We really appreciate it.

maximum number of private repositories included in license (1) reached"}

This is definitely not the intended behavior; it should be unlimited local & remote repositories. I think a last minute change may have broken this behavior; and this is the primary issue here.

As a workaround you should be able to still add as many local repositories as you like, of course this is a poor workaround as it wouldn't sync those etc. for you automatically.

We'll investigate and get back to you.

When I clicked "stop" from the Sourcegraph UI, the app then gets stuck in showing "Starting Sourcegraph..."

This should be fixed in the latest version.

points 1-4

cc @vovakulikov @gl-srgr these seem like they may be indicating some error handling missing in the new setup wizard? or any idea why there would be a disconnect between what the frontend is showing & backend reporting? It seems clear the underlying issue here is somewhere on the backend, but looks concerning from a pure frontend POV.

exec: \"docker\": executable file not found in $PATH"

cc @peterguy does the macOS app launcher not set up the PATH correctly? I also see this clear as day in my logs / can reproduce it. This means precise code intel & batch changes are broken in the macOS app bundle version.. :/

The homebrew version would not suffer from this issue.

jq parsing errors Same with Firefox's built-in JSON reader:

Sourcegraph uses JSONC for its configuration (same as VS Code); it is a variant of JSON which allows for more lenient syntax like trailing commas and comments - so normal JSON parsers will not always be able to parse it.

gl-srgr commented 1 year ago

Thank you for the writeup @cassamajor! I've reproduced some of the reported issues with the latest dmg (dl'd from the /sourcegraph-app-releases/latest link that you used, so potentially later than your version). Will write up notes in a separate comment for the team.

gl-srgr commented 1 year ago

@slimsag @vovakulikov I see these issues being external to the setup wizard so we should resolve backend only (i.e. external services management). The setup wizard's Context deadline exceeded and the inability to remove code host is representative of the non-setup wizard operations and issues so the setup wizard is not acting unexpectedly.

I've reproduced with the dmg, but I'm not able to reproduce with running my brew installed sourcegraph app (which is a very recent version). I get some weird behavior with the brew sg app sometimes that are similar to the reported issue, but these issues are less consistent and could be caused by the dmg issue (I use non-embedded postgresql and in both cases my sourcegraph will use the same pgsql vars so for all intents and purposes they are sharing the state and I swap from dmg to brew version to fix the hung external service).

Sourcegraph app and sourcegraph launcher appear to hang:

In App I don't see the remote code host external service status updating beyond the sync'ing spinner icon which is spinning indefinitely (this I've confirmed). When I say a status is just the spinning icon indefinitely for an external service, it is when the license is exceeded by that external service through the repositoryQuery value affiliated. In my testing with the brew installed sourcegraph I will see a lot of errors if there are too many repositories associated with my token, but it can eventually yield a Red status for my external service and not the spinning
In Launcher I don't see the sourcegraph.log updating after the hanging status external service is added. When I click Stop in Launcher I also get the starting sourcegraph... message that OP mentioned. With this status the launcher does not have Stop buttona so I need to exit Launcher and reopen to start Sourcegraph app.

Since this is only for the MacOS Dmg, maybe there are communications being limited/restricted e.g. request to pgsql to update/delete external service status cc: @peterguy

@slimsag if you need resources to help look into this then I will see if I can continue investigating. I'm happy to continue but took this time to write up my current findings since this could be a mix of launcher issues, license issues, and dmg.

gl-srgr commented 1 year ago

An additional clarification: When I configured my external service code host connection to use org: ["sg-test"] in lieu of the token affiliated repositoryQuery, I did not get the issue. The org does have more repositories than the license allows, so I get a red status for my external service, but it does not hang/spin.

gl-srgr commented 1 year ago

Another relevant log I believe:

When I stop app via the Launcher, and then quit the Launcher because it is hanging, I can swap to running the brew installed sourcegraph to try and resolve the state. It seems to still have the issue when I start the brew sourcegraph the first time, and then resolves the second time. The second run has a warning that the first does not:

1680489192770010000 WARN repo-updater.syncer.syncWorker.repo.sync.workerstore.Store.repo.sync.worker.Resetter dbworker/resetter.go:117 github.com/sourcegraph/sourcegraph/internal/workerutil/dbworker.(*Resetter[...]).Start Reset stalled record back to 'queued' state {"Resource": {"service.name": "sourcegraph", "service.version": "2023.03.21+209158.f11f52", "service.instance.id": "MacBook-Pro-7.local"}, "Attributes": {"name": "repo_sync_worker_resetter", "id": 49, "timeSinceLastHeartbeat": 265.881587}}

I didn't see a reset message in the sourcegraph-sp/sourcegraph.log so resetter might give us some clues.

gripssg commented 1 year ago

I am not sure if this is the same issue or if Sourcegraph has changed their plans and offerings since last time we used it. We have an outdated installation (5.0.2) that we're looking to replace with a new one. I used the installation script for Ubuntu and after adding a group in our projects query in code host config, it only clones one of the repos in the group, presenting the same message as the title: "maximum number of private repositories included in license (1) reached" Our code host is a self-hosted Gitlab installation. Is this a limitation of the Enterprise Free edition? Am I missing something obvious?

jaypan13 commented 1 year ago

Today I encountered the same error. Had downloaded latest app from here - https://storage.googleapis.com/sourcegraph-app-releases/latest/Sourcegraph%20App.dmg and tried to sync my selfhosted gitlab repos and got the same error "maximum number of private repositories included in license (1) reached" in log files. I cleared directories before installing latest app using below instructions. rm -rf $HOME/.sourcegraph-psql rm -rf $HOME/Library/Application\ Support/sourcegraph-sp rm -rf $HOME/Library/Caches/sourcegraph-sp

Latest app shows version as - 2023.03.23+209542.7216ba.

slimsag commented 1 year ago

This should be resolved in the latest version of the Cody app that we announced just yesterday. You can try it here: https://sourcegraph.com/get-cody

If you encounter any issues or have feedback, please let us know!

sourcegraph / app

Remote repos should be unlimited, but "maximum number of private repositories included in license (1) reached" #4

Additional Information