fail to install extension that has built-in extension as dependency

garylavayou commented 3 months ago

1. Brief Description

When installing extensions from the self-hosted code marketplace, the code-server log shows that the built-in vscode.git-base is not compatible with Code 1.79.2, which fails the installation of extensions that depend on it.

2. Details

As a fallback, I try to install the same extension from the default registry, i.e., https://open-vsx.org, and it succeeds. So, this should not be a problem with code-server itself.

Checking the extension data of vscode.git-base from open-vsx.org, we can find that this extension requires a fixed version of vscode 0.10.0.

{ "engine": { "vscode": "0.10.0" } }

While most extension will declare the vscode version in a foreward compatible manner, like {"vscode": "^1.82.0"} with gitlab.gitlab-workflow.

Hence, as vscode.git-base requires vscode 0.10.0, any recent version of Code-Server will fail for the dependency check when installing from the self-hosted marketplace. In contrast, open-vsx.org seems can handle this situation, and finally succeeds in the installation of related extensions.

Though we can browse vscode.git-base from https://open-vsx.org, but we cannot directly install it.
code-server --force --install-extension vscode.git-base
Installing extensions...
Extension 'vscode.git-base' not found.
Make sure you use the full extension ID, including the publisher, e.g.: ms-dotnettools.csharp
Failed Installing Extensions: vscode.git-base
So, I guess that maybe https://open-vsx.org have hide those built-in extensions from installation APIs. And thus, the dependency check of those built-in extensions is also skipped.

2.1. Code-Server Version

Code server version: 4.14.1
Code version: 1.79.2

2.2. Related Extension

The following related extension depends on the built-in extension vscode.git-base,

waderyan.gitblame
waderyan.gitblame@10.3.0
gitlab.gitlab-workflow
gitlab.gitlab-workflow@3.93.1

3. Related Logs

Logs with Code 1.79.2:

[2024-08-09 08:04:18] install-extensions.sh(44) [info]: code version [1.79.2].
[2024-08-09 08:04:18] install-extensions.sh(126) [info]: using marketplace.coder.garyhome.net to install extensions.
++ code-server --extensions-dir /opt/code-server/extensions --force --install-extension gitlab.gitlab-workflow@3.93.1
Installing extensions...
Installing extension 'gitlab.gitlab-workflow' v3.93.1...
Can't install 'vscode.git-base' extension because it is not compatible with the current version of code-server (version 1.79.2).
Failed Installing Extensions: gitlab.gitlab-workflow
ls: cannot access '/opt/code-server/extensions/gitlab.gitlab-workflow-3.93.1*': No such file or directory
[2024-08-09 08:04:19] install-extensions.sh(282) [warning]: install extension gitlab.gitlab-workflow@3.93.1 from local code marketplace failed! fallback to install from open-vsx.org.
Installing extensions...
Installing extension 'gitlab.gitlab-workflow' v3.93.1...
Extension 'gitlab.gitlab-workflow' v3.93.1 was successfully installed.

++ code-server --extensions-dir /opt/code-server/extensions --force --install-extension waderyan.gitblame@10.3.0
Installing extensions...
Installing extension 'waderyan.gitblame' v10.3.0...
Can't install 'vscode.git-base' extension because it is not compatible with the current version of code-server (version 1.79.2).
Failed Installing Extensions: waderyan.gitblame
ls: cannot access '/opt/code-server/extensions/waderyan.gitblame-10.3.0*': No such file or directory
[2024-08-09 08:04:38] install-extensions.sh(282) [warning]: install extension waderyan.gitblame@10.3.0 from local code marketplace failed! fallback to install from open-vsx.org.
Installing extensions...
Installing extension 'waderyan.gitblame' v10.3.0...
Extension 'waderyan.gitblame' v10.3.0 was successfully installed.

Logs with Code 1.90.2 (with stack trace information):

[2024-08-10 09:32:44] install-extensions.sh(38) [info]: code version [1.90.2].
[2024-08-10 09:32:44] install-extensions.sh(122) [info]: using marketplace.coder.garyhome.net to install extensions.
++ code-server --extensions-dir /opt/code-server/extensions --force --install-extension gitlab.gitlab-workflow
Installing extensions...
Installing extension 'gitlab.gitlab-workflow'...
Error while installing extensions: Can't install 'vscode.git-base' extension because it is not compatible with the current version of code-server (version 1.90.2).
Can't install 'vscode.git-base' extension because it is not compatible with the current version of code-server (version 1.90.2).
[2024-08-10T09:32:45.537Z] error parent:100 Uncaught exception: Canceled
[2024-08-10T09:32:45.537Z] error parent:100 Canceled: Canceled
at m.value (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:92:13240)
at o.y (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:88:680)
at o.z (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:88:755)
at o.fire (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:88:977)
at k.cancel (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:88:5443)
at R.cancel (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:88:5853)
at Object.cancel (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:92:13383)
at oe.cancel (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:171:41047)
at M.I (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:171:30102)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async M.installGalleryExtensions (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:171:23498)
at async a.g (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:144:614)
at async a.installExtensions (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:143:9848)
at async /usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:184:75460
at async W.run (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:184:75225)
at async ee (/usr/lib/code-server/lib/vscode/out/vs/server/node/server.main.js:184:77675)
ls: cannot access '/opt/code-server/extensions/gitlab.gitlab-workflow*': No such file or directory
[2024-08-10 09:32:45] install-extensions.sh(278) [warning]: install extension gitlab.gitlab-workflow from local code marketplace failed! fallback to install from open-vsx.org.
Installing extensions...
Installing extension 'gitlab.gitlab-workflow'...
Extension 'gitlab.gitlab-workflow' v5.4.0 was successfully installed.

4. Feature Requests

code-server may provide an option to not install dependencies.
code-marketplace may need to handle the requests to install built-in extensions.

Kira-Pilot commented 3 months ago

let's try and reproduce

code-asher commented 3 months ago

I tried to reproduce with these steps:

$ code-marketplace --extensions-dir ./extensions add https://gitlab.com/api/v4/projects/5261717/packages/generic/dist-desktop/v5.9.0/gitlab-workflow-5.9.0.vsix
Unpacked GitLab.gitlab-workflow@5.9.0 to /home/coder/code-marketplace/extensions/GitLab/gitlab-workflow/5.9.0
  - GitLab.gitlab-workflow@5.9.0 has 1 dependency
    - vscode.git
  - GitLab.gitlab-workflow@5.9.0 is not in a pack
$ export EXTENSIONS_GALLERY={"serviceUrl":"http://localhost:3001/api", "itemUrl":"http://localhost:3001/item", "resourceUrlTemplate": "http://localhost:3001/{publisher}/{name}/{path}"}
$ code-server --install-extension gitlab.gitlab-workflow
Installing extensions...
Error while installing extensions: connect ECONNREFUSED 127.0.0.1:3001
$ code-marketplace --extensions-dir ./extensions server &
2024-08-28 20:42:21.566 [info]  Started API server  address=127.0.0.1:3001
$ code-server --install-extension gitlab.gitlab-workflow
Installing extensions...
Installing extension 'gitlab.gitlab-workflow'...
Extension 'gitlab.gitlab-workflow' v5.9.0 was successfully installed.

But I was able to install the gitlab.gitlab-workflow extension (both from the UI and from the CLI) without any errors.

Did you add the built-in extensions to the marketplace? Maybe that is what is making code-server try to install them, where normally the extension would not show up in the query so code-server would not try to install it. I think there is no reason to add the built-in extensions to the marketplace (correct me if I am wrong) so they can be skipped.

code-asher commented 3 months ago

If that is what happened, we could improve the experience by one or more of the following:

When displaying the extension's dependencies, if the extension is built-in then say so to discourage adding it, for example:

Unpacked GitLab.gitlab-workflow@5.9.0 to /home/coder/code-marketplace/extensions/GitLab/gitlab-workflow/5.9.0
  - GitLab.gitlab-workflow@5.9.0 has 1 dependency
    - vscode.git (built-in)
  - GitLab.gitlab-workflow@5.9.0 is not in a pack

Prevent built-in extensions from being added at all.
Filter built-in extensions out of the query.

But I am not sure how to detect if an extension is built-in. Maybe we can rely on the 0.10.0 marker, or maybe we would have to maintain a hard-coded list.

code-asher commented 3 months ago

Oh actually there is a tag in the manifest indicating it is built-in, so we could use that to implement 2 and/or 3 at least. With 1 all we have is the name of the extension.

Although...maybe that is just added by Open VSX. We can think about it more once we know how/if the extension was added to the marketplace. Was it added by downloading from Open VSX?

code-asher commented 3 months ago

For what it is worth, the VS Code marketplace does not seem to contain vscode.git at all, but Open VSX does, so this might just be a weird thing that Open VSX does differently.

Kira-Pilot commented 2 months ago

@garylavayou We would recommend not adding built-in extensions right now (they exist already). If you have a use-case that necessitates this workflow, please feel free to re-open and tell us about it!

garylavayou commented 3 days ago

Oh actually there is a tag in the manifest indicating it is built-in, so we could use that to implement 2 and/or 3 at least. With 1 all we have is the name of the extension.

Although...maybe that is just added by Open VSX. We can think about it more once we know how/if the extension was added to the marketplace. Was it added by downloading from Open VSX?

@code-asher Yes, I have found that a built-in extension has the namespace of vscode. So, when I try to resolve and add dependencies to the marketplace, I can prevent those built-in extensions from being added.

garylavayou commented 3 days ago

@garylavayou We would recommend not adding built-in extensions right now (they exist already). If you have a use-case that necessitates this workflow, please feel free to re-open and tell us about it!

@Kira-Pilot I have removed those built-in extensions from marketplace, and the installation then will not produce the error.

coder / code-marketplace