Feature request: Reusable sets/blocks of extensions

[binaryben]

I have some "core" profiles that my other language specific profiles build on. See simplified example below:

graph TD
  Lite-->Full
  Full-->Node
  Full-->Python

I have it set so that the full core profile installs all extensions, but disables them. This results in a lot of duplication though when I need to enable a set of extensions already installed in a derived profile. Similarly, I find that when I switch from a more complete profile to the lite profile, the extensions remain installed unless explicitly set in the uninstall section of the Lite profile - again, more duplication.

I'm not sure on the best API for it (I've suggested an idea in the details section below), but it would be handy to be able to import a group of extensions into the [dis|en]abled and uninstall subsections.

This would also make it very easy to set up more complicated project specific profiles (i.e., working in a PHP/React monolithic project with some random .NET API thrown in)

Potential data structure

**Directory structure** ``` . ├── extensions/ │ ├── node.extensions.yml │ └── python.extensions.yml └── profiles/ ├── lite/ │ ├── data/ │ │ ├── extensions.yml │ │ ├── keybindings.json │ │ └── settings.json │ └── .sync.yml ├── full/ │ ├── data/ │ │ └── extensions.yml │ └── profile.yml ├── node/ │ ├── data/ │ │ └── extensions.yml │ └── profile.yml └── python/ ├── data/ │ └── extensions.yml └── profile.yml ``` **./extensions/node.extensions.yml** ```yaml group: - id: bengreenier.vscode-node-readme uuid: 40d13ed9-34ed-423f-acc1-95d0fe3aa2fa - id: christian-kohler.npm-intellisense uuid: dff6b801-247e-40e9-82e8-8c9b1d19d1b8 ``` **./profiles/lite/data/extensions.yml** ```yaml enabled: - id: aaron-bond.better-comments uuid: 7a0110bb-231a-4598-aa1b-0769ea46d28b - id: alefragnani.Bookmarks uuid: b689fcc8-d494-4dbf-a228-2c694a578afc # ... uninstall: - include: node - include: python ``` **./profiles/full/data/extensions.yml** ```yaml disabled: - include: node - include: python enabled: - id: GitHub.codespaces uuid: 4023d3e5-c840-4cdd-8b54-51c77548aa3f - id: GitHub.copilot uuid: 23c4aeee-f844-43cd-b53e-1113e483f1a6 # ... ``` **./profiles/node/data/extensions.yml** ```yaml disabled: [] enabled: - include: node ```

[daiyam]

Hi,

Thanks for pushing the profile feature.

Similarly, I find that when I switch from a more complete profile to the lite profile, the extensions remain installed unless explicitly set in the uninstall section of the Lite profile - again, more duplication.

Seems that there is an oversight in my code...

So you would like to manually manage the extensions.yml. What about if you manage them directly in your Settings? (syncSettings.extensions, you could keep the same settings.json but the switch will be done with attributes)

Are you settings public so I can test directly on it?

[binaryben]

Here is my work in progress for managing my extensions across editors/languages. As you'll see, it's getting quite complex. The two feature requests submitted would help simplify what I am trying to achieve a lot.

Is syncSettings.extensions an existing option? I haven't seen documentation for it but that would work fine. Although, I do like the yaml format for extensions.

What scope to ENV come from? I could go as far as having one profile and using Environment Variables to enable language groups as well rather than needing different profiles.

[daiyam]

For syncSettings.extensions, no, it's an idea/proposal.

The ENV variables are coming from process.env (Toogle Developer Tools, run process.env). So it should be your Environment Variables.

[binaryben]

Hmm, having a small think about it then - I'm not sure a proposed syncSettings.extensions option would solve the duplication issue at hand. I'd still have to write the extensions into enabled/disabled sections and switch them using the attributes.

[binaryben]

So you would like to manually manage the extensions.yml.

Just to answer this - sorry I missed it earlier. Not necessarily. I'd be quite happy to let Sync Settings manage it for me. But I am happy to as I am if it means I can use extensions available in one marketplace but not another.

[daiyam]

Hmm, having a small think about it then - I'm not sure a proposed syncSettings.extensions option would solve the duplication issue at hand. I'd still have to write the extensions into enabled/disabled sections and switch them using the attributes.

I was thinking to make syncSettings.extensions so that you can replicate you proposed structure into it. It won't be a simple list.

[daiyam]

I can use extensions available in one marketplace but not another.

You could already use Saved Extensions. But I think syncSettings.extensions with the new variable marketplace will be more handy.

[daiyam]

Not necessarily. I'd be quite happy to let Sync Settings manage it for me.

I might just need to exclude all the extensions found in syncSettings.extensions from the extension.yml so both can work together...

[binaryben]

I was thinking to make syncSettings.extensions so that you can replicate you proposed structure into it. It won't be a simple list.

Fair enough. I'm open minded to solutions. My main concern is having to manage multiple copies of the same sets of extensions because it is error prone (DRY). I'd be happy to put it in settings.json if that is easier than adding conditionals/attributes to extensions.yml files. Especially if it allows the potential for autoloading dotenv tools to specific which language groups to install using the env attribute.

You could already use Saved Extensions.

Yeh, I'm planning to utilise this with Git LFS for extensions that don't exist in Open VSX Registry at all. But it won't be ideal for extensions that do exist in both marketplaces but exist under different ID's

But I think syncSettings.extensions with the new variable marketplace will be more handy.

It would be especially handy if one definition of an extension could supply alternative IDs based on the marketplace.

Please let me know if there is any way I can assist in getting this working, even if it is just feedback on a proposed solution

[daiyam]

Please let me know if there is any way I can assist in getting this working, even if it is just feedback on a proposed solution

I will definitively give you betas to test.

[daiyam]

The doc for attributes isn't complete, it missing the rewrite-enable/rewrite-disable and rewrite-next-line. You can find it at my lib (https://github.com/daiyam/node-jsonc-preprocessor#rewrite-enablerewrite-disable)

[binaryben]

So to solve for mismatched ID's across marketplaces and for excluding proprietary/unavailable extensions from VSCodium, you're imagining something like:

{
  "syncSettings.extensions": [
    {
      // #rewrite-enable
      // #if(markeplace="open-vsx")
      "id": "mjmlio.vscode-mjml",
      "uuid": "ce7683f9-a9fd-4fd5-8524-eb2c362b84a7",
      // #else
      "id": "attilabuti.vscode-mjml",
      "uuid": "efc33846-2498-43d7-9699-47d0a7741345",
      // #endif
      // #rewrite-disable
    },
    // #if(editor="visual studio code")
    {
      "id": "GitHub.codespaces",
      "uuid": "4023d3e5-c840-4cdd-8b54-51c77548aa3f",
    },
    // #endif
    // #if(os="mac")
    {
      "id": "sebsojeda.vscode-apple-music",
      "uuid": "48c4d108-0dd6-4404-8c20-c6f62c59950c",
    }
    // #endif
  ]
}

I'd be keen for that 👍🏼 Then I'd just need a way to include blocks/groups of extensions.

Could your node-jsonc-preprocessor be extended to include an // #import directive? It would be ideal to be able to manage the language groups in seperate files to make maintaining them easier.

[daiyam]

#rewrite-enable is for replace the variable #{variable_name} with its value. I need to improve the doc...

Maybe something like:

{
  "syncSettings.extensions": [
    // #if(markeplace="open-vsx")
    // "mjmlio.vscode-mjml",
    // #else
    // "attilabuti.vscode-mjml",
    // #endif

    // #enable(editor="visual studio code")
    // "GitHub.codespaces",

    // #enable(os="mac")
    // "sebsojeda.vscode-apple-music",

    // #rewrite-next-line
    // { "group": "#{env.DEV_LANG}" },
  ],
  "syncSettings.extensions.groups": {
    "node": [
      "bengreenier.vscode-node-readme",
      "christian-kohler.npm-intellisense",
    ],
    "python": [
    ],
  }
}

The uuid isn't used.

On Visual Studio Code on macOS, forenv.DEV_LANG="node", it will generate:

{
  "syncSettings.extensions": [
    // #if(markeplace="open-vsx")
    // "mjmlio.vscode-mjml",
    // #else
    "attilabuti.vscode-mjml",
    // #endif

    // #enable(editor="visual studio code")
    "GitHub.codespaces",

    // #enable(os="mac")
    "sebsojeda.vscode-apple-music",

    // #rewrite-next-line
    // { "group": "#{env.DEV_LANG}" },
    { "group": "node" },
  ],
  "syncSettings.extensions.groups": {
    "node": [
      "bengreenier.vscode-node-readme",
      "christian-kohler.npm-intellisense",
    ],
    "python": [
    ],
  }
}

[daiyam]

maybe replace

    // #rewrite-next-line
    // { "group": "#{env.DEV_LANG}" },

with

    // #rewrite-next-line
    // ["#{env.DEV_LANG}"],

so multiple groups can be included

[binaryben]

Looks nearly perfect to me.

so multiple groups can be included

Was gonna add this, but you caught it first. Only thing I could add right now is that a proposed DEV_LANG might be set to DEV_LANG=node,react,csharp,php per example in the OP.

If easier to implement in Sync Settings, it might be simpler to use, ENABLE_NODE_EXTS, ENABLE_REACT_EXTS, etc?

[daiyam]

@binaryben Sorry for the long delay, I'm thinking to add support to multiple marketplaces. I will see what I can do before the end of the month...

[daiyam]

I will move those functionalities to a new extension but sync-settings will be aware of those extensions and avoid to sync them.

[daiyam]

@binaryben The new extension is available at: https://github.com/zokugun/vscode-vsix-manager

You can do something like:

{
  "vsix.sources": {
    "ms": {
      "kind": "marketplace",
      "serviceUrl": "https://marketplace.visualstudio.com/_apis/public/gallery",
    },
    "opn": {
      "kind": "marketplace",
      "serviceUrl": "https://open-vsx.org/vscode/gallery",
    },
  },
  "vsix.groups": {
    "node": [
      "ms:bengreenier.vscode-node-readme",
      "opn:christian-kohler.npm-intellisense",
    ],
    "python": [
    ],
  },
  "vsix.extensions": [
    "opn:mjmlio.vscode-mjml",

    // #enable(editor="visual studio code")
    "GitHub.codespaces",

    // #enable(os="mac")
    "sebsojeda.vscode-apple-music",

    // #rewrite-next-line
    // { "group": "#{env.DEV_LANG}" },
    "node",
  ],
}

[manuth]

Awesome! Thank you so much for the great work!

I managed it to get both VSCode and VSCodium to sync extensions with each other properly. Using this new extension, I use these settings to work around extensions with mismatching names/mismatching dependencies:

My Settings

***settings.json:*** ```js { "syncSettings.ignoredExtensions": [ "aster.vscode-subtitles", "hbenl.test-adapter-converter", "ms-dotnettools.csharp", "ms-python.vscode-pylance", "ms-vscode-remote.remote-containers", "ms-vscode-remote.remote-ssh-edit", "ms-vscode-remote.remote-ssh", "ms-vscode-remote.remote-wsl", "ms-vscode-remote.vscode-remote-extensionpack", "ms-vscode.remote-explorer", "ms-vscode.remote-server", "ms-vscode.test-adapter-converter", "ms-vsliveshare.vsliveshare", "muhammad-sammy.csharp", "naco-siren.gradle-language", "pepri.subtitles-editor", "richardwillis.vscode-gradle", "richardwillis.vscode-gradle-extension-pack", "VisualStudioExptTeam.intellicode-api-usage-examples", "VisualStudioExptTeam.vscodeintellicode", "vscjava.vscode-gradle" ], "vsix.groups": { "code": [ "aster.vscode-subtitles", "ms-dotnettools.csharp", "ms-vscode-remote.vscode-remote-extensionpack", "ms-vscode.test-adapter-converter", "ms-vsliveshare.vsliveshare", "ms-python.vscode-pylance", "naco-siren.gradle-language", "pepri.subtitles-editor", "richardwillis.vscode-gradle-extension-pack" ], "codium": [ "hbenl.test-adapter-converter", "muhammad-sammy.csharp", "richardwillis.vscode-gradle" ] }, "vsix.extensions": [ // #if(editor="visual studio code") // "code" // #else "codium" // #endif ], } ```

All I do is ignoring all mismatching extensions and make vscode-vsix-manager install those mismatching extensions separately.

[daiyam]

It's another way to do it. Glad, it's working for you.

But sync-settings is aware of the extensions managed by vsix-manager so you could remove them from syncSettings.ignoredExtensions.

[manuth]

Thank you so much for the advice! Doesn't seem to be the case for me.

Extensions without a custom source don't seem to be considered managed (notice that they aren't added to installedExtensions here):

https://github.com/zokugun/vscode-vsix-manager/blob/909cdaa29fef2562b55b6ae92733de9a577a877a/src/commands/install-extensions.ts#L131-L140

[daiyam]

Thank you so much for the advice! Doesn't seem to be the case for me.

Extensions without a custom source don't seem to be considered managed (notice that they aren't added to installedExtensions here):

https://github.com/zokugun/vscode-vsix-manager/blob/909cdaa29fef2562b55b6ae92733de9a577a877a/src/commands/install-extensions.ts#L131-L140

Yep, you are right. It's a bug in that extension...

[manuth]

Oooh sorry - I wasn't sure whether you did this on purpose. I'll open up a PR then

Thank you so much for the great work!

[daiyam]

@manuth Thanks for the PR. The new update has been published.

Can you close this issue if everything works for you? Thx

[binaryben]

Sorry @daiyam, I took a break from coding for a few months. Let me check out the new extension and I'll close this issue once all in working! Thanks for your work, it's truly appreciated

[binaryben]

Hi @daiyam, I think I am doing something wrong but not sure. I'm just testing my "ui" block of extensions at the moment. Following the instructions under Getting started in my editor config directory is installing some of the extensions.

It's loading them incredibly slowly though, and some are missing. I can't find any logs to give more insight. Any insight is appreciated - can also open as a different ticket if you prefer and close this one?

[daiyam]

@binaryben What's your config?

[binaryben]

What's your config?

profile: main
repository:
  type: file
  path: ~/.files/config/editor

And the data in the main profile is here.

[binaryben]

Marking as closed. Found some potential bugs/issues, but will open in the relevant repo!

[daiyam]

@binaryben

• you can use "vsix.debug": true, to have some logs.
• "local:iocave.monkey-patch" looks for iocave.monkey-patch-0.1.23.vsix. So you can either rename the file or just ask for "local:monkey-patch".
• use "vsix.applyChanges": false, to avoid to run the install process at the start of the editor. You will need to manually call it if you make changes. I thinks I need to improve this.
• there is some errors when trying to download some vsix, I will look into it.

[daiyam]

@binaryben I found the origin of those errors. The latest version of VSIX Manager has the fix in place.

[binaryben]

you can use "vsix.debug": true, to have some logs.

Yup, found that one after I RTFM 🙈

just ask for "local:monkey-patch"

Thanks, that got that one. Might need to set the uninstall to go in reverse order to account for dependencies like this?

use "vsix.applyChanges": false,

Did not find that one, added now.

I found the origin of those errors.

Awesome, I seem to be running the latest extension. I've added some ideas for DX improvements to that repo.