Plan for plugins moving forward

[yeraydiazdiaz]

Hello, thanks for starting this community-driven fork 🙏🏻

I noticed that @andfoy has been creating issues in the repositories for different plugins but I wonder if a different approach would be benefitial.

I was thinking the newly formed python-lsp organisation would be a good place to gather these plugins so they're not only easier to find but can benefit from the shared experience of other plugin and pylsp contributors. These plugins could also be (re)published to PyPI with proper namespacing and maybe a classifier so they're easier to find.

@andfoy issues are certainly a step in the right direction, but frankly most of the plugins look quite abandoned and some of them already have forks that have been published independently (pyls-mypy for example was forked into mypy-ls), and I don't have high hopes they will be addressed soon, so, the only solution that comes to mind is forking, with all due respect to the authors of the plugins.

I hope this doesn't come across as confrontational, I genuinely would like to see an ecosystem of well-maintained plugins for pylsp.

[andfoy]

I think plugins can live in the organization, as long as the following conditions are met:

1. Someone is going to have to maintain them

Sadly, we (the Spyder team), won't have time to take over all the plugins

2. They would need to be renamed pylsp_*, e.g., pylsp_black, pylsp_mypy, etc

This is to distinguish them from the current existing plugins that work only with the old pyls. This change however may cause confusion to the users of the plugins that already are taking action to transition, such as pyls-memestra and pyls-spyder.

If we were to preserve the original pyls prefix, we would need to contact the original owners in order to get publishing permissions in PyPi, which may be difficult since the original plugins are unmaintained.

I think the best take here would be waiting for a few weeks in order to see if some plugin maintainers appear. I'll also publish a checkbox list here of all the pyls plugins, and which of those are already migrated.

Otherwise, if someone is able to volunteer for ownership for a specific plugin, they should comment here, in order to create a fork.

[andfoy]

After a thorough discussion with @ccordoba12, we decided to give the original plugins maintainers three weeks to perform the migration, after this period, which ends on May 18th, we will proceed to fork the plugins that were not migrated.

All the maintainers will be notified on the issues that were opened initially.

[yeraydiazdiaz]

I think there's not much point for plugins to support the original pyls, for example, pyls-memestra is not looking back. In my opinion python-lsp-server is the reference implementation now and plugins should just migrate or, if maintainers are unresponsive, forked.

It'd be great to have a PyPI package naming convention like pyls[p]-<NAME> for discoverability. Forked projects could use the extra p to distinguish or otherwise a PEP 541 issue can be opened for continued maintenance.

There is an edge case where the maintainer is responsive and willing to migrate but the package does not follow the convention, but I don't believe it would be a problem to rename the package to match. There could also be a list of plugins in this repo even if they don't all match the naming convention 100%, which is something that probably should exist regardless.

Whether or not they reside in the org is probably less important but probably benefitial, although I don't know what plans the Spyder team has for the org.

[krassowski]

I will also highlight that members of JupterLab team are happy members of the community and will help with maintenance of the plugins herein if given the chance. I also think that as long as original authors are responsive (see pyls-memestra), it should be fine for the plugin to live outside of the organization.

Maybe there should be a note in the reademe encouraging plugins (future or existing) to move to this organization in the future?

[andfoy]

@yeraydiazdiaz, we'll wait for some responses until May 18th, afterwards we can start delving into forks.

[andfoy]

Python Language Server plugins

Feel free to comment with your username if you are willing to give maintenance for a eventual fork.

• [x] pyls-black: The plugin was forked into https://github.com/python-lsp/python-lsp-black and @haplo is willing to maintain it
• [x] pyls-flake8: The plugin was migrated, thanks @haplo
• [x] pyls-isort: The plugin was migrated, thanks @haplo
• [x] pyls-mypy: Although there are plans to incorporate MyPy into the PyLSP core, a PR is ongoing on https://github.com/Richardk2n/mypy-ls/pull/2 and one of maintainers is reviewing it.
• [ ] pyls-smart
• [ ] pyls-cwrap
• [ ] pyls-black-macchiato
• [x] pyls-spyder: This one depend on us
• [x] pyls-memestra: Thanks @marimeireles!

[s7726]

An attempt was made... https://github.com/rupert/pyls-black/pull/37

[haplo]

Feel free to comment with your username if you are willing to give maintenance for a eventual fork.

I'm willing to fork pyls-black, pyls-flake8 and pyls-isort, as those are the ones I use. Looks like pyls-flake8 maintainer is responsive, so that will probably not be necessary. Let's wait until the deadline for the rest.

Edit: I also use pyls-mypy, so count me in for that one too if needed.

[s7726]

@haplo I came here to reply that someone had responded to the pyls-black PR I made... Then realized that was you ;) Nice.

[haplo]

@haplo I came here to reply that someone had responded to the pyls-black PR I made... Then realized that was you ;) Nice.

Yeah sadly no response from a maintainer yet. If they remain unresponsive it should be easy enough to fork, change repository and package names to pylsp- prefix and release compatible versions.

[andfoy]

@s7726 @haplo Which plugins are you willing to maintain after May the 18th?

[haplo]

@s7726 @haplo Which plugins are you willing to maintain after May the 18th?

I use these plugins:

• pyls-black
• pyls-flake8
• pyls-isort
• pyls-mypy

I already have PRs for 3 of them, and @s7726 has one for pyls-black.

If we have to fork any of these plugins I propose doing so under the python-lsp organization. Then we can add multiple maintainers, I can be one of them.

[haplo]

pyls-isort now has a compatible release in PyPI. :rocket:

[haplo]

Status update:

• pyls-flake8 accepted the PR, but has yet to release a new version to PyPI.
• pyls-mypy is unresponsive, and from other comments has been for months. It looks like there is no actively maintained fork, so my recommendation is to fork as python-lsp/lsp-mypy, make necessary changes and release as lsp-mypy package to PyPI. I can also fork at haplo/lsp-mypy and handle the release to PyPI, just let me know.
• No response from pyls-black. I can fork it and maintain it unless @s7726 wants to handle it.

[s7726]

@haplo I'm not sure I'm in a position to take on the responsibility of a maintained fork. I think I made the necessary modifications in the PR I sent. So that should be good.

I'm willing to try. But can't make a lot of promises.

[haplo]

I think I made the necessary modifications in the PR I sent. So that should be good.

Yes, we will use your modifications in any case. If we fork we should also take the opportunity to change the module and package name. I can do that before the initial release.

I'm willing to try. But can't make a lot of promises.

You decide if you want to take on the responsibility. I'm OK forking and maintaining it, so just let me know.

[andfoy]

So, basically as of May 18th, only pyls-black is missing support for the new pylsp server, so we are going to fork that one.

[s7726]

So, basically as of May 18th, only pyls-black is missing support for the new pylsp server, so we are going to fork that one.

Is pylsp going to fork it under the larger project?

I pushed in a merged set of changes to my fork. I'm working on going through the name change aspect. I'm working through how to actually build and distribute the package. I've never done it, so it's a learning effort.

I won't be upset if someone else has more time and knowledge to get it flipped quicker.

[andfoy]

For those interested, there is a new release available of python-lsp-black! The fork is available at https://github.com/python-lsp/python-lsp-black

[andfoy]

@s7726, don't worry, we've already made the changes. If you are interested in taking part in the maintenance of the fork, please chime in this issue: https://github.com/python-lsp/python-lsp-black/issues/4

[haplo]

pyls-flake8 just released a compatible version to PyPI! :tada:

[haplo]

@andfoy I created a PR that updates the README to point to the new forks.

[igo95862]

Can a plugin be upstreamed in to this repository? (for example, there is a flake8 plugin in default installation and as a third party plugin)

[andfoy]

@igo95862, it could be discussed as long as someone takes ownership of it

[igo95862]

I am kind of interested in official mypy plugin. It is a pretty big project made by python creators. Although it should be some way opt-in as a lot of projects are not typed. Mypy has a specification for configuration file so maybe it should only activate if the config file is present.

[doolio]

Has there been any thoughts about removing the built-in plugins entirely to simplify the codebase? I'm thinking many of the built-in plugins could be replaced by python-lsp-ruff which could be kept independent or even made to be built-in if preferable.

[haplo]

Has there been any thoughts about removing the built-in plugins entirely to simplify the codebase? I'm thinking many of the built-in plugins could be replaced by python-lsp-ruff which could be kept independent or even made to be built-in if preferable.

I'm +1 to removing all built-ins, and have people install the ones they need, everything properly documented of course, including a directory of known plugins. This should be OK with a new major version of python-lsp-server IMO.

[ccordoba12]

I'm +1 to removing all built-ins, and have people install the ones they need.

That's more or less the case right now. I mean, if you install python-lsp-server you'll only get code completion, go-to-definition and hovers with Jedi. All the other of functionality is opt-in by installing Yapf, rope, Autopep8, etc.

The problem I see with splitting the project into multiple plugins is that it'd make maintenance harder. And since I'm the only maintainer, I'm -1 on it because I'm also in charge of a lot of other stuff.

[haplo]

Jedi has no alternative AFAIK, right? In that case it makes sense to keep it built-in, as long as the rest are optional installs.

Would you be open to PRs to extract yapf, rope, autopep8, etc. into separate plugins?

[ccordoba12]

Jedi has no alternative AFAIK, right? In that case it makes sense to keep it built-in, as long as the rest are optional installs.

Rope, but it's not as feature complete.

Would you be open to PRs to extract yapf, rope, autopep8, etc. into separate plugins?

Nop, due to what I said. That would require for me to maintain not a single project but 5 or 10 of them (i.e. make releases, keep them in sync and in general spread my attention between several repos).

What we could do is to modify the defaults to not endorse any linting or formatting library at all. That way, if you install python-lsp-server and python-lsp-ruff, you'll get all that you need (for people that wants to use only Ruff).

[haplo]

Nop, due to what I said. That would require for me to maintain not a single project but 5 or 10 of them (i.e. make releases, keep them in sync and in general spread my attention between several repos).

I think I speak for everyone when I thank you for your maintenance effort, anything that makes your life easier is good.

If the optional parts are neither installed nor enabled then it should be OK IMO.

What we could do is to modify the defaults to not endorse any linting or formatting library at all. That way, if you install python-lsp-server and python-lsp-ruff, you'll get all that you need (for people that wants to use only Ruff).

I like that idea. Currently many plugins (thinking of python-lsp-black and python-lsp-ruff) disable other plugins in their default configuration, this could be a way to clean that up.

[ccordoba12]

I think I speak for everyone when I thank you for your maintenance effort, anything that makes your life easier is good.

Thanks for understanding.

I like that idea.

Great! Could you submit a pull request for that? We can include it in our next minor release (1.10.0).

[mcepl]

Rope

There is https://github.com/python-rope/pylsp-rope which is built for THIS Python LSP server, not the old one.