[JENKINS-67590] publish-over-ssh plugin removed from update center

The plugin `publish-over-ssh` appears to be missing from the latest plugin repository (https://updates.jenkins.io/update-center.json) The same plugin was however available in the previous version.

We use that plugin for close to all jobs and thus we are in desperate need for this plugin to be added to the repository again.

Plugin removed from update center until security issues are resolved

Jenkins Security Advisory 2022-01-12 describes the following vulnerabilities:

SECURITY-2287 - Stored XSS vulnerability (medium severity)
SECURITY-2290 - CSRF vulnerability and missing permission checks (medium severity)
SECURITY-2307 - Path traversal vulnerability (medium severity)
SECURITY-2291 - Password stored in plain text (low severity)

Until someone adopts the plugin, fixes the issues, and releases a new version, it will remain unavailable.

Users that accept the security vulnerabilities can still download the plugin from the Jenkins artifact repository and upload it to their Jenkins installation.

Originally reported by blueicarus, imported from: publish-over-ssh plugin removed from update center

status: Resolved
priority: Minor
resolution: Fixed
resolved: 2022-02-25T19:07:52+11:00
imported: 2022/01/10

JIRAUSER132724:

You are likely not seeing that because the plugin was suspended recently due to security issues.

I really hope the plan here isn't to just leave this like that though. A plugin with over 70.000 installs just thrown out, marked as up for adoption, but closing the github issue pages to actually discuss manners of updating it. With the plugin page closed as well how exactly is adoption supposed to go down and why, given the installs and quite substantial nature of what the plugin does, is the course of action to just abandon it instead of spending the effort to update it.

Surely there should be a way to make it security compliant, but how is anyone supposed to help with that if the current setup on github doesn't allow proper discussion of the found issues. This is really poor handling and it's been going on like this for some time now. If Jenkins is starved for people to actually maintain the software then at the very least make it easier for volunteers to actually offer help.

As it stands with this plugin gone my entire pipeline is dead in the water as I push artifacts over SSH and run post build scripts to distribute and test, all of which runs through this plugin. I can imagine quite a number of people now find themselves in the same boat and you know what this does ultimately right? It causes people to not update their installs, to not update plugins or follow the recommendations you make in the name of security, because in the end it breaks their pipelines. This breeds a situation of heaps of security vulnerable jenkins installations in the wild just waiting to be used for malicious purpose. If you want people to keep their installations secure the best course, especially in the face of forced updates through dependencies going bad like log4j recently, is to make sure they keep both their installations and plugins up to date, but if that means breaking pipelines ever a few weeks then they won't do that.

I get it, it's all open source and doesn't make you a dime, but how do you expect the conversion rate into paying customers is going to go when you treat the free users with the boot to the face. Conversion just by virtue of no longer wanting to be treated that way doesn't exactly create customers with positive attitudes when problems arise or do you really want the "we are already paying for this why is is still not working" tickets to flood in all the time. I'm getting off track...

Stop abandoning plugins, fix them!

markewaite:

Thanks for your passion Vincent. I've tried to respond to your comments specifically and with accuracy.

You are likely not seeing that because the plugin was suspended recently due to security issues.

That is correct. Jenkins Security Advisory 2022-01-12 describes the following vulnerabilities:

SECURITY-2287 - Stored XSS vulnerability (medium severity)
SECURITY-2290 - CSRF vulnerability and missing permission checks (medium severity)
SECURITY-2307 - Path traversal vulnerability (medium severity)
SECURITY-2291 - Password stored in plain text (low severity)

I assume the security team felt that with 4 unresolved security issues it is safer for users to have plugin distribution suspended than to continue distributing a vulnerable plugin. Similar patterns have been followed for other vulnerable plugins.

According to the repository permissions updater, the publish over ssh plugin has no maintainer. With no maintainer and four open security vulnerabilities, it seems reasonable that the security team would choose to suspend distribution.

I really hope the plan here isn't to just leave this like that though. A plugin with over 70.000 installs just thrown out, marked as up for adoption, but closing the github issue pages to actually discuss manners of updating it. With the plugin page closed as well how exactly is adoption supposed to go down and why, given the installs and quite substantial nature of what the plugin does, is the course of action to just abandon it instead of spending the effort to update it.

Other hope the same thing. They hope that someone will adopt the plugin, fix the issues, and allow it to again be distributed. Until the issues are fixed, distribution is suspended.

The GitHub issues page was not closed. The publish over ssh plugin has never used GitHub issues to track its issues.

This Jira instance is the publish over ssh issue tracker. You can see its issues through this query. You can also see its open issues through this query.

Someone that wants to adopt the publish over ssh plugin should follow the "Adopt a plugin" instructions and submit a pull request to the repository permissions updater to add themselves as a developer.

Surely there should be a way to make it security compliant, but how is anyone supposed to help with that if the current setup on github doesn't allow proper discussion of the found issues. This is really poor handling and it's been going on like this for some time now. If Jenkins is starved for people to actually maintain the software then at the very least make it easier for volunteers to actually offer help.

The discussion was never configured to be done on GitHub. Issue tracking happens in this Jira instance. This is the same process used to handle issues for other plugins and for other suspended plugins.

If someone adopts the publish over ssh plugin and wants to track issues with GitHub instead of Jira, they can do that with a pull request to the repository permissions updater.

As it stands with this plugin gone my entire pipeline is dead in the water as I push artifacts over SSH and run post build scripts to distribute and test, all of which runs through this plugin. I can imagine quite a number of people now find themselves in the same boat and you know what this does ultimately right? It causes people to not update their installs, to not update plugins or follow the recommendations you make in the name of security, because in the end it breaks their pipelines. This breeds a situation of heaps of security vulnerable jenkins installations in the wild just waiting to be used for malicious purpose. If you want people to keep their installations secure the best course, especially in the face of forced updates through dependencies going bad like log4j recently, is to make sure they keep both their installations and plugins up to date, but if that means breaking pipelines ever a few weeks then they won't do that.

Users that want to continue using the vulnerable plugin can place the plugin in their Jenkins installation. It is not available from the update center but can be downloaded from the Jenkins artifact repository.

I get it, it's all open source and doesn't make you a dime, but how do you expect the conversion rate into paying customers is going to go when you treat the free users with the boot to the face. Conversion just by virtue of no longer wanting to be treated that way doesn't exactly create customers with positive attitudes when problems arise or do you really want the "we are already paying for this why is is still not working" tickets to flood in all the time. I'm getting off track...

Jenkins does not have a "conversion rate". Jenkins is an open source project. Jenkins does not have paying customers.

Many sponsor organizations fund the infrastructure that delivers Jenkins. Many sponsor organizations fund developers that contribute to Jenkins. Many sponsor companies provide software licenses that the Jenkins project uses to support its users. The Jenkins project is very grateful to those sponsoring organizations. None of those companies have any obligation to maintain specific plugins.

I have no empathy for your claim that anyone treats Jenkins users "with a boot to the face". It simply is not true.

Stop abandoning plugins, fix them!

If you're offering to adopt the plugin, please submit a pull request to the repository permissions updater adding yourself as a developer.

If your company needs it, maybe this is an opportunity to ask your company to sponsor maintenance of the plugin. The last maintainer of the plugin was employed by a bank. Other maintainers have included people employed by companies ranging from semiconductor companies to consulting companies to software companies.

JIRAUSER132724:

I admit I get a bit heated up so I'm sorry if it all sounds a bit harshly worded, but my perspective is that every time I hop into my jenkins install to build some binaries there is another red or yellow popup about yet another thing that has gone away or is broken and I'm fairly convinced something in jenkins leaked my smtp credentials last year which was a fun two weeks to track down, so I am mildly cranky at the fact something that I keep up to date and try my best to maintain keeps throwing curveballs made of lead into my face.

It's rather fragmenting to not utilize the issue tracking github has built in. Jira only offers issue tracking, pull requests still have to happen on github and will likely incur comments there so that creates a huge mess. I would have loved to just open an issue on github, inquire as to what exactly is broken on the plugin as the security information doesn't list the actual vectors or parts the vulnerabilities were found in and from there work out if my java is sufficiently rust-free to try and resolve them or at least get some opinions on how to best to that.

I see open source as a community effort and not just a "request to take over doing all the work" on a plugin used by enough people that even just 1% of them ought to have enough collective graymatter to figure out a solution to the problems at hand if they are known and documented.

What irks me the most is there is a security team capable of finding these vulnerabilities in the first place, which means they must have enough experience to give the information necessary to find the parts of the code responsible, but I can't manage to find that. What's the deal here? I have to request permission first and then might get a detailed report from them? I get the idea that opening this up means a roadmap to destruction, but anyone capable of exploiting that or deriving merit from it probably doesn't even need that to know what to do given the name of the security vulnerability alone. It's two edged sword as well, knowing the way in means I can at least attempt other means of throwing stones in the way.

The problem is you exercise editorial control based on analysis that only publish the findings and neither the process nor more detailed information, so what's to say after I spend the time and money to update a plugin you don't reject it because something else doesn't meet standard. The entire setup then just creates a feeling to me that while the project has high standards for the software itself there aren't many willing to spend the effort to satisfy them, hence plugins get abandoned as the workload to fix them without information and just curveballs their direction certainly isn't fun. I mean how would you feel if the work you did was for nothing because the plugin just gets removed. It says to make an effort to contact the original author of the plugin, which should be something the project should do if it doesn't already. Getting a report as the author on what I have to do to avoid delisting of the plugin based on the findings I would assume is done? It just looks strange to me; The amount of abandoned plugins with high install numbers just looks strange from the outside. Not to mention then the promotion of other plugins developed by the jenkins team as alternative when they really might not be depending on pipeline configuration.

It gets even worse when there is security notifications and whatnot popping up more and more lately and I find jenkins leaking data left right and center yet cannot identify which plugin or part is causing it because any hints from the security advisories just show me vectors but not how to test or work around them if applicable at all. I'm no java dev and infosec is not my primary, but it seems that is the baseline for operating jenkins nowadays as I effectively have to pentest the thing now to see what's what. Certainly the documentation on how to properly secure jenkins only mentions what to do in setup, but not how to identify or test it for leaks. So I am left just having to poke around in the dark or setup jenkins without outside connections to anything it receives or sends data to in order to protect things. At that point I might just stop using it and integrate the pipelines through bash or python scripts. It just kills the flexibility of jenkins, which is the one thing it still has going for it over other options. Shouldn't the focus be on retaining that.

Reading some of the Jira I get a really strange sensation. There are so many good suggestions and even straight up full guidebooks on how to fix something up or change it for the better, but a lot of it just sits there. The other big issue that I participated, contributed and discussed on here is still unresolved open and even after the consensus that a lot of the suggestions would work at least better than the current setup nothing was done and that was entirely in your ballpark mind you.

I know it sounds demanding and condescending given the open source and volunteer nature of the project, but such is the nature of it all. The projects I release in the wild get the same responses of users complaints and issues, yet I still derive satisfaction from resolving them and seeing the continued use. I'll happily work on reported issues with detailed information that make resolving things a lot easier when I don't have to dig around in the dark. I don't even see any contribution outside of that nor any gracious donations to work on features, which should never be the sole reason to work on something anyways. Perhaps the approach of "pay for it and I'll do it" would result in funds I could now throw at this issue, but I still, naively perhaps, believe that sheer merit of something is enough to warrant work on something.

What's the stance then on the abandonment of plugins and the eventual ecosystem this creates around jenkins. Isn't the plugin ecosystem half of what makes jenkins useful in the first place given it otherwise just builds maven and nothing else really. Shouldn't the objective be ease of adoption or at least making it easy to contribute, if not code, brain matter to resolving issues that come up. Remember what makes development easy in the first place, information. Reproducing issues with detailed instructions, pointing to exact error messages with line numbers, researching potential fixes from other sources, all that is something that can be contributed to plugin development outside of code and it might make one consider adoption with the work ahead clearly laid out instead of being a total blackbox you have to pick apart yourself.

Long story short then, what's the workload to fix the plugin up as it is now? Cost estimate, reproduction steps from the security team, standard you want it to follow etc. I'll happily throw something into the pot, but if I don't even know if it'll account for 10% of what's needed for a fix then I'd much rather get a nice dinner from that and find other means of distributing build artifacts.

markewaite:

Long story short then, what's the workload to fix the plugin up as it is now? Cost estimate, reproduction steps from the security team, standard you want it to follow etc. I'll happily throw something into the pot, but if I don't even know if it'll account for 10% of what's needed for a fix then I'd much rather get a nice dinner from that and find other means of distributing build artifacts.

Since my detailed answer didn't reduce your frustration or satisfy your complaints about which issue tracker is used or how the issue tracker is used or how the security vulnerabilities are disclosed, I'll keep this answer short.

If you'd like to fix the issues, then adopt the plugin and ask for access to the security issues. The security issue reports include details.

If you're not willing to adopt the plugin, then your choices are to use the vulnerable plugin or find other means of distributing build artifacts.

JIRAUSER132724:

I may sound angry and mad, but I assure you that my intentions are driven by a desire to see jenkins retain its relevance and get more support from its users. I feel like it's all entering a self-destructive spiral and that has me just worried the project will end up in a place it will just be known for being a security risk, devoid of options as all plugins are abandoned and then drifting into obscurity entirely. I'd hate to see that happen, that's what this is about for me.

What stops me from adopting the plugin and just releasing the security information that was gathered into the wild and watch as every one of the 70000 installs goes up in flames. What stops anyone from doing that or in fact just going around and finding the vulnerable instances and making a nice living with some "ethical hacking" for a nominal fee. Good faith?

That's also just a sad excuse for crowdsourcing, the driving factor behind open source. Effectively I have to beg to be allowed to work on something I can just get removed from at any point and have no more rights to the work I did. That instead of letting a handful of people contribute one has to step forward and take all heat. No wonder plugins don't get adopted.

Why not go a step further and straight release a new jenkins version that completely removes the plugin from being able to be loaded, oh right now we are back at the fact a ton of jenkins installs aren't updated and thus are probably now more vulnerable then ever. This only breeds further trouble down the line and jenkins is already infamous for being an attack vector.

I think I will adopt the plugin, just not officially, if 70000 people installed it and even just 10% end up needing it in their pipeline that's a nice potential revenue stream. If the core jenkins team doesn't care for plugins in the capacity to fix them then why include the plugin repo at all. You want to retain tight control over everything and expect people to just play ball for nothing in return.

To clarify, I am not frustrated at all, I seriously worry for the future of jenkins as a project, because with GitLab going leaps and bounds(not exactly a good thing either) there are many moving away already for better integration and what will end up being left are the ones that don't want to deal with new things, which are likely those also not updating their jenkins installs. End result, jenkins becoming an even bigger attack vector and its reputation of being a security risk growing ever more. I don't want to see that happen, because I think jenkins still has potential for being a lot better for developers to use not requiring complex setups like GitLab has at the moment. That's the strength it has, quick setup and results, but to leverage that it can't continue to fall short everywhere else. Addressing the security issues is good, but to just take a hacksaw to things, especially abandoned plugins I mean what stops the jenkins from then adopting it, isn't going increase install numbers, engagement of users contributing back to the project or hold a stand against what's fast becoming so bloated it takes 20 minutes just to download the .deb.

JIRAUSER139196:

Is there a roadmap to resolve the related security issues? Is the plugin really dead unless community picks is it up for adoption?

It'd be a shame since it is a popular and handy one.

JIRAUSER138555:

Based on the comments I assume that is indeed not going to be fixed until someone fixes this.

I can agree with the facts that you would not want this installed given the security issues (but that is my decision and not of someone else). This change brakes the current process (which it should not do if your ask me). There's even no mention of this in any of the release notes (at least I could not find any), I could not find any other plugins like this that have been removed with regards to security issues (but I did not look into this in detail) and I personally don't agree with the "lets brake this now so someone is forced to adopt and fix the plugin" way of working.

But that is just my point of view.

I did however find a bug in the matrix

We use a personal Jenkins build to install some tools (including plugins). In our case its still the `.txt` based installation using `install-plugins.sh`.

What I did there is add the following line:

`publish-over-ssh:latest:https://archives.jenkins-ci.org/plugins/publish-over-ssh/latest/publish-over-ssh.hpi`

That will download the hpi file from the archives.jenkins-ci.org page. And that works without issues. Maybe that helps.

kon:

I could not find any other plugins like this that have been removed with regards to security issues

There's a list at Suspended Plugins.

JIRAUSER140067:

I think the plugin is fixed for vulnerabilities at 1.23 version.

https://github.com/jenkinsci/publish-over-ssh-plugin/releases/tag/publish-over-ssh-1.23

Is it possible to remove suspension?

JIRAUSER139196:

Version 1.24 seems to be accepted. Many thanks to everyone involved.

kon:

Marking as resolved because the plugin is in the update center again.

jenkinsci / publish-over-ssh-plugin

[JENKINS-67590] publish-over-ssh plugin removed from update center #67

Plugin removed from update center until security issues are resolved