[INFRA-266] Jenkins and jenkins plugins not available for download via HTTPS

[jenkins-infra-bot]

mirrors.jenkins-ci.org does not support HTTPS and there is no other way to download Jenkins and Jenkins plugins.

---

Originally reported by coderanger, imported from: Jenkins and jenkins plugins not available for download via HTTPS

• assignee: timja
• status: Resolved
• priority: Major
• resolution: Fixed
• resolved: 2020-08-22T10:59:21+02:00
• imported: 2022/01/10

[jenkins-infra-bot]

coderanger:

I've reached out to a friend at OSL to see about getting HTTPS enabled for their mirror at least. Would still need address chain of trust between jenkins-ci.org and the mirrors.

[jenkins-infra-bot]

coderanger:

The various repo signing keys (ex. http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key and http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key) should also be available via HTTPS.

[jenkins-infra-bot]

coderanger:

Also the default value for the update center JSON should be to use HTTPS, since that is already available there.

[jenkins-infra-bot]

kohsuke:

I'm moving this to the public project since this issue doesn't discuss vulnerability.

[jenkins-infra-bot]

kohsuke:

OSUOSL is not the only mirror, so an integrity check that relies on HTTPS would be tricky.

I think an easier thing to do is to provide cryptographic checksums in update center metadata, which is already digitally signed. This is how debian packages and RPM files are signed, too.

[jenkins-infra-bot]

danielbeck:

More of an INFRA issue.

I doubt SSL for downloads can be done, we cannot even default to delivering UC metadata via SSL according to R. Tyler Croy in https://github.com/jenkinsci/jenkins/pull/1356

[jenkins-infra-bot]

gunio_rich:

Why isn't this resolved? Literally a one-liner fix, as described here: https://issues.jenkins-ci.org/browse/JENKINS-28009

[jenkins-infra-bot]

orrc:

It would be a few lines of HTTP server config, but we don't actually have a certificate with a SAN for mirrors.jenkins-ci.org.

In any case, enabling this now would just be giving a false sense of security, as mentioned in the pull request linked above your comment, and in INFRA-110.

Due to the volume of traffic that must be served, Jenkins core and plugin downloads are redirected to the mirror network. The vast majority of those mirrors do not support HTTPS, so any links to https://mirrors.jenkins-ci.org/ would immediately redirect most people to an http:// link.

As Tyler mentioned, end-to-end HTTPS support would definitely be desirable to have, but it's not as simple as flicking a switch. If someone can encourage all mirrors to provide HTTPS, or provide additional mirrors that can do so (or pay for lots of CloudFront bandwidth), that would be great. If you or anyone can help make that happen, that would be appreciated.

[jenkins-infra-bot]

gunio_rich:

Hm. That is an issue.

That being said, there are at least two mirrors that already do support HTTPS:

https://mirror.xmission.com/jenkins/
https://jenkins.mirror.isppower.de/

And I'd suggest that somebody from INFRA or SECURITY contact the others about the possibility about upgrading to SSL.

Is there any reason why the Jenkins SSL certificate isn't a wildcard cert?

[jenkins-infra-bot]

coderanger:

I reached out to Fastly and they would be happy to help set up an HTTPS-based CDN for you: https://twitter.com/fastly/status/590582481130287104

We (PyPI) switched from a bunch of independent mirrors to a single, Fastly-backed CDN for managing things and it has been nothing short of amazing both for users and admins. If you would like any further introductions to the Fastly team, I would love to help out with that.

[jenkins-infra-bot]

rtyler:

I've contacted Fastly, we'll work something out. Will update this ticket once I've got more details.

[jenkins-infra-bot]

danielbeck:

KK brought up Bintray as a hosting service.

[jenkins-infra-bot]

danielbeck:

The Linux packages for core Jenkins are now served via HTTPS.

[jenkins-infra-bot]

rtyler:

The Linux packages for core Jenkins are now served via HTTPS.

This is only if the package repository was added via https, e.g. https://pkg.jenkins.io/debian. We do not upgrade HTTP requests to HTTPs because that breaks lots of insolent Linux distributions which don't support Yum or Apt repos over HTTPs in their default installs (see apt-transport-https which is still an optional package in recent Debian releases)

[jenkins-infra-bot]

dskrvk:

Is the above the reason why https://pkg.jenkins.io/redhat/jenkins.repo contains an HTTP URL? It seems like the desire to keep backwards compatibility creates an insecure default option for many folks who are on modern systems.

[jenkins-infra-bot]

bjmgeek:

I'm using Jenkins behind a firewall that doesn't allow outbound http connections, only https.  When I try to download a plugin, it redirects my https requiest (with a 302 Found) to http.  I tried to use the Plugin Manager, but got an error. Then I tried from the command line with wget, and got the same errror.

$ wget https://updates.jenkins.io/latest/mailer.hpi
--2018-07-31 11:08:29--  https://updates.jenkins.io/latest/mailer.hpi
Resolving updates.jenkins.io (updates.jenkins.io)... 52.202.51.185
Connecting to updates.jenkins.io (updates.jenkins.io)|52.202.51.185|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://updates.jenkins.io/download/plugins/mailer/1.21/mailer.hpi [following]
--2018-07-31 11:08:29--  https://updates.jenkins.io/download/plugins/mailer/1.21/mailer.hpi
Reusing existing connection to updates.jenkins.io:443.
HTTP request sent, awaiting response... 302 Found
Location: http://mirrors.jenkins-ci.org/plugins/mailer/1.21/mailer.hpi [following]
--2018-07-31 11:08:29--  http://mirrors.jenkins-ci.org/plugins/mailer/1.21/mailer.hpi
Resolving mirrors.jenkins-ci.org (mirrors.jenkins-ci.org)... 52.202.51.185
Connecting to mirrors.jenkins-ci.org (mirrors.jenkins-ci.org)|52.202.51.185|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

[jenkins-infra-bot]

olblak:

Brian Minton Thanks for reporting, I'll try to have a look as soon as possible.
It seems that mirrors.jenkins-ci.org doesn't have the correct certificate

[jenkins-infra-bot]

rtyler:

mirrors.jenkins-ci.org has never been SSL enabled and we've never had any references to https://mirrors.jenkins-ci.org.

 

 

We have had an SSL certificate for pkg.jenkins-ci.org, but never mirrors.*

[jenkins-infra-bot]

olblak:

Was there any reason why SSL is not enabled on mirrors.* ?

[jenkins-infra-bot]

rtyler:

Because mirrors.jenkins-ci.org/jenkins.io redirects to only HTTP services, and many clients will appropriately complain about a protocol downgrade.

 

[jenkins-infra-bot]

olblak:

A proxy can be configured to keep the scheme, so I don't understand how it can affect people that only want to use HTTP

[jenkins-infra-bot]

llibicpep:

Ok Jenkins team, like it or not but it's time to adapt. Like it or not, but many companies leaning towards blocking HTTP traffic and enforcing HTTPS-only policies globally. For me personally that means I can't use Jenkins at all. Leaving out wether enabling HTTPS on mirrors actually adds more security or not, with my current corporate security policy without HTTPS I just can't use it at all.

[jenkins-infra-bot]

danielbeck:

Dee Kryvenko There are many ways to can help move this issue forward. The Jenkins project is transitioning to be part of the CD Foundation, and corporate memberships help fund the project (so we may not need to rely on donated mirrors in the future). You can always donate directly too, of course, but I would expect only reliable, recurring income would make us consider committing to paying for the traffic indefinitely. And finally, most of our infrastructure is open source, largely operated by volunteers, and we're always looking for further help to keep it running or even improve it:  https://jenkins.io/projects/infrastructure/

There are other alternatives if you're not interested in contributing back to the project – You could operate your own update sites (tools make this very straightforward), or pay someone to provide fully HTTPS update sites to you as a service.

[jenkins-infra-bot]

llibicpep:

Daniel Beck how is this relevant, is this ticket a fundraiser? Is this how Jenkins Jira works now? Make it obvious then and... convenient, replace "vote" button with "donate"

Just to clarify my comment above to alleviate this controversy, previous comments from
Jenkins representatives made it clear that this ticket is not considered as rapidly growing priority (or so I read it), rather a small "inconvenience". So before even getting to fundraising if that's so necessary I want to raise awareness and make sure it's crystal clear to everyone as to where the market goes. With the recent major data leaks and increasing compliance requirements and regulations - so basic things as mandatory HTTPS not just becoming more and more common but also pops up as a hard requirement. The sooner we all realize what is that new world we all live in - the better. That said, HTTPS-enabled mirror list does not sounds like a rocket science. As an example a similar technology has already been in use for years by centos community - their yum network. Once we all accept that this request is absolutely necessary to address - the next step would be to define how exactly is to move forward with it. Is this an improvement to the existing mirror network, a voluntary or mandatory SSL certificate to join mirror network, or a whole new CDN platform? Is it even known by now how many mirror network members already have certificates and it's just a matter of config change for them? Then and only then it'll get to fundraising - when it's clear what needs to be done and how much does it cost. With such a great and huge community I'm sure it's not gonna be a problem, and I know there's certain vendors just can't wait to become partners - I already saw one offer above. This is not a new concept in the world, pretty much any open source project got to find means to host some content, pypi/gem/npmjs/yum/maven just to name a few. Nothing unsolvable here.

And once again I just want to emphasize on what it means not doing this request - for growing number of businesses it becomes a road block to even start/keep using Jenkins and look for alternatives. Entry toll to the Jenkins world becomes a little to high in terms of effort - to setup an internal mirror for a PoC project would be just a little too much. I would imagine vendors who earn money providing Jenkins Enterprise support (hi CloudBees) must be the most interested parties to keep this toll as low as possible.

[jenkins-infra-bot]

llibicpep:

I'll give a simple use case mostly for CloudBees if they read this topic - a small innovation team operates in hardly restricted AWS environment with blocked HTTPS traffic. Before being in a position even to start internal discussions as to using Jenkins for anything which potentially can open doors for Enterprise contract in the future - some internal poc/demo has to be done. Most businesses that is not software development related wouldn't even know what is Jenkins and why they should care. The whole plan falling apart since Jenkins just not able to install any plugins - you can imagine the odds CloudBees will ever even receive the invitation for a pitch.

[jenkins-infra-bot]

timja:

This should be resolved now, monitoring it for a few days

[jenkins-infra-bot]

timja:

mirrors.jenkins-ci.org still is http only because the software it runs (mirrorbrain only supports that)

We're switched plugins over to the https://get.jenkins.io service yesterday which runs on mirrorbits.

This service is https only.

Jenkins distribution packages / wars have been using get.jenkins.io for a few months now.

Let us know if you hit any issues

[jenkins-infra-bot]

[Is related to: INFRA-160]