search

Materials-Consortia / materials-consortia.github.io

OPTIMADE website
https://optimade.org
1 stars 8 forks source link

Incomplete SSL support #8

Closed blokhin closed 4 years ago

blokhin commented 4 years ago
merkys commented 4 years ago

Possibly unrelated to SSL. Maybe because http://optimade.org gets redirected to https://www.optimade.org, and redirection for https://optimade.org is not set up?

rartino commented 4 years ago

Things are just set up that way:

> host optimade.org optimade.org mail is handled by 0 optimade-org.mail.protection.outlook.com. optimade.org mail is handled by 10 smtp.sgsi.ucl.ac.be.

> host www.optimade.org www.optimade.org is an alias for materials-consortia.github.io. materials-consortia.github.io has address 185.199.111.153 materials-consortia.github.io has address 185.199.108.153 materials-consortia.github.io has address 185.199.109.153 materials-consortia.github.io has address 185.199.110.153

One cannot have a CNAME alias for an apex domain, which is why the github pages instructions call for using an ALIAS/ANAME-type record (which aren't actual DNS records and must be supported in the DNS software). Perhaps @gmrigna can ask whether this is supported by the UCLouvain DNS.

blokhin commented 4 years ago

😢

rartino commented 4 years ago

Actually, I left something out; my response above was about getting optimade.org to serve our pages alongside www.optimade.org. But perhaps just having the former URL redirect to www.optimade.org is good enough? For that we just need the DNS for optimade.org to have an A pointer to some machine that responds with the redirect. Perhaps @gmrigna can pass that question to your IT department?

blokhin commented 4 years ago

@gmrigna we need your help with the DNS configuration here.

I‘d still recommend to add the DNS A-records, so we don’t miss the bare-domain address.

And www is still supported, at least with the warning for the SSL protocol.

gmrigna commented 4 years ago

@blokhin I have asked our sysadmins... It basically has to go at a central level in the University where I do not have full control.

gmrigna commented 4 years ago

It turns out that our DNS servers do not support "ANAME", so they had to go for the solution in which the different server IP addresses are specified explicitly. If these change on github, we will have to make the corresponding changes...

rartino commented 4 years ago

As far as I can see, the DNS change seems ok. So, maybe we should try to re-apply #19 ?

rartino commented 4 years ago

Should

or

be our main address?

gmrigna commented 4 years ago

Given the uncertainties related to "optimade.org" (e.g. "https" indicates not secure) , I would go for "www.optimade.org"

blokhin commented 4 years ago

With the new DNS A-records, no-www https://optimade.org (and http://) will work fine. See e.g. how https://optimade.science is configured. The certificate issuing will be handled by Github.

Then, the no-ssl-www http://www.optimade.org will work, given the current DNS CNAME-record points to a no-www version. The certificate warning will only be shown for ssl-www https://www.optimade.org.

rartino commented 4 years ago

Right, I think actually all of these certificate warnings can be resolved now that the DNS is set up, so it is more a "stylistic" choice.

I know there is a fraction of people who argue loudly about generally dropping 'www' on websites. On the other hand, at OPTIMADE we are heavily using subdomains for various things, so it would make sense from a structural perspective to let 'www.optimade.org' be our user-facing website, and let 'https://optimade.org' forward there.

@blokhin Do I understand correctly that you prefer https://optimade.org to be the main website? Do you feel strongly about this? Why?

blokhin commented 4 years ago

@rartino my only concern is that I have seen with my own eyes someone trying to type "optimade.org" in the browser address field and ending up with the error. Irrespective of the www or no-www canonical form, any DNS or SSL errors at the main website make very bad first impression and, as a result, inability to sell the optimade idea in general. I think this issue really deserves to be prioritized.

rartino commented 4 years ago

I agree completely with that all four alternatives of www.optimade.org and optimade.org with 'http' and 'https' should take you to our website, without yielding errors.

But if no one really wants to argue that https://optimade.org must be our main URL, I say we go with https://www.optimade.org.

This seems to work right now, with a forward from optimade.org -> www.optimade.org, except https://optimade.org seems to give a certificate error. Let's look into that.

rartino commented 4 years ago

Sigh: there is no easy fix: https://github.com/isaacs/github/issues/1675; the latest entries in that ticket from others with the same issue are just a day old...

So we need to setup a redirect for optimade.org -> www.optimade.org.

How do we want to go about doing this? Two alternatives:

The former is clearly more elegant, but invokes another service that may one day stop serving us for free. Maybe safer to stay with github?

Or, as I think about it, I can set up a repo that does both. If netlify goes away, we can just let github host it with the meta-tag-redirect.

rartino commented 4 years ago

Ok, I've set this up in a new materials-consortia repo: optimade-website-apex-redirect which has been connected to netlify. The setup is presently served under: https://jolly-euler-861a21.netlify.app which you'll notice just forwards everything to https://www.optimade.org/

@gmrigna I'm sorry about this, but the limitations of trying to serve both https://optimade.org and https://www.optimade.org from github pages apparently weren't clear to us. So, now we instead need the bare domain to point to netlify instead, since that way we can simply redirect all attempts at accessing https://optimade.org/<anything> with https://www.optimade.org/<anything>.

This means that you once again need to go to your sysadmins and ask them to change the 'A' pointers for the APEX domain that they just set up to instead be:

optimade.org.   A     104.198.14.52

which points at the IP of netlify's load balancer. Sorry.

The same thing applies as before; if netlify changes their IP, or goes out of business, etc., we'll have to change that DNS record again; but I don't anticipate that to happen soon.

gmrigna commented 4 years ago

The change has been requested... I will keep you posted.

gmrigna commented 4 years ago

It is done...

dig A optimade.org +short 104.198.14.52

rartino commented 4 years ago

And it appears to work :-) :

Now all work for me.

gmrigna commented 4 years ago

On my side, it still indicates that "https://optimage.org" is not secure... Might it be a cache problem?

rartino commented 4 years ago

Is optimage a new magical branch of Materials-consortia? :-D

(I think you've just misspelled the URL...)

gmrigna commented 4 years ago

No, I just mistyped here...

gmrigna commented 4 years ago

Screen Shot 2020-06-30 at 09 30 56

gmrigna commented 4 years ago

Once I accept the certificate, it is OK even after quitting and restarting the browser... I think that this is perfectly acceptable. @blokhin What do you say?

rartino commented 4 years ago

Ah, you are right; the netlify ssl certificate isn't yet in place, since I couldn't configure that until the DNS was in place. I don't know why it seemed to work for me (perhaps I had already a cached approval). I've made the config change at netlify. It should be fixed in a couple of hours.

blokhin commented 4 years ago

@rartino I have the following error with the certificate now. If it persists, we could continue using Github, I assume.

2020-06-30_20-52-18

rartino commented 4 years ago

@blokhin Indeed, I'm still waiting for the certificate to properly be provisioned by netlify. It may take a couple of days to get rid of this warning.

rartino commented 4 years ago

Oh. I have found the problem. Among the DNS records provided for the APEX domain, there is for some reason two CAA records:

optimade.org.       21564   IN  CAA 0 iodef "mailto:abuse@uclouvain.be"
optimade.org.       21564   IN  CAA 0 issue "sectigo.com"

The meaning of that CAA record is to restrict who can issue SSL certificates for the domain. The second line there says only "sectgo.com" may issues certificates for 'optimade.org', which means that the letsencrypt one that netlify tries to create for us fails.

@gmrigna Can you please ask your sysadmins to remove the record:

optimade.org.       21564   IN  CAA 0 issue "sectigo.com"

As long as that is there, we can never get a free SSL certificate for the bare domain.

(And while you are bothering them, perhaps also ask them to insert the schemas record I asked about elsewhere:)

schemas.optimade.org. CNAME materials-consortia.github.io

Thanks!

rartino commented 4 years ago

@gmrigna So, the CAA record for optimade.org seems gone, great! But, the github A pointers are back again?...

gmrigna commented 4 years ago

What do you mean? If I do: dig A optimade.org +short I get 104.198.14.52

rartino commented 4 years ago

Oh, I get:

dig A optimade.org +short
185.199.109.153
185.199.108.153
185.199.110.153
185.199.111.153

But then I suppose it is a caching issue. Seems odd since the record was changed several days ago, though.

gmrigna commented 4 years ago

Our sysadmins confirmed that only "sectgo.com" may issue certificates for 'optimade.org'. So, I have requested that they issue one...

rartino commented 4 years ago

That works.

You can see what I need here: https://www.netlify.com/blog/2016/04/11/installing-your-own-ssl-certificates-a-step-by-step-guide/

I.e., a 'PEM formatted certificate' a 'Private key', and a 'CA certificate chain' formatted as in the example text in the middle of that page.

The question is how to send these securely to me; since the email trouble last week at my university was due to a problematic security breach so it may be best not to send it in unencrypted in email... I'll try to think of something.

gmrigna commented 4 years ago

@rartino According to our sysadmins, there is a final change to be made for things to work correctly.

One should change: materials-consortia.github.io. 3600 IN CAA 0 issuewild "digicert.com" materials-consortia.github.io. 3600 IN CAA 0 issue "digicert.com" materials-consortia.github.io. 3600 IN CAA 0 issue "letsencrypt.org"

into

materials-consortia.github.io. 3600 IN CAA 0 issuewild "sectigo.com" materials-consortia.github.io. 3600 IN CAA 0 issue “sectigo.com"

Is that possible?

rartino commented 4 years ago

No, I don't think GitHub allows users to change the CAA records under github.io.

Did they say what they are trying to accomplish? As far as I understand, these only apply to our sites hosted on GitHub; https://www.optimade.org, https://providers.optimade.org, and https://schemas.optimade.org, and those are all already working fine with SSL and would, in fact, break if we implemented these changes.

We are asking for a sectigo SSL certificate for serving a web page at the APEX domain https://optimade.org (that will only be used to forward to https://www.optimade.org); it really only needs to apply to that one domain "optimade.org", not any subdomains. As soon as we can upload the data for that certificate to netlify, everything we need should be handled.

Did they already create such a certificate? Perhaps we can transfer them using the Teams team that your university set up for the conference.

gmrigna commented 4 years ago

The sectigo SSL certificate has been obtained for serving a web page at the APEX domain https://optimade.org However, since there are two different certificates (one by sectigo for optimade.org and another one by letsencrypt for www.optimade.org), optimade.org is still considered unsecure... I will send you what I have about the certificate by email.

gmrigna commented 4 years ago

Our discussion has ended up reaching Belnet (https://en.wikipedia.org/wiki/BELNET), the Belgian internet provider for educational institutions... The only way out (to secure both www.optimade.org and optimade.org) that is now suggested to us is to move the actual html page on a UCLouvain machine and to set up a hook on github to trigger the update (using jekyll) of the html page on the machine in Louvain. Would that be acceptable for us?

rartino commented 4 years ago

that is now suggested to us is to move the actual html page on a UCLouvain machine and to set up a hook on github to trigger the update (using jekyll) of the html page on the machine in Louvain. Would that be acceptable for us?

For our main website at https://www.optimade.org/ ? No, I'd rather give up the 'https://optimade.org' forward in that case; i.e., keep the setup we have working right now.

However, this feels like a game of telephone going wrong. Before we let this run away into an overcomplicated setup, can we just try the setup I've been suggesting? I assume that UCLovain has a deal with Sectigo to sign your SSL certificates? Can you please take the CSR you showed me and get it processed by Sectigo into a certificate, and then give me the certificate and private key. Then we can try to install it and see if this setup works (and I'm quite sure it will.)

(Or is the problem here one of policy where you are not allowed to communicate that certificate + private key to someone outside your organization? As in, it isn't even ok if you set this up at netlify, since netlify would get the key + certificate?)

rartino commented 4 years ago

@gmrigna I've been discussing directly with sectigo about their policy. They have told me that our setup can be approved, and to the best of my understanding the simplest adjustment to get there is if we add CAA records for www.optimade.org to allow both letsencrypt and sectigo. I.e. we need: 

www.optimade.org.       CAA 0 issue "sectigo.com"
www.optimade.org.       CAA 0 issue "letsencrypt.org"

Of course, while still keeping the existing CNAME:

www.optimade.org.          CNAME    materials-consortia.github.io.

Can we try this? Note: it is particularly important that the letsencrypt line above is not excluded, otherwise SSL for our present website will break.

gmrigna commented 4 years ago

@rartino: I got in touch with our local sysadmins and leaving letsencrypt as a CAA is precisely what they do not want to do. Their current suggestion is that we have a virtual machine in UCLouvain (with all the privileges that we want).

rartino commented 4 years ago

@gmrigna Ok, that is a bit of a weird thing to object to, given that we already effectively have that CAA record via the CNAME. But since I have a dialog with sectigo, I'll inquire if there is any alternative solution that would work.

blokhin commented 4 years ago

@rartino please, let me know, if there's anything I can help. I'm back from vacations now.

rartino commented 4 years ago

Looks like we are online!

Can anyone else confirm that all these work without any certificate warnings?:

https://optimade.org

http://optimade.org

https://optimade.org/contributors

https://www.optimade.org

http://www.optimade.org

https://www.optimade.org/contributors

sauliusg commented 4 years ago

Great news! Congrats!

from Vilnius (IP 78.60.2.35), all links work without problems!

On 2020-07-15 16:16, Rickard Armiento wrote:

Looks like we are online!

Can anyone else confirm that all these work without any certificate warnings?:

https://optimade.org

http://optimade.org

https://optimade.org/contributors

https://www.optimade.org

http://www.optimade.org

https://www.optimade.org/contributors

-- Dr. Saulius Gražulis Vilnius University Institute of Biotechnology, Saulėtekio al. 7 LT-10257 Vilnius, Lietuva (Lithuania) fax: (+370-5)-2234367 / phone (office): (+370-5)-2234353 mobile: (+370-684)-49802, (+370-614)-36366

gmrigna commented 4 years ago

Great!!! Thanks @rartino for solving the issue with sectigo