Wildcard mult-level subdomain issue

[mjamaleddine]

Environment information

AWS Amplify Console

Description

We need a wildcard subdomain with multi-levels. Our goal is to have a test environment where we can test the multi-tenant feature. So the domain should look like this *.test.domain.com But if setting this up in the AWS Amplfiy console it gets stuck at the SSL Configuration step for multiple hours. Is there maybe a manual way to accomplish this. The domain is hosted at Route 53.

[Jay2113]

@mjamaleddine 👋 , thanks for reaching out. Based on the issue description, it seems like your domain is hosted on a 3rd party DNS provider. When connecting such domains to an Amplify app, our recommendation is to configure the DNS records (that are displayed in the console) within your DNS provider soon after adding the custom domain in the Amplify console. The AWS Certificate Manager (ACM) immediately starts attempting to verify ownership. Over time, the checks become less frequent. If you add or update your CNAME records a few hours after you create your app, this can cause your app to get stuck in the SSL Configuration phase.

[mjamaleddine]

@Jay2113 actually our domain is hosted and registered on route 53. I realized this issue is happening with multilevel-subdomains with or without wildcards.

[Jay2113]

@mjamaleddine I wasn't able to reproduce this behavior when setting up a multi-level wildcard subdomain using a domain hosted on Route 53. Are you experiencing this issue consistently? Can you verify if the DNS records (CNAME records for ACM server validation and CloudFront distribution) are being created in the Route 53 hosted zone?

[mjamaleddine]

@Jay2113 I think the issue is the setup process.

So in order to use a multi-level wildcard subdomain I need to do the following steps:

1. Go to Route 53 and Create a new hosted zone for the subdomain (level 1). I have a hosted zone for domain.com so now I create one for dev.domain.com
2. Go to amplify -> custom domain -> add domain
3. Now I can select dev.domain.com from the dropdown. I exclude the root and enter * as a subdomain of dev.domain.com
4. Now it gets stuck at the SSL configuration step.

I think the CNAME record gets created on this new hosted zone instead of on the root hosted zone. And that is where the SSL configuration looks. So if I manually now create the CNAME record for validation and CloudFront on the root hosted zone, the SSL Configuration will proceed.

[Jay2113]

@mjamaleddine thanks for your continued patience and for sharing your setup. You've correctly stated that the CNAME records are added to the root hosted zone and this is an expected behavior. I'm curious about your decision to set up a new hosted zone for your subdomain instead of using the root hosted zone in Route 53. Could you please explain the reasoning behind it?

[mjamaleddine]

@Jay2113 tell me how else I am supposed to setup a multi-level domain in the amplify console? If I used the root and tried to enter a multi-level the console will complain that this is not a valid subdomain. Have you tried the flow?

[Jay2113]

Hi @mjamaleddine, apologies for the delayed response. You have correctly stated that when attempting to add a subdomain like test.dev under a root domain such as example.com, an error is returned.

I'm curious if you could try the following configuration to set up a subdomain dev.example.com as the root domain, and then add a test subdomain under it.

This approach can help resolve the issue you were facing with the subdomain setup. Please let me know if this alternative configuration works for your use case.

[MAN-Sendance]

I am struggling with the same issue.

[MAN-Sendance]

Hi @mjamaleddine, apologies for the delayed response. You have correctly stated that when attempting to add a subdomain like test.dev under a root domain such as example.com, an error is returned.

I'm curious if you could try the following configuration to set up a subdomain dev.example.com as the root domain, and then add a test subdomain under it.

This approach can help resolve the issue you were facing with the subdomain setup. Please let me know if this alternative configuration works for your use case.

This does not work for me.

[MAN-Sendance]

This is a major issue since multi-level subdomains are a common thing.

[MAN-Sendance]

The process is functioning as described here. However, two significant issues need to be addressed.

Problem 1: Domain Ownership Verification Notice

The information about domain ownership verification (see the image below) only appears for a few milliseconds, making it easy for users to miss this crucial step. This can confuse as users might not know what to do next. There is a workaround by navigating to Actions > View DNS Records, but this should not be necessary.

Expected Notification:

Current Notification (Misleading):

Problem 2: Incorrect Hostname Display for DNS Records The hostname provided for the DNS record is displayed with the root domain (second-level and top-level domain), such as _hash.fourth.third.second.tld (_a14544abef5.docs.backend.example.com). However, for the verification to work, the DNS entry should only include the part without the second and top-level domain, for example, _hash.fourth.third (_a14544abef5.docs.backend). This discrepancy is very misleading and can prevent the verification from being successful.

[Jay2113]

Hi @MAN-Sendance , thanks for sharing detailed information. Let me address both of the problems individually,

Problem 1: Domain Ownership Verification Notice The information about domain ownership verification (see the image below) only appears for a few milliseconds, making it easy for users to miss this crucial step. This can confuse as users might not know what to do next. There is a workaround by navigating to Actions > View DNS Records, but this should not be necessary.

I agree and I'll create a separate feature request to improve the visibility and clarity of the domain ownership verification notice.

Problem 2: Incorrect Hostname Display for DNS Records The hostname provided for the DNS record is displayed with the root domain (second-level and top-level domain), such as _hash.fourth.third.second.tld (_a14544abef5.docs.backend.example.com). However, for the verification to work, the DNS entry should only include the part without the second and top-level domain, for example, _hash.fourth.third (_a14544abef5.docs.backend). This discrepancy is very misleading and can prevent the verification from being successful.

Are you hosting the domain with a third party DNS provider? I did some testing with a Route 53 domain and i observed the ACM server validation record had the hostname as _bd205c7a6a9c1828cc784485f43c514a.test.dev.previews.jraval.cloud. and I was able to activate the domain successfully without removing the second-level and top-level domain from the hostname.

[MAN-Sendance]

Are you hosting the domain with a third party DNS provider? I did some testing with a Route 53 domain and i observed the ACM server validation record had the hostname as _bd205c7a6a9c1828cc784485f43c514a.test.dev.previews.jraval.cloud. and I was able to activate the domain successfully without removing the second-level and top-level domain from the hostname.

Yes, exactly, I am using a third-party DNS provider.

[Jay2113]

Thanks for the confirmation @MAN-Sendance. Could you perform a quick test on a non-production app? Follow these steps:

1. Connect a domain (using a third-party DNS provider) to an Amplify app.
2. Add the ACM server validation record CNAME record in the DNS provider as displayed in the Amplify console.
3. Navigate to the Mx toolbox tool.
4. Enter the hostname for the ACM server validation record CNAME record.
5. Select CNAME Lookup from the dropdown menu.
6. Click on the CNAME Lookup button.
7. Validate if the hostname's value and status are green. If not, retry in a couple of minutes.

Please share the test results and a screenshot to help us identify if the record is validated.

[Jay2113]

Hi @MAN-Sendance, just following up on my previous comment, did you get a chance to review and test it?

[MAN-Sendance]

Hi @MAN-Sendance, just following up on my previous comment, did you get a chance to review and test it?

sorry, I haven't had time yet.

[MAN-Sendance]

@Jay2113 I have executed your test plan. The result is still the same.

DNS entry WITH top-level domain and second-level domain:

DNS entry WITHOUT top-level domain and second-level domain:

[Jay2113]

@MAN-Sendance thanks for sharing the results they are really helpful. Which third party DNS provider/registrar are you using?

[MAN-Sendance]

@MAN-Sendance thanks for sharing the results they are really helpful. Which third party DNS provider/registrar are you using?

We are using the services of the German company Dogado.

[ferdingler]

However, for the verification to work, the DNS entry should only include the part without the second and top-level domain, for example, _hash.fourth.third (_a14544abef5.docs.backend).

Why do you say that the CNAME record doesn't need to include the top-level domain? It definitely needs to include the fully qualified domain name, not just the subdomain. However, its possible that your DNS registrar only lets you specify the subdomain when creating a CNAMe record, this is because your DNS provider is going to append the actual top-level domain to it, so they only need you to provide the subdomain part.

But bottom line, for the DNS verification to work, doing a DNS resolution on the fully qualified domain should resolve to the expected verification record. Example:

dig <hash>.docs.backend.example.com. 

Example expected response:

; <<>> DiG 9.10.6 <<>> <hash>.docs.backend.example.com. 
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20889
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;<hash>.docs.backend.example.com.  IN A

;; ANSWER SECTION:
<hash>.docs.backend.example.com.  500   IN CNAME <secret-verification-value>.acm-validations.aws.

;; AUTHORITY SECTION:
jddtvkljgg.acm-validations.aws. 900 IN  SOA ns-1553.awsdns-02.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 149 msec
;; SERVER: 172.16.153.81#53(172.16.153.81)
;; WHEN: Wed Aug 21 13:21:47 PDT 2024
;; MSG SIZE  rcvd: 244

[MAN-Sendance]

However, for the verification to work, the DNS entry should only include the part without the second and top-level domain, for example, _hash.fourth.third (_a14544abef5.docs.backend).

Why do you say that the CNAME record doesn't need to include the top-level domain? It definitely needs to include the fully qualified domain name, not just the subdomain. However, its possible that your DNS registrar only lets you specify the subdomain when creating a CNAMe record, this is because your DNS provider is going to append the actual top-level domain to it, so they only need you to provide the subdomain part.

But bottom line, for the DNS verification to work, doing a DNS resolution on the fully qualified domain should resolve to the expected verification record. Example:

dig <hash>.docs.backend.example.com. 

Example expected response:

; <<>> DiG 9.10.6 <<>> <hash>.docs.backend.example.com. 
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20889
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;<hash>.docs.backend.example.com.  IN A

;; ANSWER SECTION:
<hash>.docs.backend.example.com.  500 IN CNAME <secret-verification-value>.acm-validations.aws.

;; AUTHORITY SECTION:
jddtvkljgg.acm-validations.aws.   900 IN  SOA ns-1553.awsdns-02.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 149 msec
;; SERVER: 172.16.153.81#53(172.16.153.81)
;; WHEN: Wed Aug 21 13:21:47 PDT 2024
;; MSG SIZE  rcvd: 244

Please don't accuse me of making false statements. I was merely sharing my observations. It's possible that my DNS provider automatically appends the SLD and TLD, but this highlights that some providers may only accept the domain without SLD and TLD, potentially confusing users who are unaware of this. I have just been highlighting a potential user experience issue based on my experience.

[Jay2113]

@MAN-Sendance We apologize if the previous response came across as accusatory, that was certainly not our intention. We appreciate you taking the time to share your observations and highlight a potential user experience issue. You make a fair point that different DNS providers may have varying requirements for domain input formats, which could lead to confusion for users who are unaware of these nuances. Based on your feedback, we will improve our troubleshooting documentation to address this issue and improve the overall experience. Thanks!