Update for API changes and add subdomain support

[maservant]

• Add support for requesting certificates for specific subdomains (i.e. sub.example.com vs *.example.com)
• Use params argument of requests.get() to properly URL-encode parameters with special characters.
• This fixes issues such as AWS domain validation records with characters not allowed in URLs.
• As a general rule, building URLs by concatenating strings doesn't work. Some characters need to be escaped.
• Interpret responses as XML, not "LXML" which is meant for parsing HTML.
• Add mandatory "EmailType" parameter to setHosts request and preserve MX record priority.
• No longer deletes my MX records each time I renew my certificate (yay!)

[maservant]

Also added a .py extension to the script and changed the shebang line to force the use of Python 3 on systems where Python 2 is the default interpreter.

[trwnh]

thank you for this pr! my only concern is if multipart tld is handled, i'm not sure i see how. for example, .co.uk

[maservant]

Which TLDs are affected, if any?

I ran the namecheap.domains.getTldList API command and downloaded the .xml response. I ran the following script:

from bs4 import BeautifulSoup as bs

with open('xml.response.xml', 'r') as tld_list_xml:
    tld_list = bs(tld_list_xml.read(), 'xml').find_all('Tld')
    tlds_with_dot = []

    for tld in tld_list:
        if '.' in tld['Name']:
            tlds_with_dot.append(tld['Name'])

    print(tlds_with_dot)

Which gives the following response: ['ae.org', 'br.com', 'cn.com', 'co.bz', 'co.com', 'co.in', 'co.uk', 'com.au', 'com.cn', 'com.co', 'com.de', 'com.es', 'com.mx', 'com.pe', 'com.ph', 'com.se', 'com.sg', 'com.vc', 'de.com', 'eu.com', 'gb.net', 'gr.com', 'hu.net', 'in.net', 'jp.net', 'jpn.com', 'me.uk', 'mex.com', 'net.au', 'net.cn', 'net.pe', 'net.ph', 'net.vc', 'nom.es', 'nom.pe', 'org.au', 'org.cn', 'org.es', 'org.mx', 'org.pe', 'org.ph', 'org.uk', 'org.vc', 'ru.com', 'sa.com', 'se.net', 'uk.com', 'uk.net', 'us.com', 'us.org', 'za.com']

The issue goes deeper: from the point of view of the ACME protocol, your challenge should be posted as a TXT record to _acme-challenge.sub.domain.co.uk. If you're using the .uk TLD directly (which is allowed since 2014), then it should be posted to _acme-challenge.sub.domain.foobar.uk.

The problem is that in the first case the "Address" parameter of your Host element should be set to "_acme-challenge.sub.domain", and in the second case it should be "_acme-challenge.sub.domain.foobar". The only way to tell which one to use would be to call the API endpoint above (we can't hardcode the list of TLDs with dots since it is subject to change at any time). Note that ths isn't a problem I introduced; the previous version of the code had the same issue. The real problem is that Namecheap's API is badly designed. You shouldn't need to specify that "eggs-with-spam" is your SLD and "co.uk" is your TLD, since "eggs-with-spam.co.uk" is uniquely registered to you, and Namecheap are the ones who made up the above list in the first place.

To conclude

Your point is valid, the new version of the script will crash on any of the TLDs in the above list, but the current version will crash too. Namecheap really needs to design their API better, it looks like it's designed for resellers and not actual developers.

EDIT: We'd also need to handle the (rare) case where the TLD is no longer in use (for example sub.domain.[province code].ca in Canada), but existing domains are grandfathered in. I know of plenty of working domains ending in .qc.ca, but that TLD doesn't appear in the above list since new registrations aren't allowed.

[trwnh]

Note that ths isn't a problem I introduced; the previous version of the code had the same issue. The real problem is that Namecheap's API is badly designed.

so true ;_; but in that case, the README should retain the notice that SLD/TLD extraction is extremely naive and does not support multipart tlds -- i'd probably add a sub-bullet point and also you can credit yourself there! something like

- SLD/TLD extraction is extremely naive and does not support multipart tlds
  - update: it now handles subdomains thanks to @maservant

the last thing is that it should probably update the three occurrences of auth with auth.py

[trwnh]

:tada:

[maservant]

I updated the README exactly as you suggested. Note: I will open an issue on my end regarding the SLD/TLD bug. The problem is that the "Address" field is what gets prepended to the domain when you make a DNS query, so Namecheap already knows which part is the SLD and which part is the TLD. The API shouldn't ask you for the SLD and TLD, it should tell you which part is the SLD and which part is the TLD. I will check the docs again on Namecheap's website, maybe I missed a function that does exactly that.