search

crits / crits

CRITs - Collaborative Research Into Threats
https://crits.github.io
Other
886 stars 259 forks source link

Punycode domain TLDs not accepted (and/or decoded?) #956

Open rj-chap opened 6 years ago

rj-chap commented 6 years ago

When attempting to add a punycode-based domain, we receive the error "No valid TLD found". We see in https://github.com/crits/crits/issues/518 that we can edit TLDs, but I'm also wondering where punycode decoding sits, since I can't find any "puny" or "xn--" strings in the code base.

Do we have the ability to add a punycode domain and have it parsed properly in the system?

Example: xn--aaaa-83dr2faaam5l[.]xn--p1acf

(The brackets above surrounding the dot between the root domain and TLD are only there to ensure the link does not become clickable.)

rj-chap commented 6 years ago

Are we able to add a TLD of "xn--*"?

ManoftheSea commented 6 years ago

etld.parse() should probably do a punycode decode (if it looks like punycode), before checking against known TLDs.

mgoffin commented 6 years ago

If punycode formatted domains and TLDs aren’t in the public suffix that you can upload and update through the UI, then it will not work.

There is no special code for parsing these currently.

brlogan commented 6 years ago

This is something I've been meaning to tackle for months, but has been low priority. At least part of it would be doing a punycode decode when the domain is submitted as an Indicator, Object, or Domain. For now the user must decode from punycode before submitting the domain to CRITs.