nextdns / metadata

This repository contains the data behind our Security, Privacy and Parental Control features.
https://nextdns.io
638 stars 138 forks source link

Threat intelligence blocking tiktok #428

Closed rodeodomino closed 1 year ago

rodeodomino commented 4 years ago

The threat intelligence feeds list is blocking sf16-muse-va.ibytedtos.com which is preventing tiktok links from loading properly in mobile safari. Thanks!

Voltairine-de-Cleyre commented 4 years ago

sf16-muse-va.ibytedtos.com

I'm not sure of the TI filter that's doing it. This is the raw WHOIS, though: (I posted it due to Gandi being the host, as they are notorious for hosting malware, the Registrant is in Kentucky & the Organization's name is "Lemon".)

Domain Name: ibytedtos.com Registry Domain ID: 2201411935_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.gandi.net Registrar URL: http://www.gandi.net Updated Date: 2020-09-14T12:42:35Z Creation Date: 2017-12-18T02:39:55Z Registrar Registration Expiration Date: 2022-12-18T02:39:55Z Registrar: GANDI SAS Registrar IANA ID: 81 Registrar Abuse Contact Email: abuse@support.gandi.net Registrar Abuse Contact Phone: +33.170377661 Reseller:  Domain Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited Domain Status:  Domain Status:  Domain Status:  Domain Status:  Registry Registrant ID: REDACTED FOR PRIVACY Registrant Name: REDACTED FOR PRIVACY Registrant Organization: Lemon Inc Registrant Street: REDACTED FOR PRIVACY Registrant City: REDACTED FOR PRIVACY Registrant State/Province:  Registrant Postal Code: REDACTED FOR PRIVACY Registrant Country: KY Registrant Phone: REDACTED FOR PRIVACY Registrant Phone Ext: Registrant Fax: REDACTED FOR PRIVACY Registrant Fax Ext: Registrant Email: 56e8bba85d9fa2f377b82336dddc9453-21392617@contact.gandi.net Registry Admin ID: REDACTED FOR PRIVACY Admin Name: REDACTED FOR PRIVACY Admin Organization: REDACTED FOR PRIVACY Admin Street: REDACTED FOR PRIVACY Admin City: REDACTED FOR PRIVACY Admin State/Province: REDACTED FOR PRIVACY Admin Postal Code: REDACTED FOR PRIVACY Admin Country: REDACTED FOR PRIVACY Admin Phone: REDACTED FOR PRIVACY Admin Phone Ext: Admin Fax: REDACTED FOR PRIVACY Admin Fax Ext: Admin Email: 6fca5a57f41872bee38ff4f0fd2b71e2-20984083@contact.gandi.net Registry Tech ID: REDACTED FOR PRIVACY Tech Name: REDACTED FOR PRIVACY Tech Organization: REDACTED FOR PRIVACY Tech Street: REDACTED FOR PRIVACY Tech City: REDACTED FOR PRIVACY Tech State/Province: REDACTED FOR PRIVACY Tech Postal Code: REDACTED FOR PRIVACY Tech Country: REDACTED FOR PRIVACY Tech Phone: REDACTED FOR PRIVACY Tech Phone Ext: Tech Fax: REDACTED FOR PRIVACY Tech Fax Ext: Tech Email: 6fca5a57f41872bee38ff4f0fd2b71e2-20984083@contact.gandi.net Name Server: A6-65.AKAM.NET Name Server: A9-66.AKAM.NET Name Server: A13-67.AKAM.NET Name Server: A12-66.AKAM.NET Name Server: A18-64.AKAM.NET Name Server: A1-97.AKAM.NET Name Server:  Name Server:  Name Server:  Name Server:  DNSSEC: Unsigned URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/ >>> Last update of WHOIS database: 2020-10-13T06:18:02Z <<<  For more information on Whois status codes, please visit https://www.icann.org/epp  Reseller Email:  Reseller URL:   Personal data access and use are governed by French law, any use for the purpose of unsolicited mass commercial advertising as well as any mass or automated inquiries (for any intent other than the registration or modification of a domain name) are strictly forbidden. Copy of whole or part of our database without Gandi's endorsement is strictly forbidden. A dispute over the ownership of a domain name may be subject to the alternate procedure established by the Registry in question or brought before the courts. For additional information, please contact us via the following form: https://www.gandi.net/support/contacter/mail/
rodeodomino commented 4 years ago

Thanks!

Screen Shot 2020-10-13 at 12 05 59 PM

It definitely could be a different issue or blocklist – I just don't know how to troubleshoot any further... I added it to the allowlist and all seems to work fine.

Voltairine-de-Cleyre commented 4 years ago

Just be careful. It's odd that TikTok would contact a server hosting that sub/domain.

crssi commented 4 years ago

You can start here https://github.com/nextdns/metadata/blob/master/security/threat-intelligence-feeds.json to see which sources are involved in Threat Intelligence Feeds.

Then you search over those and you will see that your culprit is actually listed in a bunch of those:

http://phishing.mailscanner.info/phishing.bad.sites.conf
https://openphish.com/feed.txt
https://raw.githubusercontent.com/olbat/ut1-blacklists/master/blacklists/malware/domains
https://raw.githubusercontent.com/olbat/ut1-blacklists/master/blacklists/phishing/domains
https://phishing.army/download/phishing_army_blocklist.txt

This is a lot and I guess its not a mistake, due to number of listing. You can contact each one and argue to remove it.

Cheers

crssi commented 4 years ago

As @Voltairine-de-Cleyre mentined, I would really not put it on a allowlist.

liamengland1 commented 4 years ago

It's a legitimate tiktok domain.

romaincointepas commented 4 years ago

Is it breaking the use of TikTok in any way?

Edit: Nevermind, misread the original issue. Could you share a full example (with path, etc.) of one of those links?

rodeodomino commented 4 years ago

Yup @romaincointepas

For context, this issue happens when somebody shares a video through the TikTok app and is opened in iOS safari. I've tried the same link on desktop and it doesn't call the same URL in question (so it's really just on my phone that it's even asking for it). If you remove move of the URL parameters, TikTok shows a different page layout where it doesn't have the same problem (but these are all added during a share).

An example link is: https://m.tiktok.com/v/6883213902075759877.html?_d=secCgsIARCbDRgBIAIoARI%2BCjzUMqGXP1oASsjjyIlnH29wyckO7gvjRXgnXnXo5oa3qa0IqklLd2n2fJpJUvnFxJls3fwT%2FfDBOAH7BNwaAA%3D%3D&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAA8oecaE1zBK_dC2iQmACrhuYJ4Tb99mceyDNsp8NyQgtK0GN994rUKyEVc2rGF2Gp&share_item_id=6883213902075759877&share_link_id=918F3582-D993-4839-A3B1-95932103B389&timestamp=1602885682&tt_from=sms&u_code=d4khfbf9e43ehj&user_id=6659919579957805061&utm_campaign=client_share&utm_medium=ios&utm_source=sms

crssi commented 4 years ago

Interestingly it opens just fine in Firefox for iOS, but not in Safari.

With a fast check the Threat Intelligence Feeds is not the only list that blocks it, for example oisd also blocks it.

Voltairine-de-Cleyre commented 4 years ago

If any Safari iOS content blockers are enabled, that may be one culprit if it opens in Firefox iOS. Not that it's you wanting to reach it. Also, it may be that Safari iOS defaults to sending a very functional (as in it seems to mostly work) UA string for that of Safari MacOS unlike Firefox iOS which requires requesting the "desktop version" of websites instead of sending an altered UA.On Sat, Oct 17, 2020 at 14:05, crssi notifications@github.com wrote:
Interestingly it opens just fine in Firefox for iOS, but not in Safari. With a fast check the Threat Intelligence Feeds is not the only list that blocks it, for example oisd also blocks it. Why don't you try to persuade the maintainers of the lists where it is denied?

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

romaincointepas commented 4 years ago

Like @crssi figured out, it's blocked by:

http://phishing.mailscanner.info/phishing.bad.sites.conf
https://openphish.com/feed.txt
https://raw.githubusercontent.com/olbat/ut1-blacklists/master/blacklists/malware/domains
https://raw.githubusercontent.com/olbat/ut1-blacklists/master/blacklists/phishing/domains
https://phishing.army/download/phishing_army_blocklist.txt

We would really appreciate if someone could take the lead on this and report this to each of those feeds.

drego85 commented 4 years ago

Hi Guys, I have whitelisted the domain from phishing.army!

At the next update it will be removed and will never be inserted.

crssi commented 4 years ago

@romaincointepas http://phishing.mailscanner.info/phishing.bad.sites.conf is getting the data from https://www.phishtank.com/, but checking for sf16-muse-va.ibytedtos.com doesn't show its in the list. I cannot find a reporting site.

https://openphish.com/feed.txt <- how this is processed by NextDNS?... there is just one URL icluding this domain in the list.

@drego85 Thank you ❤️

drego85 commented 4 years ago

@crssi from my database, Phishtank never reported that domain.

But OpenPhish did it several times, with these urls:

http://sf16-muse-va.ibytedtos[.]com/obj/ad-audit-eyes-va/2333e9a34a9f2f94c2bc50d2cfb526d4.html
http://sf16-muse-va.ibytedtos[.]com/obj/ad-audit-eyes-va/47813990d258d303e6efa8d811d2a469.html
https://sf16-muse-va.ibytedtos[.]com/obj/ad-audit-eyes-va/a425a17ec99cd94792b134ee9d55620e.html

It was first reported on: 2020-09-11 12:39:27 UTC.

I believe mailscanner.info also acquires from OpenPhish, although it does not declare this.

The three urls shown by OpenPhish really look like phishing (PayPal clone).

It is possible that sf16-muse-va.ibytedtos[.]com is used as a CDN?

crssi commented 4 years ago

@drego85 Thank you

This is also reply from OpenPhish team

I checked our records and the domain hosts an active phishing targeting PayPal at the following address:

hxxps://sf16-muse-va[.]ibytedtos[.]com/obj/ad-audit-eyes-va/a425a17ec99cd94792b134ee9d55620e.html

OpenPhish includes only the actual phishing URL in its feeds and does not maintain any lists that block specific domains.

Please let me know if you have any other questions.

Best regards,
The OpenPhish Team

I guess we have done all we could to help. It now on @romaincointepas for further actions.

romaincointepas commented 1 year ago

Fixed