hagezi / dns-blocklists

DNS-Blocklists: For a better internet - keep the internet clean!
GNU General Public License v3.0
6.53k stars 217 forks source link

TikToktopus: profiles are blank / live function broken #657

Closed yokoffing closed 1 year ago

yokoffing commented 1 year ago

What adblocker/DNS cloud service are you using?

ControlD

Which blocklists are used?

Pro++

Which domain(s) should be unblocked?

I haven’t investigated. Reporting breaking on mobile.

Why should the domain(s) be unblocked? If necessary, please describe the steps to reproduce.

A4051DA0-0F82-411D-A741-D9344021C085

hagezi commented 1 year ago

Can't reproduce.

yokoffing commented 1 year ago

I’ll double-check other lists in ControlD (malware, phishing, etc.). There have been false positives lately.

hagezi commented 1 year ago

Have opened several profiles, no problem, which does not mean that it occurs over time, Tiktok is sometimes bitchy when it does not get rid of its fingerprints. Does this occur reproducibly for you? Restarted the app and then tried again?

yokoffing commented 1 year ago

It works fine after I accessed it with no DNS Blocking (used native resolver). I’m familiar with the concept you’re talking about. I wouldn’t be surprised if it applies here.

hagezi commented 1 year ago

The data octopus Tiktok is the plague when it comes to attempts to get rid of fingerprinting data. DoH bypasses are used (Google DNS), new log domains are constantly sprouting up and in the last instance the app pretends it has no internet connection.

I'll keep an eye on the behaviour and if it occurs, I'll have to see which blocked domain is causing it. There are a few Tiktok users in the family network.

yokoffing commented 1 year ago

Thank you, friend! Also, calling Tiktok a data octopus = 💯 😆

durablenapkin commented 1 year ago

TikToktopus

hagezi commented 1 year ago

@yokoffing Family has tested until the doctor came, no problem with empty profiles.

yokoffing commented 1 year ago

I haven't noticed any issue since I disabled secure DNS, opened app from native DNS, and then re-enabled secure DNS again. I'll be curious to see if the issue occurs again weeks from now.

dazzah87 commented 1 year ago

I’m using the Pro++ list and can easily reproduce the empty profile pages. I haven’t done any investigation yet but these domains are currently blocked:

I have no idea what they do but I see similar domains not blocked. Maybe you have some insight of what they do or if they are „safe“ to add to your or my personal Allowlist? Otherwise, I’d go through them one by one and see if unblocking helps with the empty profile pages.

hagezi commented 1 year ago

@lyrad87 These are fingerprinting domains. Please go through them one by one and see if unblocking helps with the empty profile pages. Many thanks!

dazzah87 commented 1 year ago

I tried with all of them one by one and nothing worked. Then I just added .tiktokv.com and I saw the profile pages. Then I deleted .tiktokv.com from the Allowlist and now I see the profile pages again with all of the before mentioned domains being blocked. TikTok is so frustrating. If only I could just delete it but happy wife happy life.

SeriousHoax commented 1 year ago

I don't use tiktok so don't have it installed on my phone. I checked on my PC where I'm using Adguard Home with Hagezi's DNS - Pro Plus filter. Here's how it is with this list as well as Adguard Extension as my adblocker.

1

With Adguard extension, the only dns-based block I see on my query is mssdk-va.tiktok.com But as you can see, it shows empty at first and but for some reason, everything is fine if I reload my browser tab.

Anyway, after I allowed mssdk-va.tiktok.com in Adguard Home, everything is fine. So, looks like it's the culprit.

2

I don't want to install TikToktopus on my phone, so can't test it there @yokoffing @lyrad87 Make sure to disable phone's wifi after you whitelist the domain to make sure DNS query is not cached.

hagezi commented 1 year ago

Many thanks @SeriousHoax!

dazzah87 commented 1 year ago

Thanks for the ping and research! The mentioned domain has never appeared in my logs so I’m not sure if it’s the (only) culprit. Adding it to the Allowlist wouldn’t hurt I’d assume. After shortly allowing *.tiktokv.com for testing purposes, my wife hasn’t encountered an empty profile page.

hagezi commented 1 year ago

I think mssdk-va.tiktokv.com is called in mobile app.

dazzah87 commented 1 year ago

In my case, my wife is using the mobile Android app. She has never, to my knowledge, used TikTok in a browser.

yokoffing commented 1 year ago

Idk how much time to give between reports, but so far, so good — in terms of overt breakage.

I notice sometimes that TikTok recycles content that I’ve already see. I have no idea if this is due of DNS blocking or TikTok just being bad.

hagezi commented 1 year ago

I noticed that too, I think it has to do with the fact that we block logging completely. Makes sense...

yokoffing commented 1 year ago

IMG_0577

Started back again today. Can’t make this up lol.

hagezi commented 1 year ago

Now we just have to find out which blocked domain leads to this ... :grin:

Perhaps it is also due to the bamboo network line to the server where damian's profile is stored. :grimacing:

hagezi commented 1 year ago

Live function broken: https://github.com/hagezi/dns-blocklists/issues/797#issuecomment-1501120725

jostrasser commented 1 year ago

Hi @hagezi

Unblocking *mssdk* wasn't the solution I think.

TikTok Live - Top Viewers is also broken and isn't loading (white screen). Behavior is the same, if I disable my NextDNS profile it will load correctly but after re-enabling it I cannot reproduce it anymore.

I think we have to investigate more. It's really difficult.

hagezi commented 1 year ago

Thanks, then the next ones would be pitaya, frontier, gecko ...

jostrasser commented 1 year ago

Thanks, then the next ones would be pitaya, frontier, gecko ...

I hope somebody here has another device with the same behaviors to test ;)

I'll try to prepare a new iOS device tomorrow. Hopefully I run into the same problems.

hagezi commented 1 year ago

@jostrasser If this happens again, please selectively white list them, close the app, disable/enable wifi so they are not cached and wait a few minutes for NextDNS to stop caching them.

hagezi commented 1 year ago

I have taken back the whitelisting of *mssdk*.

yokoffing commented 1 year ago

Perhaps it is also due to the bamboo network line to the server where damian's profile is stored. 😬

Lol. This happened on every profile. That just happens to be the one I landed on for the screenshot.

yokoffing commented 1 year ago

Hopefully, this is not seen as blasphemy: just hear me out (lol). What if we remove all the Tiktok domains and then add chunks of them week-by-week? A controlled release.

Since there's no documentation or information anywhere, we can slowly test for breakage or changes.

So far I see:

hagezi commented 1 year ago

The other way would be to whitelist selectively and see which symptoms disappear. Personally, I think this is the better option. Alternatively, you can also test the Multi normal/pro, in which only the following domains are blocked:

||*-ad.byteoversea.com^
||*-mon-*.bytegsdk.com^
||*-mon-*.byteoversea.com^
||ads*.tiktok.com^
||analytics*.tiktok.com^
||ib.snssdk.com^
||log*.bytegsdk.com^
||log*.byteoversea.*^
||log*.snssdk.com^
||log*.tiktok*.com^
||mon*.byteoversea.com^
||mon*.tiktok*.com^
||rtc-log*.tiktok*.com^
||rtlog*.byteoversea.com^
||rtlog*.snssdk.com^
||rtlog*.tiktok*.com^
||xlog*.byteoversea.com^
||xlog*.snssdk.com^
||xlog*.tiktok*.com^

No pitaya, frontier, gecko, mssdk, mcs domains.

hagezi commented 1 year ago

Which I and all the "hardcore tik tokers" on Android/iOS in the family can't reproduce:

No idea why this is so selective and more people haven't come forward about this.

yokoffing commented 1 year ago

Alternatively, you can also test the Multi normal/pro, in which only the following domains are blocked:

I'll switch to that for a few days and see what changes.

I and all the "hardcore tik tokers" on Android/iOS in the family can't reproduce

Based on my experience with a previous issue (mcs domain), TikTok domains change based on location. It's very annoying.

jostrasser commented 1 year ago

FYI: I am using the Multi PRO with the same behavior.

hagezi commented 1 year ago

@jostrasser And which domains are blocked before the problem occurs?

jostrasser commented 1 year ago

@jostrasser And which domains are blocked before the problem occurs?

mon-i18n.tiktokv.com pitaya-task-i18n.tiktokv.com rtlog16-normal-useast2a.tiktokv.com rtlog22-normal-useast2a.tiktokv.com gecko16-normal-useast2a.tiktokv.com mssdk19-normal-useast2a.tiktokv.com log16-normal-useast2a.tiktokv.com pitaya-i18n.tiktokv.com frontier-i18n.tiktokv.com mssdk16-normal-useast2a.tiktokv.com gecko31-normal-useast2a.tiktokv.com log22-normal-useast2a.tiktokv.com

As mentioned in the following issue: https://github.com/hagezi/dns-blocklists/issues/797#issuecomment-1501120725

I always used the PRO list.

hagezi commented 1 year ago

@jostrasser pitaya, frontier, gecko, mssdk, mcs domains are no longer included in the pro since yesterday.

hagezi commented 1 year ago

Only log and mon tiktokv.com domains are included:

log.tiktokv.com
log-i18n.tiktokv.com
log-phx.tiktokv.com
log-ru.tiktokv.com
log-va.tiktokv.com
log-va-useast2a.tiktokv.com
log15-normal-alisg.tiktokv.com
log15-normal-useast2a.tiktokv.com
log16-normal.tiktokv.com
log16-normal-alisg.tiktokv.com
log16-normal-c-useast1a.tiktokv.com
log16-normal-c-useast2a.tiktokv.com
log16-normal-delay-alisg.tiktokv.com
log16-normal-ind.tiktokv.com
log16-normal-useast1a.tiktokv.com
log16-normal-useast2a.tiktokv.com
log16-platform-ycru.tiktokv.com
log16-tmp-normal-useast1a.tiktokv.com
log17-normal-alisg.tiktokv.com
log17-normal-useast1a.tiktokv.com
log17-normal-useast2a.tiktokv.com
log19-applog-useast1a.tiktokv.com
log19-applog-useast2a.tiktokv.com
log19-normal.tiktokv.com
log19-normal-alisg.tiktokv.com
log19-normal-useast1a.tiktokv.com
log19-normal-useast2a.tiktokv.com
log22-applog-useast1a.tiktokv.com
log22-applog-useast2a.tiktokv.com
log22-normal.tiktokv.com
log22-normal-alisg.tiktokv.com
log22-normal-useast1a.tiktokv.com
log22-normal-useast2a.tiktokv.com
log3-normal-c-alisg.tiktokv.com
log3-normal-c-useast1a.tiktokv.com
log31-normal-alisg.tiktokv.com
log31-normal-useast2a.tiktokv.com
log53-normal-c-useast1a.tiktokv.com
log58-normal-c-alisg.tiktokv.com
log58-normal-c-useast1a.tiktokv.com
log58-normal-useast1a.tiktokv.com
log58-normal-useast2a.tiktokv.com
log61-normal-c-alisg.tiktokv.com
log61-normal-c-useast1a.tiktokv.com
log77-normal-c-alisg.tiktokv.com
log77-normal-c-useast1a.tiktokv.com
log9-normal-c-useast1a.tiktokv.com
logbk.tiktokv.com
mon.tiktokv.com
mon-i18n.tiktokv.com
mon-phx.tiktokv.com
mon-ru.tiktokv.com
mon-sg.tiktokv.com
mon-useast2a.tiktokv.com
mon-va.tiktokv.com
mon15-checkout-alisg.tiktokv.com
mon15-checkout-useast1a.tiktokv.com
mon15-normal.tiktokv.com
mon15-normal-useast1a.tiktokv.com
mon15-normal-useast1a-checkout.tiktokv.com
mon16-normal.tiktokv.com
mon16-normal-alisg.tiktokv.com
mon16-normal-c-useast1a.tiktokv.com
mon16-normal-c-useast2a.tiktokv.com
mon16-normal-useast1a.tiktokv.com
mon16-platform-ycru.tiktokv.com
mon16-tmp-normal-useast1a.tiktokv.com
mon19-normal.tiktokv.com
mon19-normal-alisg.tiktokv.com
mon19-normal-useast1a.tiktokv.com
mon22-normal.tiktokv.com
mon22-normal-alisg.tiktokv.com
mon22-normal-useast1a.tiktokv.com
mon31-normal.tiktokv.com
mon31-normal-alisg.tiktokv.com
mon31-normal-useast1a.tiktokv.com
mon32-normal-useast1a.tiktokv.com
monbk.tiktokv.com
rtc-logger-va.tiktokv.com
rtlog.tiktokv.com
rtlog-i18n.tiktokv.com
rtlog-phx.tiktokv.com
rtlog-ru.tiktokv.com
rtlog-va.tiktokv.com
rtlog-va-useast2a.tiktokv.com
rtlog15-normal-useast1a.tiktokv.com
rtlog15-normal-useast2a.tiktokv.com
rtlog16-applog-useast1a.tiktokv.com
rtlog16-applog-useast2a.tiktokv.com
rtlog16-normal.tiktokv.com
rtlog16-normal-alisg.tiktokv.com
rtlog16-normal-ind.tiktokv.com
rtlog16-normal-useast1a.tiktokv.com
rtlog16-normal-useast2a.tiktokv.com
rtlog16-tmp-normal-useast1a.tiktokv.com
rtlog17-normal.tiktokv.com
rtlog17-normal-alisg.tiktokv.com
rtlog17-normal-useast1a.tiktokv.com
rtlog17-normal-useast2a.tiktokv.com
rtlog19-applog-useast1a.tiktokv.com
rtlog19-applog-useast2a.tiktokv.com
rtlog19-normal.tiktokv.com
rtlog19-normal-alisg.tiktokv.com
rtlog19-normal-useast1a.tiktokv.com
rtlog19-normal-useast2a.tiktokv.com
rtlog22-applog-useast2a.tiktokv.com
rtlog22-normal.tiktokv.com
rtlog22-normal-alisg.tiktokv.com
rtlog22-normal-useast1a.tiktokv.com
rtlog22-normal-useast2a.tiktokv.com
rtlog31-normal.tiktokv.com
rtlog31-normal-useast1a.tiktokv.com
rtlog58-normal-alisg.tiktokv.com
log16-applog-useast5.us.tiktokv.com
log19-applog-useast5.us.tiktokv.com
mon.us.tiktokv.com
mon16-platform-useast5.us.tiktokv.com
rtc-logger.us.tiktokv.com
rtlog16-applog-useast5.us.tiktokv.com
xlog16-platform-useast5.us.tiktokv.com
xlog.tiktokv.com
xlog-ru.tiktokv.com
xlog-va.tiktokv.com
xlog-va-useast2a.tiktokv.com
xlog16-normal-c-useast1a.tiktokv.com
xlog16-normal-c-useast2a.tiktokv.com
xlog16-platform-ycru.tiktokv.com
xlog16-tmp-normal-useast1a.tiktokv.com
jostrasser commented 1 year ago

@jostrasser pitaya, frontier, gecko, mssdk, mcs domains are no longer included in the pro since yesterday.

I missed that, sry. ;)

yokoffing commented 1 year ago

Search is not working for me. I get "No results are found". Comments are broken again. Here's some logs:

logs1a

logs2a

hagezi commented 1 year ago

Then I guess again that it's because of sf*, or some TikTok servers.

yokoffing commented 1 year ago

@hagezi I will allowlist sf16-muse-va.ibytedtos.com and see what happens.

I don't need to worry about mon16-platform-useast5.us.tiktokv.com?

hagezi commented 1 year ago

Unblocked, except for ultimate:

*mssdk*.tiktok*.com
*mssdk*.byteoversea.com
*mssdk*.snssdk.com
sf*-*.ibytedtos.com
sf*-*.tiktok*.com

For mssdk: https://github.com/badmojr/1Hosts/issues/1415

yokoffing commented 1 year ago

mon16-platform-useast5 is unblocked since when? Today? Those screenshots from yesterday were all blocked requests from Pro++.

Edit: I still have issues today. I'll try again tomorrow.

hagezi commented 1 year ago

mon ist not unblocked, only sf and mssdk*

yokoffing commented 1 year ago

Allowlisting mon16-platform-useast5 fixed my issues from yesterday.

hagezi commented 1 year ago

OK, thanks, then I will have to unblock the mon domains.

yokoffing commented 1 year ago

TikTok works well for me. Removing mon and your recent adjustments helped.

hagezi commented 1 year ago

I think we can close here for now.

nightznero commented 1 year ago

@hagezi sorry for bothering bud but... is there any list of which domains should be whitelisted or at least list with what domains has been at the end deleted? Cuz some others filter lists block some and dunno which one Is safe to whitelist. Best Regards, NIGHTZNERO.

hagezi commented 1 year ago

@nightznero

At Tiktok, this seems to depend entirely on the humidity, moon phase, star constellation, sunrise and whatnot. As you can see from this thread, depending on the region, it's a guess. One has problems, the other doesn't.

I have removed all domains from the lists light to pro++ that somehow, somewhere caused problems. I didn't feel like playing the guessing game any more.

That's why there are two list versions: https://github.com/hagezi/dns-blocklists#native

I use the aggressive version in the family and have no problems.

The best solution is not to use TikTok and to block it completely.