AdguardTeam / AdguardFilters

AdGuard Content Blocking Filters
https://adguard.com/
GNU General Public License v3.0
3.18k stars 631 forks source link

www.nicovideo.jp #150584

Closed adguard-bot closed 1 year ago

adguard-bot commented 1 year ago

Issue URL (Ads)

https://www.nicovideo.jp/watch/sm42203375

Comment

Username: @stm-x

Tracker on nicovideo.jp. The first screenshot is observed when browsing videos when logged in. It seems that the tracker can no longer be blocked due to directory changes on the site side. The second screenshot is observed when viewing videos without login I propose following rules. ||nicovideo.jp//user/actions/watch-events.json ||nicovideo.jp//user/actions/watch-events/nonmember.json

Screenshots

Screenshot 1 ![Screenshot 1](https://reports-img.adguard.com/fNO0TFJ.png)
Screenshot 2 ![Screenshot 2](https://reports-img.adguard.com/7alZjIV.png)

System configuration

Information value
AdGuard product: AdGuard Browser extension v4.1.55
Browser: Chrome
Stealth mode: disabled
Filters: Ad Blocking:
AdGuard Base

Privacy:
AdGuard Tracking Protection

Language-specific:
AdGuard Japanese
jellizaveta commented 1 year ago

Cannot reproduce nicovideo.jp//user/actions/watch-events.json

stm-x commented 1 year ago

@jellizaveta Yes, That tracker can only be reproduced when logged in to the site. Also, as shown in the screenshot, the "?" mark may not be given ( in the case of embedded players example: https://www.nicovideo.jp/). Therefore, the "?" mark may be better removed.

@Yuki2718 can you confirm?

Yuki2718 commented 1 year ago

I can reproduce. It seems astral.nicovideo.jp is dedicated for these analytics (TBH not much privacy-invasive, only sending event type and minimal set of cookies). I wonder if the domain can safely be blocked, as I don't use nicovideo. @stm-x Can you test ||astral.nicovideo.jp^?

stm-x commented 1 year ago

Ok, I will block the domain for about 3 days and report the results.

stm-x commented 1 year ago

After looking at loggers for about 3 days, I found several that I thought were trackers. ! for PC ! https://www.nicovideo.jp/watch/sm42216393 When logged in, select related videos from menu beside video player. /ログイン時、動画プレイヤー横メニューから関連動画選択  ||public.api.nicovideo.jp/*/recommend/log/response-log

! for mobile ! https://sp.nicovideo.jp/watch/sm42216393 analytics:When visiting the site. /サイト訪問時 click:Comment list, volume auto-adjustment ON,OFF. コメントリスト,音量自動調整表示ON,OFF error:Error(I forgot to take a screenshot.) ||sp.nicovideo.jp/api/counter/analytics/ ||sp.nicovideo.jp/api/counter/click/ ||sp.nicovideo.jp/api/counter/error/

! 2023-05-12 https://sp.live.nicovideo.jp logger:When watching live streaming. /生放送ストリーミング中 example: https://sp.live.nicovideo.jp/watch/lv341339885 Note:that it must be live on the air to reproduce. ||sp.live.nicovideo.jp/api/logger?

stm-x commented 1 year ago

About the blocking of ||astral.nicovideo.jp^, I have tried the various nicovideo services on the website (PC,mobile) and the iOS app version with no breakage and no any issues.

stm-x commented 1 year ago

The PC version of live.nicovideo.jp still uses public.api.nicovideo.jp/v2/user/actions/watch-events.json? The following rules probably still be necessary. ||nicovideo.jp/*/user/actions/video/watch-events.json? -> ||nicovideo.jp/*/user/actions/watch-events.json

Yuki2718 commented 1 year ago

Thanks for the thorough analysis, fixed all.

stm-x commented 1 year ago

Thanks!

stm-x commented 1 year ago

UPDATE: The site has been updated and the tracker URL has changed. www.nicovideo.jp/v1/recommend/log/ -> log.recommend.nicovideo.jp/v2/display_log

Screenshot ![タイトルなし](https://github.com/AdguardTeam/AdguardFilters/assets/86018494/6bbab791-8cf5-41af-a7d4-87d43ec2f5da) ![タイトルなし2](https://github.com/AdguardTeam/AdguardFilters/assets/86018494/b7b52227-1e55-4591-9723-4be57c060980)

I have blocked this domain for about 3 days now and have not had any problems with the mobile app and the web version. Also, as before, live.nicovideo.jp still uses the old URLs, so the current rules must be kept.

@Yuki2718 Can you check this?

stm-x commented 1 year ago

@Yuki2718 Thank you. So live.nicovideo.jp still uses public.api.nicovideo.jp/v1/recommend/log/, so the previous rule is still needed (see the second screenshot)

Thanks again😊

Yuki2718 commented 1 year ago

Whoops, missed that. Thanks for pointing out.

stm-x commented 1 year ago

Since live.nicovideo.jp and embedded players seem to have migrated to astral.nicovideo.jp, the following rules are no longer needed. ||nicovideo.jp/v2/user/actions/watch-events.json ||nicovideo.jp/*/user/actions/watch-events/nonmember.json? ||nicovideo.jp/*/user/actions/video/watch-events.json? ||nicovideo.jp/v1/recommend/log/

@Yuki2718

stm-x commented 6 months ago

Hello, @Yuki2718. It's been a long time.

There have been some changes to the site.

  1. The tracker domain has changed. astral.nicovideo.jp -> stella.nicovideo.jp Videos can no longer be viewed if the tracker is blocked. This can be bypassed using the noopjson rule. ||stella.nicovideo.jp/v2/watch/member.json$redirect=noopjson ||stella.nicovideo.jp/v1/watch/nonmember.json$redirect=noopjson
    Screenshot

タイトルなし

  1. There is nicoad tracking in the following directories. https://www.nicovideo.jp/video_top https://www.nicovideo.jp/video_top/genre/entertainment Tracker: ||nicoad.nicovideo.jp/v1/instream/tracking/ ||nicoad.nicovideo.jp/v1/tracking/
    Screenshot

タイトルなし2

  1. Also, there is a known tracker that is only blocked by AdGuard, can you please add these to EasyPrivacy? Thank you. ||nvapi.nicovideo.jp/v1/bandit-machine/ ||nicovideo.jp/api/logger? ||nicolive.cdn.nimg.jp/relive/sp/browser-logs.*.js ||nicolive.cdn.nimg.jp/relive/sp/trackingService.*.js
Yuki2718 commented 6 months ago

@slavaleleka How can I test the new prevent-fetch? nicovideo.jp#%#//scriptlet('prevent-fetch', 'stella') doesn't prevent fetch here with 4.3.46 or Beta.

Yuki2718 commented 6 months ago

I can't find nicovideo.jp/api/logger? or nicolive.cdn.nimg.jp/relive/sp/trackingService. request even with mobile UA and logged in.

Yuki2718 commented 6 months ago

Disabled for https://twitter.com/kuroneko8823/status/1777441034689224774. I can reproduce on Android. I worried something like this as $redirect sometimes donesn't work, this is why I wanted to use scriptlet if it worked (but not).

stm-x commented 6 months ago

I can't find nicovideo.jp/api/logger? or nicolive.cdn.nimg.jp/relive/sp/trackingService. request even with mobile UA and logged in.

My bad. I didn't explain myself well enough. Those two were used on sp.live.nicovideo.jp ,not sp.nicovideo.jp example: https://sp.live.nicovideo.jp/watch/lv344865419 I just looked at the logs about the tracker,nicolive.cdn.nimg.jp/relive/sp/trackingService. is still in use, but I didn't see nicovideo.jp/api/logger?

Screenshot ![タイトルなし3](https://github.com/AdguardTeam/AdguardFilters/assets/86018494/09f3fc04-8989-4219-adf2-03baa815f063)

As for the latter, maybe it has been discontinued.

stm-x commented 6 months ago

Also astral.nicovideo.jp is still being used by the app and needs to be blocked.

Screenshot ![IMG_1311](https://github.com/AdguardTeam/AdguardFilters/assets/86018494/af904804-6450-4290-a990-9b71c7d65756)

I apologize for the incorrect explanation.

otksm commented 6 months ago

https://github.com/easylist/easylist/commit/380aced24a53f23b7c0665eace7987e306a7ffc3

@Yuki2718 FWIW if you watch the uBO logger, no-fetch-if won't able to block the eventType: "end" impression when leaving the video page or closing the tab. You will also need an additional filter using no-xhr-if to cover for it (for some reason).

Yuki2718 commented 6 months ago

I can't reproduce on Chrome. Can you show the screenshot of the log?

otksm commented 6 months ago

I am also not reproducing on Chromium. Perhaps a Firefox-specific issue?

logger

Yuki2718 commented 6 months ago

@Alex-302 Can I add the scriptlet rules to TPF even though they don't work until the new version? Otherwise can be forgot.

Alex-302 commented 6 months ago

@Yuki2718 Maybe) Which rules?

Yuki2718 commented 6 months ago
nicovideo.jp#%#//scriptlet('prevent-fetch', 'stella')
nicovideo.jp#%#//scriptlet('prevent-xhr', 'stella')
Alex-302 commented 6 months ago

ok. But why only one word instead of domain/URL?

Yuki2718 commented 6 months ago

Added with full domain

Alex-302 commented 6 months ago

Thanks.