Dynamic/embedded podcast ads not blocked

Fabian42 commented 3 months ago

🧐 Have you fully read the README and viewed similar issues?

[X] I have read the README
[X] I have read the FAQ
[X] I have checked previous issues, open and closed
[X] I have completed all troubleshooting steps and the issue is not present on stock but returns after re-applying SpotX-Bash

Сountry of your account

Germany

What is your Spotify Plan?

Free

Do you have Spicetify installed?

No

ℹ Computer information

- Spotify version: spotify-1:1.2.31.1205-2
- OS: Manjaro
- Shell: Bash

📝 Description

After applying, audio ads still play. Here an ad plays at the beginning. I do have the Spicetify package installed, but I used spicetify restore before applying SpotX. I also performed the reinstallation steps from the troubleshooting page without doing anything Spicetify-related in the meantime. (I removed a ton of locale warnings (and the ASCII art) from the output, which appear all the time on my system, but never have any effects. I tried to fix them, but did not succeed. It should be unrelated to this problem.)

🖥️ Terminal Input/Output

Searching for Spotify directory...

Found Spotify Directory: /opt/spotify

Latest supported version: 1.2.34.783 Detected Spotify version: 1.2.31.1205

✔ Created backup ✔ Applied free-tier plan patches ✔ Enabled experimental features ✔ Finished

📸 Screenshots

No response

jetfir3 commented 3 months ago

Without full input/output it's hard to be completely sure.. but the link you shared is for a podcast. If all the ads you are receiving are with podcasts then this is expected behavior at this time.

Linux Spotify clients AFTER v1.2.26.1187 do not support complete podcast ad blocking. You'll have to decide if you want to downgrade to 1.2.26.1187 for podcast ad block or use 1.2.31+ and deal with podcasts ads.

Banner ads in client and ads between music should be blocked, so everything seems to be working as expected. Podcast ads on newer clients may be blocked again in the future.

Fabian42 commented 3 months ago

I just replaced the spotify package with spotify-snapstore, which lists its version as 1.2.22.982.g794acc0a-1, then re-applied SpotX, but still nothing. I do not know how I would get an older version of the spotify package. The downgrade script finds nothing for it.

jetfir3 commented 3 months ago

spotify-snapstore being the snap version of Spotify? If so, snap builds are not supported since they are "read-only containers"... this is covered in the FAQ.

Not sure how you're installing Spotify client but you may want to look into flatpak builds or spotify-launcher and see if older builds are possible there. Or deal with podcasts ads on the latest version and wait for podcast ads to be blocked again for future builds.

But how to downgrade or install Spotify client is outside the scope of the initial issue reported here as well as this repo in general. As previously mentioned, podcast ads are to be expected for now while using Linux client v1.2.31. The initial issue reported is something currently expected so nothing to fix for now.

Fabian42 commented 3 months ago

I did essentially this now:

flatpak install com.spotify.Client
sudo flatpak update --commit=d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84 com.spotify.Client
spotx -P /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify
flatpak run com.spotify.Client

Still ads. flatpak info com.spotify.Client confirms the version as 1.2.26.1187.g36b715a1.

jetfir3 commented 3 months ago

Are the ads still only podcast ads? You never specifically answered this question when I asked previously but I'm assuming this is still the case. I can't replicate this behavior on the v1.2.26 Linux client. I have specific podcasts which always play ads and use them for testing -- unpatched and ads play.. patched and ads stop (I'm also confirming with additional logging that the patches are indeed working and it's not just by chance that ads aren't playing during the test run).

If it's still ONLY podcast ads playing then this will still get marked as a "won't fix" for now. But I am curious about what could be happening.

Full input and output from terminal (minus any personal info) would be nice. Cleaning anything beyond that may or may not be removing helpful info. If re-patching Spotify to get terminal output you'd need to run it with the -f flag to force re-patching.
Screenshot showing full client when this happens

output of the following two commands:

ls -la /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify/

ls -la /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify/Apps/

upload Spotify binary located at /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify/spotify

Fabian42 commented 3 months ago

Since ads in between music tracks are only intermittent and I normally never use Spotify, I know nothing about those.

[fabian@laptop ~]$ spotx -fP /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify
*

████╗███╗  ███╗ █████╗█╗  █╗  ███╗  ██╗ ████╗█╗ █╗
█╔══╝█╔═█╗█╔══█╗╚═█╔═╝╚█╗█╔╝  █╔═█╗█╔═█╗█╔══╝█║ █║
████╗███╔╝█║  █║  █║   ╚█╔╝██╗███╔╝████║████╗████║
╚══█║█╔═╝ █║  █║  █║   █╔█╗╚═╝█╔═█╗█╔═█║╚══█║█╔═█║
████║█║   ╚███╔╝  █║  █╔╝ █╗  ███╔╝█║ █║████║█║ █║
╚═══╝╚╝    ╚══╝   ╚╝  ╚╝  ╚╝  ╚══╝ ╚╝ ╚╝╚═══╝╚╝ ╚╝

Using Spotify Directory: /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify

Latest supported version: 1.2.34.783
Detected Spotify version: N/A

*
✔ Detected & restored backup
*********
✔ Applied free-tier plan patches
*********************************************************************************************************************
✔ Enabled experimental features
***************************************************
✔ Finished

Each * represents this locale error that I have been getting everywhere for a few weeks now, which do nothing except spamming the console:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "en_GB:de:ja",
        LC_ALL = (unset),
        LC_ADDRESS = "de_DE.UTF-8",
        LC_NAME = "de_DE.UTF-8",
        LC_MONETARY = "de_DE.UTF-8",
        LC_PAPER = "de_DE.UTF-8",
        LC_IDENTIFICATION = "de_DE.UTF-8",
        LC_TELEPHONE = "de_DE.UTF-8",
        LC_MEASUREMENT = "de_DE.UTF-8",
        LC_TIME = "en_SE.UTF-8",
        LC_NUMERIC = "en_GB.UTF-8",
        LANG = "en_GB"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

A screenshot would do nothing, it just looks as if it was playing the podcast. But it clearly does not, since the ads are in German.

[fabian@laptop ~]$ ls -la /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify/
total 309016
drwxrwxrwx 5 root   root        4096 Mar 26 16:47 .
drwxr-xr-x 3 root   root        4096 Dec  4 17:42 ..
drwxrwxrwx 2 root   root        4096 Mar 26 16:47 Apps
drwxrwxrwx 2 root   root        4096 Dec  4 17:42 icons
drwxrwxrwx 2 root   root        4096 Dec  4 17:42 locales
-rw-rw-rw- 1 root   root      687134 Nov 30 07:16 chrome_100_percent.pak
-rw-rw-rw- 1 root   root     1072982 Nov 30 07:16 chrome_200_percent.pak
-rw-rw-rw- 1 root   root    10717392 Nov 30 06:01 icudtl.dat
-rw-rw-rw- 1 root   root      255064 Dec  4 17:38 libEGL.so
-rw-rw-rw- 1 root   root     6508624 Dec  4 17:38 libGLESv2.so
-rw-rw-rw- 1 root   root   207641280 Dec  4 17:38 libcef.so
-rw-rw-rw- 1 root   root     4050304 Dec  4 17:38 libvk_swiftshader.so
-rw-rw-rw- 1 root   root      617576 Dec  4 17:38 libvulkan.so.1
-rw-rw-rw- 1 root   root     8163591 Nov 30 08:56 resources.pak
-rw-rw-rw- 1 root   root      281350 Nov 30 07:35 snapshot_blob.bin
-rwxrwxrwx 1 fabian fabian  37855840 Mar 26 16:47 spotify
-rwxrwxrwx 1 fabian fabian  37855840 Mar 26 04:14 spotify.bak
-rw-rw-rw- 1 root   root      647433 Nov 30 09:03 v8_context_snapshot.bin
-rw-rw-rw- 1 root   root         107 Nov 30 07:13 vk_swiftshader_icd.json
[fabian@laptop ~]$ ls -la /var/lib/flatpak/app/com.spotify.Client/x86_64/stable/d22ab2f6059cca52bc56754ca1860fb41b0d9c6333975bd8f2eb375fde5a4d84/files/extra/share/spotify/Apps/
total 19712
drwxrwxrwx 2 root   root      4096 Mar 26 16:47 .
drwxrwxrwx 5 root   root      4096 Mar 26 16:47 ..
-rw-rw-rw- 1 root   root   3519048 Dec  4 17:38 login.spa
-rw-rw-rw- 1 fabian fabian 8318398 Mar 26 04:14 xpui.bak
-rw-rw-rw- 1 fabian fabian 8320176 Mar 26 16:47 xpui.spa

Binary: https://drive.google.com/file/d/1wuIxSVoR_eHkEprqTOPLTFAnb_r8bLfP

jetfir3 commented 3 months ago

A screenshot would do nothing, it just looks as if it was playing the podcast. But it clearly does not, since the ads are in German.

Thanks for info. You can delete the file on Google Drive.

If the track does not switch to an ad-track during ad-playback then the ad is being embedded into the podcast stream itself. This won't be blocked.

A "traditional" podcast ad will show itself around the 10-15 second mark on select podcasts. The podcast itself will play from 00:00 - 00:10 or so.. then the Now Playing screen will switch to an entirely new "track" while the ad plays.. displaying ad-related pictures and mentioning the ad/company name as the "Now Playing" track name. These can be fast forwarded or played at normal speed. After the ad ends the Now Playing track will change names again, back to the original podcast.

This is how both normal "music" ads and "podcast" ads are handled. SpotX-Bash blocks these music ads on both macOS and Linux, all podcast ads on macOS and all podcast ads on Linux with the exception of client v1.2.31.x.

The ads you're describing never switch "tracks" and instead are dynamically embedded into the audio stream itself. Blocking won't be possible. The only way to potentially skip these would be if Spotify added timestamps in their API, similar to Youtube and a system implemented like sponsorblock.. where a 3rd-party API can control the timestamps and control skipping sections of the audio/video track, skipping past the ads within the stream. Spotify currently has no system in place to handle timestamps.

Fabian42 commented 3 months ago

Some other Spotify adblockers want to mute ads, that would also be OK, but they also currently do not work. I tried browser extensions, client modifiers like this one, custom clients, downloaders, other podcast platforms, … The ads are all the same on all the podcast platform, but not originally part of the podcast (so not the host talking about sponsors). They seem to all be ≈28.5s long. Spotify could of course just stream a file to me that contains both, then it would be basically impossible to differentiate, but then I am surprised that the ad is exactly the same on Google Podcasts, Amazon Music and Deezer. So either there is one system which all these four podcast services use as an ad source or the uploader actually just inserted German ads into an English stream (probably not). There seems to be no SponsorBlock-like service for anything other than YouTube, so I will contact the host to please upload the full episodes on YouTube, that is the only option I still see. (Or I could pay for something, but on Amazon even that is currently bugged. That does not exactly motivate me to pay for yet another service and hoping that it works there.)

jetfir3 commented 3 months ago

Actually, I won't say blocking this will not be possible... but.. not something which is expected of this repo at this time. Something to add in the future perhaps.

Interesting that you're seeing same ad on all sources. I guess everyone's doing this now. The ad you are hearing is certainly not part of the original audio, but is dynamically inserted into the stream.. the ad is also targeted and based on your IP and possibly other metrics.

I've managed to target the ad and see how it's being inserted into the audio stream... but not sure how to handle it just yet. We'll see what the future holds.

SpotX-Official / SpotX-Bash