search

librespot-org / librespot

Open Source Spotify client library
MIT License
4.53k stars 545 forks source link

Suddenly unable to play any tracks #972

Closed kosimst closed 2 years ago

kosimst commented 2 years ago

I am using LibreSpot with the JACK Audio backend, and it worked perfectly, but since a few days I can't play any tracks. I didn't change any configurations on my side, so I assume there is a problem with spotify like it already occured here: #510

Setup

Custom compiled with --no-default-features and --features jackaudio-backend running as a user service on a Raspberry Pi.

Used Arguments:

LibreSpot allows connections from any device just fine, but can't play any song, no matter of its length. Mots of the time it just skips every song after a few seconds without any audio played, but sometimes it also crashes completely.

Logs:

Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z WARN  librespot_playback::player] Skipping to next track, unable to load track <SpotifyId { id: 38045326986839703698416089418933192304, audio_type: Track }>: ()
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z WARN  librespot_playback::player] Skipping to next track, unable to load track <SpotifyId { id: 38045326986839703698416089418933192304, audio_type: Track }>: ()
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: thread '<unnamed>' panicked at 'Map must not be polled after it returned `Poll::Ready`', /home/pi/.cargo/registry/src/github.com-1285ae84e5963aae/futures-util-0.3.17/src/future/future/map.rs:62:17
Mar 02 10:48:40 raspberrypi librespot[714]: stack backtrace:
Mar 02 10:48:41 raspberrypi librespot[714]:    0:   0x8febfc - std::backtrace_rs::backtrace::libunwind::trace::h532c701585cd392d
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
Mar 02 10:48:41 raspberrypi librespot[714]:    1:   0x8febfc - std::backtrace_rs::backtrace::trace_unsynchronized::h15e96deae50ea7f1
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
Mar 02 10:48:41 raspberrypi librespot[714]:    2:   0x8febfc - std::sys_common::backtrace::_print_fmt::h2de65accbd58b3e4
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:67:5
Mar 02 10:48:41 raspberrypi librespot[714]:    3:   0x8febfc - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h85671ae76efc2464
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:46:22
Mar 02 10:48:41 raspberrypi librespot[714]:    5:   0x8f709c - std::io::Write::write_fmt::hcccb69ae7e0e5712
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/io/mod.rs:1697:15
Mar 02 10:48:41 raspberrypi librespot[714]:    6:   0x900c68 - std::sys_common::backtrace::_print::hedd1975fac8ef971
panicking.rs:211:50pberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:49:5
Mar 02 10:48:41 raspberrypi librespot[714]:    9:   0x900710 - std::panicking::default_hook::h76fa136e1f70a214
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/panicking.rs:228:9
Mar 02 10:48:41 raspberrypi librespot[714]:   14:   0x61fd54 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h6cdb6d970c4ac361
Mar 02 10:48:41 raspberrypi librespot[714]:   15:   0x60d7d8 - <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll::hfe2e84411f277e84
Mar 02 10:48:41 raspberrypi librespot[714]:   16:   0x5fbd10 - <librespot_playback::player::PlayerInternal as core::future::future::Future>::poll::h28a4cde177e46443
Mar 02 10:48:41 raspberrypi librespot[714]:   17:   0x4c7774 - std::thread::local::LocalKey<T>::with::h85f1e7d59454df33
Mar 02 10:48:41 raspberrypi librespot[714]:   18:   0x4bbad4 - futures_executor::local_pool::block_on::hefeadbfdb9a235f9
Mar 02 10:48:41 raspberrypi librespot[714]:   19:   0x567ff4 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha9233c878f46eaea
Mar 02 10:48:41 raspberrypi librespot[714]:   20:   0x5287ac - core::ops::function::FnOnce::call_once{{vtable.shim}}::he863134c8af815b9
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694:9
Mar 02 10:48:41 raspberrypi librespot[714]:   22:   0x9061e4 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h3a9e9491476f36e5
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694:9
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:42 raspberrypi librespot[714]: [2022-03-02T15:48:42Z ERROR librespot_playback::player] Player Commands Error: channel closed
xadrta commented 2 years ago

+1 here from Norway.

willtonkin commented 2 years ago

+1 from here in Denmark 🇩🇰

bjacobse commented 2 years ago

+1 from here in Denmark Thank you for your advice

Boxer66 commented 2 years ago

+1 from Norway. Fixed hosts in Arch and Win11 and working fine for now.(ncspot)

orjanv commented 2 years ago

+1 from Norway, glad I found this thread, and the hosts entry did the job.

fuerstr commented 2 years ago

+1 from Switzerland With "104.199.65.124 ap-gew4.spotify.com" in the /etc/host file it works, thanks! I am not sure what is causing the problem and how to properly fix it.

Baktus79 commented 2 years ago

+1 from Norway Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts did solve the problem. Have had this issue for a while and suddenly came across this thread.

Anyone got any answers from Spotify? This is lazy work at their end.

agramner commented 2 years ago

+1 from Sweden Used the "fallback ap hack". Set --ap-port=[some random port]. Log then said Using fallback "ap.spotify.com:443" and it works!

Bettehem commented 2 years ago

+1 from Finland Adding a line containing 104.199.65.124 ap-gew4.spotify.com to /etc/hosts solved the issue.

michaelherger commented 2 years ago

I've now had reports from Austria and Switzerland, too... are they shutting down old infrastructure?

user7743 commented 2 years ago

I know its not quite AT & CH but I can confirm in Germany I have had this issue for quite some time.

thomas-ah commented 2 years ago

+1 from The Netherlands (using ncspot under macos).

Fixed by adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts :)

westham commented 2 years ago

+1 from Sweden Adding a line containing 104.199.65.124 ap-gew4.spotify.com to /etc/hosts solved the issue.

emilBeBri commented 2 years ago

Nice that also worked for me, in denmark, thx

philgzl commented 2 years ago

+1 from Denmark 🙏🙏🙏

roderickvd commented 2 years ago

Anyone got any answers from Spotify? This is lazy work at their end.

We have no lines of communication to Spotify.

I've now had reports from Austria and Switzerland, too... are they shutting down old infrastructure?

It would surprise me because many legacy devices would suffer. I don’t think Spotify announced the EOL for such hardware.

On the other hand I don’t understand as this is just their own resolver, which last I checked was also used by their own clients. One difference is that I saw so when I was working on new-api with HTTP. Who knows the Mercury is phased out in that server cluster, and legacy clients fall back?

Seeing how this issue is only becoming worse we could consider blacklisting this host name.

VB1987 commented 2 years ago

+1 from The Netherlands.

Fixed by adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts

jarmoniuk commented 2 years ago

+1 from 🇳🇱, the above fix helped me too

CaptainAweesome commented 2 years ago

I am on Balena OS using balena sound to play tracks from Spotify.

I added 104.199.65.124 ap-gew4.spotify.com to /etc/hosts. But it is non persistant, since it is running in a container.

I tried adding it to pi.hole local DNS records. I can ping ap-gew4.spotify.com from Spotify container and Host OS with reply. With no luck tracking down the real issue.

Log-snippet:

<spotify> [2022-07-17T12:58:45Z INFO  librespot_playback::player] Loading <Guilty of Love> with Spotify URI <spotify:track:6SskWYTJwI77yhvNJscIiu>
<spotify> [2022-07-17T12:58:45Z ERROR librespot_core::channel] channel error: 2 1
<spotify> [2022-07-17T12:58:45Z ERROR librespot_playback::player] Unable to load encrypted file.
<spotify> [2022-07-17T12:58:45Z WARN  librespot_playback::player] Unable to load <SpotifyId { id: 300406345378731222655173408678663613098, audio_type: Track }>
agramner commented 2 years ago

@CaptainAweesome Maybe try extra_hosts in docker compose? See https://docs.docker.com/compose/compose-file/compose-file-v3/#extra_hosts

Or if possible modify https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh and add command line arg --ap-port=[some random port]

kirenida commented 2 years ago

I tried adding it to pi.hole local DNS records. I can ping ap-gew4.spotify.com from Spotify container and Host OS with reply. With no luck tracking down the real issue.

Try blocking apresolve.spotify.com in pihole, that worked for me.

CaptainAweesome commented 2 years ago

Try blocking apresolve.spotify.com in pihole, that worked for me.

This actually worked. I am astounded. Thank you very much!

@agramner I did not find any recommended ways to add parameters to containers in Balena. Since Balena-sound is a custom build of Balena OS with many containers, without any excessive customizability. None the less, I thank you for your reply!

CaptainAweesome commented 2 years ago

This actually worked. I am astounded. Thank you very much!

I'm now back to the same errors as before. Unable to load tracks from Spotify. apresolve.spotify.com is blocked in pi.hole and nslookup apresolve.spotify.com gives the following address: 0.0.0.0

Even though I can access http://apresolve.spotify.com/ from my browser all of a sudden.

juhi24 commented 2 years ago

Had this problem today in Finland. Blocking apresolve.spotify.com on pihole was a successful workaround.

jarmoniuk commented 2 years ago

@CaptainAweesome how do you precisely run librespot in a container? Do you use docker or docker-compose or something else?

Btw, it doesn't look like it's a librespot bug.

If you're using plain docker, try adding --add-host apresolve.spotify.com:127.0.0.1 to the docker command.

the above will block apresolve.spotify.com.

CaptainAweesome commented 2 years ago

@ajarmoniuk

I am on Balena OS using balena sound to play tracks from Spotify.

It's a way of deploying docker containers to devices and managing said devices remotely. Here is the balena-sound startup for Spotify: https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh

I have now instead of adding apresolve.spotify.com to Blacklist, added apresolve.spotify.com 127.0.0.1 to Local DNS Records inside pi.hole. And strangely it works now, again. Thank you!

jelmerderonde commented 2 years ago

+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it. Also blocked apresolve.spotify.com in pihole to be on the safe side ;-).

sucama2022 commented 2 years ago

Hello, I am having the same problem with Moodeaudio version 8.1.2. with raspberry pi4. I can't listen to Spotify. I am in Argentina, can someone help me with the dns of this area, to add: xxx.xxx.xxx.xxx ap-gew4.spotify.com to /etc/hosts? Thank you!

sucama2022 commented 2 years ago

sucama@sucama-B250M-D3H:~$ ping ap-gew4.spotify.com PING ap-gew4.spotify.com (34.158.0.131) 56(84) bytes of data. 64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=1 ttl=101 time=246 ms 64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=2 ttl=101 time=245 ms 64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=3 ttl=101 time=246 ms 64 bytes from 131.0.158.34.bc.googleusercontent.com (34.158.0.131): icmp_seq=4 ttl=101 time=246 ms

dsolano04 commented 2 years ago

Hi Sucama, I've just solved the issue by changing /etc/hosts as follows:

104.199.65.124 ap-gue1.spotify.com

Basically, ap-gue1.spotify.com is the host volumio/spotify try to resolve from Argentina but it doesn't work, so you need to point it to ap-gew4.spotify.com by adding its IP address (104.199.65.124) in the /etc/hosts file.

Make sure after the change to restart the network interfaces and restart volumio. You can also restart the whole server to make it easier.

Hope it helps.

programamapache commented 2 years ago

Hi Sucama, I'm from Argentina too and I had been having this problem since a week ago. Yesterday I edited hosts file with: 0.0.0.0 apresolve.spotify.com and now works fine. Tested spotify all day without problems. I'm using HiFiBerry OS with Raspberry 3+. Just for information, along all the trials I performed with no luck (updating Spotify in mac and iOS devices, clean install of HiFiBerry OS on the raspy, downgrading HiFiBerry OS to prior version) I installed spotify app in my LG TV and it worked fine. I could connect from spotify desktop and control the TV but not the raspberry. So I'm not sure if this is a DNS problem alone.

Hope it helps.

EdoardoLaGreca commented 2 years ago

I had the same issue, I'm from Italy 🤌. I'm using a Raspberry Pi 3B with Raspberry Pi OS and Raspotify.

I added the following line to /etc/hosts and now everything works fine:

0.0.0.0     apresolve.spotify.com
dotto95 commented 2 years ago

I had the same issue, I'm from Italy 🤌. I'm using a Raspberry Pi 3B with Raspberry Pi OS and Raspotify.

I added the following line to /etc/hosts and now everything works fine:

0.0.0.0       apresolve.spotify.com

Thank you @EdoardoLaGreca :) Same country same fix ;)

kingcharles1666 commented 2 years ago

+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me

d-ilievski commented 2 years ago

+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me

Same for me :) I was struggling for 3 days tinkering with my devices to find this thread and learn it's from Spotify's side. Thanks, guys, you are the best!

kalj commented 2 years ago

Awesome workarounds! However, I am a bit curious what the difference is between defining

0.0.0.0     apresolve.spotify.com

and

104.199.65.124 ap-gew4.spotify.com

Both seem to fix the problem for me. But is either preferable for some reason?

sucama2022 commented 2 years ago

Hi , Thanks To programamapache for this: 0.0.0.0 apresolve.spotify.com Work for me in Argentina.. Moodeaudio versio 8.1.2 , Raspberry pi4 Thanks Very much!

kirenida commented 2 years ago

Awesome workarounds! However, I am a bit curious what the difference is between defining

Have you tried opening apresolve.spotify.com in your browser?

If you do that, you will get something like:

{"ap_list":["ap-gew1.spotify.com:4070","ap-gew1.spotify.com:443","ap-gew1.spotify.com:80","ap-gew1.spotify.com:4070","ap-guc3.spotify.com:443","ap-gae2.spotify.com:80"]}

which is a list of Spotify content delivery servers most suitable for your location (as determined from Spotify's side)

As stated previously in this thread, it looks like Spotify has started rolling out some new method of serving content through some of these servers. That new method works with the official Spotify client, but not with librespot. (Which I don't consider "lazy" as stated in a previous comment - why should they care about unofficial clients?)

With apresolve.spotify.com blocked, librespot defaults to some server that doesn't have that new method enabled (yet).

If you just define a single server, there is a chance that the response from apresolve could change at some point in time, and then you would need to enter a new combination of IP address + URL.

As I'm not experienced with the inner workings of librespot or Spotify, I'm just going with my basic knowledge and logic, so what I've just written may be wrong in some way. If that is so, somebody please correct me :)

coaxial commented 2 years ago

@kirenida:

why should they care about unofficial clients?

Because they deprecated the official libspotify so there isn't an "official client" anymore.

kingosticks commented 2 years ago

Sorry to be pedantic but libspotify was deprecated in May 2015, they've not given a hoot for 7 years.

Edit: And it is lazy since they promised a replacement back and then and didn't deliver. Useless? Lazy? Pick one!

Anyway, grumbling aside, what's interesting is there is another potential breakage coming in September 2022. This might be a deadline for getting new api branch merged (sorry, I know deadlines are not helpful @roderickvd and it's on everyone to step up and help). https://developer.spotify.com/community/news/2022/07/15/mobile-streaming-sdks-update/

roderickvd commented 2 years ago

I was hoping to make some time to do a quick update release to blacklist this “broken” (to us) access point, but chances of me completely upgrading to the HTTP-based API before September are next to zero:

  1. the current state of new-api is functional but needs bugs ironed out and dev merged in. I don’t realistically have time for all of that before September. I have called multiple times for people to take on that chore but no one has answered the call.

  2. new-api now retrieves files over audio files over HTTP but not encryption keys or commands.

Now I don’t know what Spotify will be deprecating and what not, but:

Again I don’t know what that Spotify SDK deprecation may bring but if this project wants to stay up to date then we really need more contributors.

kingosticks commented 2 years ago

Good summary. I have a few Python bits to finish in my downstream project this weekend, after which I'm going to focus on getting gstspotifysrc where I need it to be. I'll switch it to the new-api (and worry about the release implications of that later) and then be some help to you here. I don't need all of librespot's features for my stuff but I'll do what I can.

However, I don't profess to having advanced Rust skills.

TheN00r commented 2 years ago

+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me

Same for me :) I was struggling for 3 days tinkering with my devices to find this thread and learn it's from Spotify's side. Thanks, guys, you are the best!

Can confirm from the Netherlands that adding 104.199.65.124 ap-gew4.spotify.com to /storage/.config/hosts.conf did the trick as well!

Thanks!

michaelherger commented 2 years ago

Just a little heads up. As outlined before there's no need to fiddle with DNS, hosts files, IP addresses etc. You can work around this issue using a librespot parameter:

Used the "fallback ap hack". Set --ap-port=[some random port]. Log then said Using fallback "ap.spotify.com:443" and it works!

SuisChan commented 2 years ago
  • the encryption keys over HTTP have not been reversed engineered yet (with decompilation attempts from some having been taken down).

As a fallback - we can use widevine, but it will probably need a lot of re/writing code because it differs a lot from the current implementation.

kingosticks commented 2 years ago

What do you mean by that @SuisChan? Widevine is a (commercial?) closed source library, isnt it? Reverse engineering that software would be like poking a (small, angry) bear.

SuisChan commented 2 years ago

What do you mean by that @SuisChan? Widevine is a (commercial?) closed source library, isnt it? Reverse engineering that software would be like poking a (small, angry) bear.

Everything is not quite right, since we can get a private key, we can use it accordingly, and the algorithms have been public for a long time (leaks and not only), besides, I have experience working with it, so I can confirm that it works.

khink commented 2 years ago

+1 from NL. Adding 104.199.65.124 ap-gew4.spotify.com to /etc/hosts fixed it for me

roderickvd commented 2 years ago

Please take #1026 for a spin. If this works (it does for me -- I also get ap-gew4 in the Netherlands usually) I can do a point release the coming days.

Boobybandit commented 2 years ago

Please take #1026 for a spin. If this works (it does for me -- I also get ap-gew4 in the Netherlands usually) I can do a point release the coming days.

For me adding ap-gew4 (manually) did not work for 2 out of 3 devices. Adding "0.0.0.0 apresolve.spotify.com" however did work on all 3 devices. Also Netherlands here, so that's a bit weird.

If I understand correctly then adding ap-gew4 is rerouting the stream while adding 0.0.0.0 is blocking some resolve api? Anyways, for now everything works here (faster than usual btw).