Preload next track when approaching the end of current one & gapless playback support

[sashahilton00]

Issue by plietar Wednesday Dec 30, 2015 at 12:01 GMT Originally opened as https://github.com/plietar/librespot/issues/18

---

Currently we don't start loading the next track until the current one has completed. This causes a short delay between tracks.

Instead the next track should start loading before that.

[sashahilton00]

Comment by fbrinker Saturday Jan 02, 2016 at 10:11 GMT

---

Maybe with (optional) crossfading, if possible

[sashahilton00]

Comment by TonioRoffo Friday Nov 11, 2016 at 19:00 GMT

---

Crossfading and gapless playback are two different things. In HW spotify connect gear, gapless is standard.

[sashahilton00]

Comment by RafaPolit Friday Jan 20, 2017 at 21:54 GMT

---

Gapless is really a must for me, since most of what I listen to is either live or classical music, where much of the 'transitions' are mid-sound (specially in Opera, for instance).

Any time frame on this new feature? Thanks for all the hard work, best regards, Rafa.

[sashahilton00]

Comment by plietar Friday Jan 20, 2017 at 22:14 GMT

---

Gapless isn't too hard, and I'm working on a refactor which will also enable that. I can't give it an ETA, but probably somewhere within the next month.

Cross-fade is a lot trickier, and I don't have any immediate plans for it

[sashahilton00]

Comment by cortegedusage Tuesday Feb 14, 2017 at 08:12 GMT

---

+1 for gapless for me. (Just for the record, no pressure, keep up the good work.)

[sashahilton00]

Comment by johnstok Wednesday May 31, 2017 at 18:35 GMT

---

+1 – would be great if this was implemented.

[sashahilton00]

Comment by RafaPolit Tuesday Jun 27, 2017 at 17:00 GMT

---

Any news on this? I'm eagerly awaiting for this and 6 months ago you said it wasn't hard. Was it more difficult than expected or this is simply not a priority for others? For me its critical. Please? Thanks.

[sashahilton00]

Comment by sashahilton00 Tuesday Jun 27, 2017 at 18:18 GMT

---

Paul mentioned he was busy until July, most things have been put on hold till then. I'm sure it will be added in due course. Cross fade is harder due to the need to mix audio streams

[sashahilton00]

Comment by RafaPolit Tuesday Jun 27, 2017 at 19:14 GMT

---

Yeah, I wasn't talking about crossfading, just gapless playback for concerts, and classical pieces that flow seamlessly into the next movement (most critical in Opera).

[sashahilton00]

Comment by pauLee Friday Nov 17, 2017 at 14:37 GMT

---

Any progress on gapless playback? I still hear breaks between the tracks.

[sashahilton00]

With regards to this, one thing I have noticed, is that the loading of tracks is synchronous, hence if one is listening to a playlist and skips a few songs, it loads each of the skipped tracks, leading to a few seconds of silence. Ideally, when firing load_track, we'd dump any existing track load requests.

[RafaPolit]

Please give some love to this issue. Most other issues can be accomplished by other means or extra steps. This is the one that is un-accomplishable, no matter how you see it.

It's the only thing still keeping me with other approaches.

Best regards, Rafa.

[herrernst]

Would also love to see somebody working on gapless playback (I don't care about crossfading). This is the last issue that make playback imperfect :/

[st0nec0ld]

Anything new? The gap between two songs is realy annoying.

[WhiteHatTux]

I started looking into it. As the main delay for me is with the network, the first step I see is to implement a PlayerCommand:PreLoad which will just make sure, that the file is available, when it will be requested for real. This part I have actually implemented.

Something, that I have not yet figured out is, how the PreLoad Command could be triggered. Something like: "end of track is 15 seconds away. Start Preload now".

[WhiteHatTux]

@st0nec0ld Do you get the delay between every song, or only between songs, that are not yet cached?

[berrywhite96]

@WhiteHatTux I think it make sense to trigger the loading of the next song on every new track. So if a new track starts, load the next track. Also the RAM on the pi is big enough, see raspberry pi 3 b+, to handle this easily. I personally skip often through tracks. In detail: hear the song for 3 seconds and then skip. These 3 seconds are enough (in my network) to load a track, so in this scenario there wouldnt be a gap.

This gap is really annoying, loading tracks needs maybe 1-2 seconds (in my network with 320kbps), but my av receiver dont get a signal for a second and needs then some time to recognize the codec of the audio stream, so the gap is longer than the track needs to load.

[github-ronk]

+1 for this...

Thanks :)

[giggywithit]

Vorbis is designed from the ground up to be used for streaming no? MPD can do gapless playback and apparently buffers. I know that it is also possible to playback a vorbis file as it is being created (downloaded)

Forget hacks of how to anticipate what the user is going to do.

[kingosticks]

I think it's safe to implement gapless the same way everyone else in the world does it. i.e. buffer the output slightly. If you skip through tracks you'll have a small gap but that's fine - gapless here makes no sense at all.

Issues with external audio devices caused by 'stopping' the audio output are a separate issue.

[giggywithit]

If I use the 'play ' command in a directory filled with files from a live album, the files are played back gaplessly. If I use the 'play ' command with one file in the directory and add one after I have executed the command, it will stop playback after playing the initial file, not seeing the next. It would seem apparent that one could implement an addition to the 'play' command to optionally seek another file during playback and adding it to its process. 'play' also has the capacity to playback files over http etc.. If this were the case, 'play' could be used to simply play files that are being loaded into a cache directory.

play is an audio player as part of the SOX project

[giggywithit]

Hello Nick Steel

It seems that you have been around these places for some time and would be the person to ask the question...

What I am looking for, since there are no reliable spotify players for the raspberry pi and seems that it will be quite some time until one that works effectively, is the best method to stream audio to the raspberry pi from another computer on the network. I have an ubuntu laptop that I connect directly to the raspberry pi via the cat5 wired interface and share the internet connection that I get on the laptop with it. I like the idea of streaming pcm audio rather than compressing it. I like using raspian lite headless.. Minimal but full effectiveness preferred.

Can you offer a better solution? Absolutely appreciated

[kingosticks]

I think you'd be better off asking your question on the official raspberry pi forums. Hijacking this issue here with your unrelated question isn't helpful. Please don't do that.

[RafaPolit]

Another month gone by, is this going anywhere? Or are the people in need of gapless just doomed to look elsewhere?

[jebos]

was there any work on this? any updates?

[awiouy]

@jebos No

[CooperWhitlow]

+1 would be super awesome for crossfade

[laurensnl]

+1 for gapless playback. Would be great!

[sashahilton00]

no work on this issue yet. I think most maintainers are busy with work or studies atm. PR's always welcome, otherwise I suspect this will remain in progress for a while longer. also, please use reactions to indicate support. any more +1 comments will be deleted.

[jebos]

Though, in theory - what needs to be done (for gapless)?

1) keep the sink open, right now it looks like the sink is closed and recreated for each track. 2) have buffering active and start loading of next track into buffer as soon as current track is fully buffered. 3) play from buffer

Any suggestion on how to do this? Keeping the sink open (at least for some time) seems achiveable, I'm not sure about the buffering part, though (without large changes, at least)

[ashthespy]

FWIW: I managed to implement a proof of concept of the preloading - but didn't find time to implement a buffer yet from the sink side. What it does it implement a new channel that communicates "almost end of track" to spric which triggers the prefetching of the next track provided by handle_next from spirc. I can clean it up and push it if someone is interested in it.

Though, in theory - what needs to be done (for gapless)?

1. keep the sink open, right now it looks like the sink is closed and recreated for each track.

2. have buffering active and start loading of next track into buffer as soon as current track is fully buffered.

3. play from buffer

Any suggestion on how to do this? Keeping the sink open (at least for some time) seems achiveable, I'm not sure about the buffering part, though (without large changes, at least)

Keeping the sink open as it is now will lead to underruns in alsa as there is no data to be played - we would need to implement a small buffer inside player that is responsible to feed the sink.

[sashahilton00]

@ashthespy by all means push it to a branch. i was just getting started on working out how to signal from player back to spirc, so if you have a PoC, that would save me a headache :)

[ashthespy]

@ashthespy by all means push it to a branch. i was just getting started on working out how to signal from player back to spirc, so if you have a PoC, that would save me a headache :)

Here you go https://github.com/librespot-org/librespot/compare/master...ashthespy:gapless It's not an optimised implementation in anyway but should work with the lewton, and prefetch a track when there are 2 seconds left in the current track.

[jebos]

Hi, I think there should be a check that the current track duration is longer than 2 seconds (paranoid me) and maybe the amount of time should depend on the selected quality (assuming that higher qualtiy results in slower prefetch time)

What do you think about doing a procentual amount like 5% of duration and bounding that on 0 seconds (min) and 5 seconds. (max)

[plietar]

IMO the best thing to do is start loading the next track as soon as the current one is fully cached. This may be a little complicated to implement for now, so a 2 second window sounds fine.

[ashthespy]

Hi, I think there should be a check that the current track duration is longer than 2 seconds (paranoid me) and maybe the amount of time should depend on the selected quality (assuming that higher qualtiy results in slower prefetch time)

What do you think about doing a procentual amount like 5% of duration and bounding that on 0 seconds (min) and 5 seconds. (max)

I had initially set it to trigger when there was 20% remaining, but then switched to a hard-coded 2 seconds to test - but it might be a good idea to make this bitrate/file size dependent.

IMO the best thing to do is start loading the next track as soon as the current one is fully cached. This may be a little complicated to implement for now, so a 2 second window sounds fine.

That would need another channel with the fetcher right? Also - any ideas about the buffer implementation? I I put something together, but now trying to implement tracking the playback state independent of the buffer - need to look into if there is some way to queue the Vorbis decoder?

[sashahilton00]

If we want to keep behaviour consistent with the Spotify clients, I believe that they have hardcoded 30 seconds before the end of the track as the preload trigger for all bitrates/files.

With regards to keeping to sink open, how about implementing a feature flag such as --disable-sink-keepalive, and then by default keep the sync open between songs unless that flag was present? If we preload things 30 seconds before the end of the track à la Spotify, there shouldn't be any delay in filling the buffer (causing an underrun), or if there is it should be <1s, at which point do we care? If it is an issue, one could just disable it for the current behaviour.

IMO the best thing to do is start loading the next track as soon as the current one is fully cached. This may be a little complicated to implement for now, so a 2 second window sounds fine.

This sounds potentially tricky from a logistics point of view. Let's say I play track 1. Track 2 is cached after a few seconds, then what? Does it cache Track 3, then 4, etc. or does it wait until Track 2 starts, then start caching Track 3? In the former case, if Spotify loads a large playlist or album, the cache is going to fill up quickly, which would potentially have to be managed through some sort of cache expiry mechanism mentioned in a previous issue (will add issue number later if I can find it), and in the latter case, why bother with the extra complexity of another channel and a buffer to preload a track at the beginning of a song? We could do that with minor modifications to @ashthespy's branch. Also, as an afterthought, for those on limited data plans (e.g. mobile broadband), librespot could use up a lot of bandwidth pointlessly loading tracks that end up not being played in the former case.

Also @ashthespy could you create a PR so that we can move discussion of implementation details over to that, review, commit, etc. Thanks.

[kingosticks]

This sounds potentially tricky from a logistics point of view. Let's say I play track 1. Track 2 is cached after a few seconds, then what? Does it cache Track 3, then 4, etc

No. That's a different strategy and it's unnecessary.

Signalling at x seconds before end of track to start fetching the next song is fine. x can be really small, you only need to fetch enough to start playing, you do not need the entire song.

[ashthespy]

If we want to keep behaviour consistent with the Spotify clients, I believe that they have hardcoded 30 seconds before the end of the track as the preload trigger for all bitrates/files.

With regards to keeping to sink open, how about implementing a feature flag such as --disable-sink-keepalive, and then by default keep the sync open between songs unless that flag was present? If we preload things 30 seconds before the end of the track à la Spotify, there shouldn't be any delay in filling the buffer (causing an underrun), or if there is it should be <1s, at which point do we care? If it is an issue, one could just disable it for the current behaviour.

I don't know about that - I tried something along these lines already (an option to keep the sink open b/w songs at https://github.com/librespot-org/librespot/compare/master...ashthespy:greedysink) but on my fruity pi systems, the time taken to fetch the next song, decrypt, and decode the audio (even from cache) leads to noticeable under runs from alsa.

Also @ashthespy could you create a PR so that we can move discussion of implementation details over to that, review, commit, etc. Thanks.

263 Sure, I might be pressed for time the next few weeks - so it might be slow progress!

[loeffelpan]

Any progress here? Is there a branch who could be built by someone for debian? Just for testing.

[devgianlu]

@ashthespy is working on it here in Rust. I've implemented it in librespot-java.

[loeffelpan]

@ashthespy Is your gapless branch ready to use? I would give it a try to use is in my raspotify installation.

[ashthespy]

@loeffelpan No, haven't had much time off late, prefetch is implemented, gapless needs some restructuring of the player (unless I am missing something).

My tentative approach is to have two audio buffers - on for the currently playing track, and another buffer where a few packets of the next track are decoded and kept ready for playback. That way the current playback state can be tracked independent of the packet decoder. If anyone has suggestions for alternatives - do share!

[loeffelpan]

I don't have any suggestions. Just want to be up to date.

Sounds good. Do you have any plan to finish that project? No pressure, but i'm very exited to try the result with my audiobooks hearing gapless! <3

[nkappler]

Hi, any news on this already? Would love it as well :)

Let me know if you need help with this, I might have a look at it aswell...

[loeffelpan]

I switched to librespot-java. Works flawlessly now after some issues.

[Identity9165]

librespot-java

I spent an hour with maven and it's not compiling. binaries don't help either. :(

[devgianlu]

@msc1 You need to install Java, then download a precompiled binary and run java -jar ./librespot-core-jar-with-dependencies.jar. Check the read me, you can contact me on Gitter if you still have problems.

[Identity9165]

yes I have them installed already, I couldn't found a solution for the following error:

env: DietPi v6.22.3 (Debian 9) RPi 3 Model B (armv7l)

> root@spotifypi:~# java -version
> openjdk version "1.8.0_212"
> OpenJDK Runtime Environment (build 1.8.0_212-8u212-b01-1~deb9u1-b01)
> OpenJDK Client VM (build 25.212-b01, mixed mode)
> root@spotifypi:~# javac -version
> javac 1.8.0_212
> root@spotifypi:~# mvn -v
> Apache Maven 3.3.9
> Maven home: /usr/share/maven
> Java version: 1.8.0_212, vendor: Oracle Corporation
> Java home: /usr/lib/jvm/java-8-openjdk-armhf/jre
> Default locale: en_GB, platform encoding: UTF-8
> OS name: "linux", version: "4.14.98-v7+", arch: "arm", family: "unix"

building with maven fails around api client

> /root/librespot-java-0.5.2/api-client/src/main/java/xyz.gianlu.librespot.api.client/Main.java:[3,26] cannot access javafx.application.Application
>   bad class file: /root/.m2/repository/org/openjfx/javafx-graphics/12-ea+8/javafx-graphics-12-ea+8-linux.jar(javafx/application/Application.class)
>     class file has wrong version 55.0, should be 52.0
>     Please remove or make sure it appears in the correct subdirectory of the classpath.
> 

any help would be appreciated, thanks.

[devgianlu]

Check https://github.com/librespot-org/librespot-java/issues/86#issuecomment-489096807