search

Yetangitu / Spodcast

Spodcast is a caching Spotify podcast to RSS proxy. Using Spodcast you can follow Spotify-hosted netcasts/podcasts using any player which supports RSS, thus enabling the use of older hardware which is not compatible with the Spotify (web) app.
GNU General Public License v3.0
359 stars 14 forks source link

Podcast episodes are not completely downloading #14

Closed lohningerthomas closed 2 years ago

lohningerthomas commented 2 years ago

I noticed that for some podcasts the end of the episodes are missing.

For example I downloaded this show: https://open.spotify.com/show/5JYitG4bOM3sVmAQRdX1Na and for every episode the last minute or so is missing in the downloaded file.

After running the debug output I noticed that Librespot stopped downloading before all chunks were completed:

DEBUG:Librespot:Session:Chunk 156/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

DEBUG:urllib3.connectionpool:https://audio4-ak-spotify-com.akamaized.net:443 "GET /audio/7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa?__token__=exp=1657893805~hmac=9f418cd6ab2eaa013c4147924038e9eda979bb052984e084697417f1b16659e9 HTTP/1.1" 206 131072

DEBUG:Librespot:Session:Chunk 157/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

DEBUG:urllib3.connectionpool:https://audio4-ak-spotify-com.akamaized.net:443 "GET /audio/7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa?__token__=exp=1657893805~hmac=9f418cd6ab2eaa013c4147924038e9eda979bb052984e084697417f1b16659e9 HTTP/1.1" 206 131072

DEBUG:Librespot:Session:Chunk 158/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

INFO:spodcast.podcast:transcoding ogg->mp3

This has to be some issues with certain shows, as either all of the episodes of a show are missing the last minute or they are all fine.

An example for a working podcast would be: https://open.spotify.com/show/7BTOsF2boKmlYr76BelijW

I have no clue what the issue could be and am open for any suggestion!

diveflo commented 2 years ago

same thing happening for e.g. this podcast: https://open.spotify.com/show/6UUIXmp1V0fK4ZpK7vzAbQ

hugolinhares commented 2 years ago

It is happening to this podcast too: https://open.spotify.com/show/0znjxREK2kgs1iL0i7W46V

Yetangitu commented 2 years ago

I'll have a look at these examples to see what is going on, the problem can found somewhere in librespot-python or within spodcast but since spodcast seems to be one of the few tools using the netcast/podcast endpoints it is not surprising for these problems to show up here.

lohningerthomas commented 2 years ago

A quick fix that seems to work for me is to set the chunk size to 1 - the download gets quite slow but it seems to download all chunks and therefore the whole episode

Yetangitu commented 2 years ago

The file size reported by librespot-python - and I assume Spotify - seems to be unreliable for some shows. I removed the dependency on this reported size, this works without loosing the chunked download functionality.

Yetangitu commented 2 years ago

Give the new release (0.5.2) a try to see if it works for you, it Works for Me™ so I assume it will.

lohningerthomas commented 2 years ago

Can confirm, works for me as well - thy for the fix!

hugolinhares commented 2 years ago

Thank you for the fix!! Can you can update the docker image too? Thy again.

Yetangitu commented 2 years ago

Thank you for the fix!! Can you can update the docker image too? Thy again.

@heywoodlh Your turn...

heywoodlh commented 2 years ago

Done, image tag has been updated: 0.5.2

Yetangitu commented 2 years ago

Thanks

arwk commented 2 years ago

I noticed something weird:

When fetching an entire Podcast (https://open.spotify.com/show/3JgH71EJblvCEXAca1XRsT), most of the episodes are completely downloaded. But a few of them are missing some chunks. However if I proceed downloading the specific episode (https://open.spotify.com/episode/4988PIBdSWNVgYqe3jGzRL), everything gets downloaded. Unfortunately I don't know if the issue is reproducible and if always the same episodes are not complete, or if this is some stochastic behaviour.

I used this script to process the log (--log-level debug) generated by spodcast. It outputs all episodes with missing chunks.

with open("nur-verheiratet.log") as f:
    last_fileid = None
    last_line = None
    for line in f.readlines():
        if line.startswith("DEBUG:Librespot:Session:Chunk"):
            file_id = line.split()[-1]
            if file_id != last_fileid:
                if last_line != None:
                    a, b = last_line.split()[1].split("/")
                    if a != b:
                        print(last_line)
                last_fileid = file_id
            last_line = line

On my machine they are

DEBUG:Librespot:Session:Chunk 28/451 completed, cached: False, stream: file_id: 0c9562623a92522548116128fc4bc313dec40768
DEBUG:Librespot:Session:Chunk 194/402 completed, cached: False, stream: file_id: e73bf556faf7bf4a7669d5de9cca36920c15a58c
DEBUG:Librespot:Session:Chunk 105/437 completed, cached: False, stream: file_id: 44745459c28e3070ae2f837c507ca25705c45299
DEBUG:Librespot:Session:Chunk 279/332 completed, cached: False, stream: file_id: b50c057a1ccda851ce21c6ed0db6061884030fb5
$ spodcast --version
spodcast 0.5.2