madiele / vod2pod-rss

Vod2Pod-RSS converts a YouTube or Twitch channel into a podcast with ease. It creates a podcast RSS that can be listened to directly inside any podcast client. VODs are transcoded to MP3 on the fly and no server storage is needed!
MIT License
218 stars 6 forks source link

Issues on apple devices (apple podcast, IOS's pocket casts) #67

Closed madiele closed 1 year ago

madiele commented 1 year ago

from a comment on reddit it seems apple podcast has some incompatibility issues

2023-05-07T07:31:12.144Z INFO  [vod2pod_rss::transcoder] detected youtube url, need to find the stream url if not in cache
2023-05-07T07:31:12.146Z DEBUG [vod2pod_rss::transcoder] getting stream_url for yt video: https://www.youtube.com/watch?v=...
2023-05-07T07:31:14.157Z DEBUG [vod2pod_rss::transcoder] generating ffmpeg command
2023-05-07T07:31:14.157Z INFO  [vod2pod_rss::transcoder] generated ffmpeg command:
ffmpeg -ss 0 -i https://mk7—sour.googlevideo.com/videoplay…. -acodec libmp3lame -ab 192k -f mp3 -bufsize 5760 -maxrate 5760k -timeout 300 -hide_banner -loglevel error pipe:stdout
2023-05-07T07:31:14.158Z INFO  [vod2pod_rss::transcoder] streaming to client
2023-05-07T07:31:14.443Z DEBUG [vod2pod_rss::transcoder] channel closed
2023-05-07T07:31:14.443Z INFO  [vod2pod_rss::transcoder] connection to client dropped, stopping transcode
2023-05-07T07:31:14.448Z DEBUG [vod2pod_rss::transcoder] ffmpeg stderr closed

seems like it is not liking what is receiving and killing the connection right away, unfortunately I don't have an apple device to replicate the issue, while I wait for the occasion if anyone finds more info please share it here, pull request with a fix are also welcome if there is a bug

pmd2016 commented 1 year ago

I can confirm it doesn't work on Apple Podcasts.

When running in the desktop, this is the output in the logs:

2023-05-08T10:15:00.203Z INFO  [vod2pod_rss::url_convert] trying to convert youtube channel url https://www.youtube.com/@...
2023-05-08T10:17:53.471Z INFO  [app] processing transcode at 192k for https://www.youtube.com/watch?v=...
2023-05-08T10:17:53.482Z INFO  [vod2pod_rss::transcoder] detected youtube url, need to find the stream url if not in cache
2023-05-08T10:18:13.805Z INFO  [vod2pod_rss::transcoder] generated ffmpeg command:
ffmpeg -ss 0 -i https://.............googlevideo.com/videoplayback?.....  -acodec libmp3lame -ab 192k -f mp3 -bufsize 5760 -maxrate 5760k -timeout 300 -hide_banner -loglevel error pipe:stdout
2023-05-08T10:18:13.811Z INFO  [vod2pod_rss::transcoder] streaming to client

When I add the podcast to Apple Podcasts, and try to download an episode, nothing happens and the logs only show this:

2023-05-08T10:26:10.509Z INFO [vod2pod_rss::url_convert] trying to convert youtube channel url https://www.youtube.com/@...

However, this isn't limited to Apple Podcasts, it happens in Pocket Casts as well.

madiele commented 1 year ago

However, this isn't limited to Apple Podcasts, it happens in Pocket Casts as well.

I tried to replicate without success on android's pocketcasts, hosting both on https and http and could not replicate the issue

Is your version of Pocket casts running on an apple device too? My next guess is it's an apple thing, maybe a problem with how ios handles the requests, more research is needed

madiele commented 1 year ago

I might have an idea of what is really happening!

this article gave me an hint https://blog.logrocket.com/streaming-video-in-safari/

Safari sometimes likes to download files in uncompleted batches and stitch them together which I didn't support by design since it's kinda weird to not stream an mp3 from the second you are seeking up until the end, but apparently safari really wants to be clever! :smile:

I will see to add support for sparse Range-Request headers

The bad new is I fear this will mean a more glitchy audio stream on apple devices since seeking can be up to half a second off due to a technical limitation.

madiele commented 1 year ago

@pmd2016 while I add support can you please try to set CARGO_LOG=DEBUG and send me the section with this log in particular?

received content range

pmd2016 commented 1 year ago

Is your version of Pocket casts running on an apple device too?

Yes, I'm using an iPhone running iOS 16.4.1.

Safari sometimes likes to download files in uncompleted batches and stitch them together...

For the record: when playing an episode in the Safari browser on my MacBook, it does work.

try to set CARGO_LOG=DEBUG

Setting this does not result in more info in the logs. However, I did find this line in the Redis logs:

1:M 07 May 2023 18:11:55.218 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

I'm running on an old Raspberry Pi 2B, so maybe that's the issue?

madiele commented 1 year ago

I've added partial ranges support in the beta image, can you try if it fixes the issues?

in the docker compose change this line

- image: madiele/vod2pod-rss

to this line

- image: madiele/vod2pod-rss:beta

then do a docker compose up -d

once you have done testing feel free to revert the change

Setting this does not result in more info in the logs. However, I did find this line in the Redis logs:

that is a common warning for redis, to fix it you have to find an online guide for your distro and edit a file in your host machine, the app will still function but it might start erroring out with redis errors under low RAM conditions, if it happens invest some time into fixing it

pmd2016 commented 1 year ago

Thanks, @madiele!

I'll test this tonight when I'm home.

pmd2016 commented 1 year ago

Ok, so I just ran the beta update. Still no luck:

When I play an episode in the browser, the log is updated with this:

2023-05-09T16:36:14.048Z INFO  [vod2pod_rss::url_convert] trying to convert youtube channel url https://www.youtube.com/@Formula1
2023-05-09T16:36:49.475Z INFO  [app] processing transcode at 192k for https://www.youtube.com/watch?v=G9p62hiP2pE
2023-05-09T16:36:49.485Z INFO  [vod2pod_rss::transcoder] detected youtube url, need to find the stream url if not in cache
ffmpeg -ss 0 -i https://rr3---sn-32o-bg9e.googlevideo.com/videoplayback?expire=1683671819&ei=q3ZaZNmNNsL8-gbLorfoDQ&ip=80.60.188.181&id=o-APTRBYgdWaHB3VkPch5GTnNYrAlzP5uAMMDIC6rwr6Qa&itag=251&source=youtube&requiressl=yes&mh=wY&mm=31%2C29&mn=sn-32o-bg9e%2Csn-32o-5hn6&ms=au%2Crdu&mv=m&mvi=3&pl=25&gcr=nl&initcwndbps=2786250&spc=qEK7B1aGoU7QQMKUicj2qdNk_cY2_tk&vprv=1&svpuc=1&mime=audio%2Fwebm&gir=yes&clen=7270939&dur=593.201&lmt=1683635694742060&mt=1683649544&fvip=2&keepalive=yes&fexp=24007246&c=ANDROID&txp=4432434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgSLdDcwXXnjzUL455QhdWyA3vmpabI6nv4PZN4d1WVHECIQCWbydFG8oysDRY5ngfNXim_Fh_h_b-IpSL3lfA4-WYoQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgGnZZGyaCxsM4ilf51PTiY0V07Nc77nyMbVdfJNiYgfcCIAcIUr3RBv2CthKrNY2tqzYgqljnwFYLfsXzinTG5Cz4 -acodec libmp3lame -ab 192k -f mp3 -bufsize 5760 -maxrate 5760k -timeout 300 -hide_banner -loglevel error pipe:stdout
2023-05-09T16:37:09.645Z INFO  [vod2pod_rss::transcoder] generated ffmpeg command:
2023-05-09T16:37:09.660Z INFO  [vod2pod_rss::transcoder] streaming to client

But when I play an episode in one of the apps, things get stuck:

2023-05-09T16:40:04.690Z INFO  [vod2pod_rss::url_convert] trying to convert youtube channel url https://www.youtube.com/@Formula1
2023-05-09T16:41:00.665Z ERROR [vod2pod_rss::transcoder] [tls @ 0xf98b10] Error in the pull function.
[matroska,webm @ 0xf94dd0] Read error
[tls @ 0xf98b10] The specified session has been invalidated for some reason.
    Last message repeated 1 times
https://rr3---sn-32o-bg9e.googlevideo.com/videoplayback?expire=1683671819&ei=q3ZaZNmNNsL8-gbLorfoDQ&ip=80.60.188.181&id=o-APTRBYgdWaHB3VkPch5GTnNYrAlzP5uAMMDIC6rwr6Qa&itag=251&source=youtube&requiressl=yes&mh=wY&mm=31%2C29&mn=sn-32o-bg9e%2Csn-32o-5hn6&ms=au%2Crdu&mv=m&mvi=3&pl=25&gcr=nl&initcwndbps=2786250&spc=qEK7B1aGoU7QQMKUicj2qdNk_cY2_tk&vprv=1&svpuc=1&mime=audio%2Fwebm&gir=yes&clen=7270939&dur=593.201&lmt=1683635694742060&mt=1683649544&fvip=2&keepalive=yes&fexp=24007246&c=ANDROID&txp=4432434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgSLdDcwXXnjzUL455QhdWyA3vmpabI6nv4PZN4d1WVHECIQCWbydFG8oysDRY5ngfNXim_Fh_h_b-IpSL3lfA4-WYoQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgGnZZGyaCxsM4ilf51PTiY0V07Nc77nyMbVdfJNiYgfcCIAcIUr3RBv2CthKrNY2tqzYgqljnwFYLfsXzinTG5Cz4: Input/output error
madiele commented 1 year ago

Shame, I guess it will have to wait until I can get my hands on a apple device then.

Hopefully that is soon 🤞

pmd2016 commented 1 year ago

To rule out that it’s my server, should we check with a RSS feed that works for you on Android?

madiele commented 1 year ago

sure, send me an email here madiele92@gmail.com and I will reply with a link to an adhoc instance I just made for the purpose, I don't want to risk a DDOS on my little raspberrypi :laughing:

madiele commented 1 year ago

@pmd2016 let's continue here in case this helps other people

apparently my version works with apple podcast and pocket casts.

I'm starting to thing a setup problem.

I remember you saying on reddit that you are behind a private wireguard VPN it's possible that apple podcast and pocket casts just don't like that due to their server not being able to reach yours...

since I use wireguard too for some stuff I will try your setup and see what happens

just to confirm in pocketcasts you give the link of your lan address?

like http://192.168.1.33:8085/transcodize_rss?url=.....

and then try to listen when in your lan or with your vpn on

correct?

pmd2016 commented 1 year ago

My setup is a bit different. I'm using an Apache reverse proxy (with LE SSL), without VPN and I'm running locally on 192.168.2.31:8087. I just tried to add a show with my LAN address and that works. What doesn't work is when I use my public domain. Unfortunately, there's no log entries from the failed attempts to connect or stream episodes...

madiele commented 1 year ago

here some guesses:

you could also try to set the logs to DEBUG and give me received content range section, that would help me know if there is something weird with your headers.

I'm sorry I still can't help you, hope we'll find an answer eventually 😆

madiele commented 1 year ago

I will close the issue due to inactivity, and is seems like is more a setup issue and not a vod2pod-rss issue, if someone else has the same problem or if @pmd2016 has new info I'm willing to reopen it though, just leave a comment

fbone commented 9 months ago

Thanks for the great job! It works for me on iPhone. Apple podcast seems to support only HTTPS URL with valid certificates. I had to use an nginx reverse proxy with a valid certificate to make it work. It works fine with streaming but download of audio doesn't work. It may be the same on other platforms.

madiele commented 9 months ago

Thanks for the kind words ✌️ As for downloading I had experience some issue in the past, it does not surprise me if it has still some bugs, feel free to open a separate issue and I will eventually check it out