[Question] Is there a way to see the raw commands that are passed to yt-dlp from TubeSync?

[mylogon341]

I'm trying to fault find why sponsor segments are still in my videos when they get to plex but can't seem figure out what is going wrong. My source has sponsor block enabled with sponsors ticked

but when the item is set off to redownload (via "Redownload Media" button) the logs do not appear to show the initial command to yt-dlp. The real problem is that once the media completes, it appears to just clean up notify the plex of the new media - it doesn't appear to do the post processing where it removes the sponsorblock segments.

To test internal dependencies in the container, I attached to the container and ran yt-dlp -S "res:720" "https://www.youtube.com/watch?v=VRt7qRwj43g" --remux-video mp4 --sponsorblock-remove sponsor and I got the following.

Viewing the downloaded file, it showed that the sponsor segment was indeed removed.

So I'm just a little confused on what is going wrong for me. I will do some more digging through my logs to see if there is anything amiss. Thanks

ps Incase you're looking to try and reproduce this, these are my source settings Channel ID: UCtUbO6rBht0daVIOGML3c8w

[meeb]

tubesync doesn't call yt-dlp with a command in a shell, it calls it via the Python API directly so there isn't a command that can be easily displayed (without making one up) in the logs.

I have experienced an issue where tubesync downloads a media item before people have flagged sponsors properly in sponsorblock, so the download occurred before the sponsors segments are identified so they are not cut from the saved video. This is "fixed" by re-downloading the media item. Could this be what you're experiencing?

[mylogon341]

I can see how that occurs, but this is the video that I was trying to redownload – it 100% has a sponsor segment and is about a week old at this point and redownloading still isn't cutting the segments out or showing anything in the logs that mentions sponsorblock.

[meeb]

I can't see any reason why the sponsor segment wouldn't get truncated. I'll add in some logging that puts the yt-dlp API call parameters into the logs, that should hopefully assist with debugging this.

[mylogon341]

Hey. Would you be able to put an estimate on this request for additional logging at all? Thanks

[mylogon341]

Coming back to this, I'm looking back through my logs and there is a strange error that shows every time a video is re-downloaded (maybe every time). Here are the full logs for a re-downloaded video.

2023-09-17 23:16:50,020 [tubesync/INFO] Downloading media: 4Stzj2_Rlo4 (UUID: e0b761d8-aeea-4fd0-ae38-e141f68670f8) to: "/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv"
2023-09-17 23:16:50,274 [tubesync/INFO] Resizing 1280x720 thumbnail to 430x240: https://i.ytimg.com/vi/4Stzj2_Rlo4/maxresdefault.jpg
2023-09-17 23:16:50,450 [tubesync/INFO] Saved thumbnail for: 4Stzj2_Rlo4 from: https://i.ytimg.com/vi/4Stzj2_Rlo4/maxresdefault.jpg
WARNING: [youtube] unable to extract initial player response; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [youtube] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
2023-09-17 23:16:55,005 [tubesync/INFO] Downloading media: 4Stzj2_Rlo4 (UUID: e0b761d8-aeea-4fd0-ae38-e141f68670f8) to: "/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv"
WARNING: [youtube] unable to extract initial player response; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
2023-09-17 23:16:59,241 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 19.9% of 59.35MiB at 2.33MiB/s, 00:20 remaining
WARNING: [youtube] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
2023-09-17 23:17:01,508 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 35.0% of 59.35MiB at 4.43MiB/s, 00:08 remaining
2023-09-17 23:17:02,093 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 40.1% of 59.35MiB at 4.94MiB/s, 00:07 remaining
2023-09-17 23:17:03,270 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 40.1% of 59.35MiB at 1.66MiB/s, 00:21 remaining
2023-09-17 23:17:03,739 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 49.7% of 59.35MiB at 3.95MiB/s, 00:07 remaining
2023-09-17 23:17:06,960 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 54.7% of 59.35MiB at 2.25MiB/s, 00:11 remaining
2023-09-17 23:17:08,043 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 69.6% of 59.35MiB at 2.61MiB/s, 00:06 remaining
2023-09-17 23:17:08,091 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 59.8% of 59.35MiB at 2.54MiB/s, 00:09 remaining
2023-09-17 23:17:10,408 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 69.5% of 59.35MiB at 1.48MiB/s, 00:12 remaining
2023-09-17 23:17:15,177 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 99.6% of 59.35MiB at 2.66MiB/s, 00:00 remaining
2023-09-17 23:17:15,254 [tubesync/INFO] [youtube-dl] finished downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 59.35MiB in 00:00:19
ERROR: Object of type CommaSepChoiceField is not JSON serializable
2023-09-17 23:17:15,351 [tubesync/INFO] Successfully downloaded media: 4Stzj2_Rlo4 (UUID: e0b761d8-aeea-4fd0-ae38-e141f68670f8) to: "/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv"
2023-09-17 23:17:15,455 [tubesync/INFO] Scheduling task to download thumbnail for: You're a Dream of the Universe (According to Science) from: https://i.ytimg.com/vi/4Stzj2_Rlo4/maxresdefault.jpg
2023-09-17 23:17:15,488 [tubesync/INFO] Scheduling media server updates
2023-09-17 23:17:16,145 [tubesync/INFO] [youtube-dl] downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 100.0% of 59.35MiB at 3.44MiB/s, 00:00 remaining
ERROR: Unable to rename file: [Errno 2] No such file or directory: '/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv.part' -> '/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv'
2023-09-17 23:17:16,147 [tubesync/INFO] [youtube-dl] finished downloading: 2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv - 59.35MiB in 00:00:14
ERROR: Object of type CommaSepChoiceField is not JSON serializable
2023-09-17 23:17:16,231 [tubesync/INFO] Successfully downloaded media: 4Stzj2_Rlo4 (UUID: e0b761d8-aeea-4fd0-ae38-e141f68670f8) to: "/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv"
2023-09-17 23:17:16,364 [tubesync/INFO] Scheduling media server updates
2023-09-17 23:17:20,050 [tubesync/INFO] Updating media server: Plex server at http://192.168.2.5:32400
2023-09-17 23:17:20,383 [tubesync/INFO] Resizing 1280x720 thumbnail to 430x240: https://i.ytimg.com/vi/4Stzj2_Rlo4/maxresdefault.jpg
2023-09-17 23:17:20,523 [tubesync/INFO] Saved thumbnail for: 4Stzj2_Rlo4 from: https://i.ytimg.com/vi/4Stzj2_Rlo4/maxresdefault.jpg

As you can see, I'm getting two errors ERROR: Object of type CommaSepChoiceField is not JSON serializable which I don't know what it means (I'm guessing it's a python thing), and ERROR: Unable to rename file: [Errno 2] No such file or directory: '/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv.part' -> '/downloads/video/Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q]/2023-09-12_kurzgesagt_youre-a-dream-of-the-universe-according-to-science_[4Stzj2_Rlo4]_720p-avc1-mp4a.mkv' which seems strange - there are no .part files left over so I'm not sure why it's trying to do it again. I've not seen this before when using yt-dlp locally.

Also, there doesn't appear to be any mention of sponsorblock at all. I do wonder if the errors above are causing the sponsorblock process to fail in some way.

Does any of this make sense to you by chance?

ps. I'm using a Postgres db because I was getting issues with the default config in settings errors while I was just navigating around the web. I'm not sure if that explains anything 🤷

[meeb]

Yes CommaSepChoiceField is used for the sponsorblock settings, so I would guess that error is related to why your sponsorblock isn't working. Try resetting the sponsorblock flags for this source.

The second error (Unable to rename file) implies that once yt-dlp has downloaded the media it doesn't exist on disk to rename. This may be related to the sponsorblock failure which runs as a post download filter.