search

meeb / tubesync

Syncs YouTube channels and playlists to a locally hosted media server
GNU Affero General Public License v3.0
1.96k stars 123 forks source link

{Title} fails for unicode characters #527

Open kanjieater opened 3 months ago

kanjieater commented 3 months ago

image Expected title 2024-06-02 - 5000cc.ogg

2024-06-02 - 神回】「5000ccのバイクに乗ってる」「岡田は俺に歯向かったから梅田会のヤクザにブルドーザーで轢き殺される」としおの同級生・嘘つきアホの深田が伝説となった映像【す

In addition, instead of failing on {titleFull} being too long, it would be great if it could just truncate the file name to be the appropriate length.

meeb commented 3 months ago

Thanks for the issue. Can you provide an example YouTube video that exhibits this behavior please? The titles processed through tubesync do support unicode so something else must be going on here. What file system is your media being stored on?

kanjieater commented 3 months ago

Looks like this user is facing similar issues, so you can use the video they mentioned https://github.com/meeb/tubesync/issues/522#issue-2421013006

Heres a direct link to another one https://youtu.be/D__D6nqc3kY?si=VnrU0KgXPFkZEnm1

meeb commented 3 months ago

That's a different issue, that's the title being too long for the file system not that the title isn't rendering at all.

With the media item https://www.youtube.com/watch?v=D__D6nqc3kY can you please skip then unskip the video in tubesync? This will refresh the metadata which may be one cause if your issue.

kanjieater commented 3 months ago 
2024-07-29 09:56:16 tubesync  |          n = 17IElUrESNzf90aU- ; player = https://www.youtube.com/s/player/1f8742dc/player_ias.vflset/en_US/base.js
2024-07-29 09:56:16 tubesync  | 2024-07-29 09:56:16,600 [tubesync/WARNING] [youtube] D__D6nqc3kY: nsig extraction failed: Some formats may be missing
2024-07-29 09:56:16 tubesync  |          n = rRHObeNStJKwbj_Pi ; player = https://www.youtube.com/s/player/1f8742dc/player_ias.vflset/en_US/base.js
2024-07-29 09:56:16 tubesync  | 2024-07-29 09:56:16,891 [tubesync/INFO] Saved 380292 bytes of metadata for: サイコパスおじさん【岡田斗司夫 切り抜き】 / 9d21e0c7-0709-4f1b-b460-7004adf61770
2024-07-29 09:56:18 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:18 -0500] "GET /tasks HTTP/1.1" 200 3820 "https://ts.something.xyz/media/9d21e0c7-0709-4f1b-b460-7004adf61770" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:20 tubesync  | 2024-07-29 09:56:20,845 [tubesync/INFO] Downloading media: D__D6nqc3kY (UUID: 9d21e0c7-0709-4f1b-b460-7004adf61770) to: "/downloads/audio/サイコパスおじさん【岡田斗司夫 切り抜き】/2024-04-27 - .ogg"
2024-07-29 09:56:21 tubesync  | WARNING: [youtube] D__D6nqc3kY: nsig extraction failed: Some formats may be missing
2024-07-29 09:56:21 tubesync  |          n = NdTLWNKGy8lTBZlBr ; player = https://www.youtube.com/s/player/1f8742dc/player_ias.vflset/en_US/base.js
2024-07-29 09:56:21 tubesync  | WARNING: [youtube] D__D6nqc3kY: nsig extraction failed: Some formats may be missing
2024-07-29 09:56:21 tubesync  |          n = E85m5vDS6ou-oWUJJ ; player = https://www.youtube.com/s/player/1f8742dc/player_ias.vflset/en_US/base.js
2024-07-29 09:56:22 tubesync  | 2024-07-29 09:56:22,030 [tubesync/INFO] [youtube-dl] finished downloading: 2024-04-27 - .ogg.ja.vtt - 262.01KiB in 00:00:00
2024-07-29 09:56:22 tubesync  | 2024-07-29 09:56:22,625 [tubesync/INFO] [youtube-dl] downloading: 2024-04-27 - .ogg - 45.0% of 23.22MiB at 21.10MiB/s, 00:00 remaining
2024-07-29 09:56:22 tubesync  | 2024-07-29 09:56:22,747 [tubesync/INFO] [youtube-dl] downloading: 2024-04-27 - .ogg - 60.1% of 23.22MiB at 27.46MiB/s, 00:00 remaining
2024-07-29 09:56:23 tubesync  | 2024-07-29 09:56:23,104 [tubesync/INFO] [youtube-dl] downloading: 2024-04-27 - .ogg - 89.9% of 23.22MiB at 15.99MiB/s, 00:00 remaining
2024-07-29 09:56:23 tubesync  | 2024-07-29 09:56:23,187 [tubesync/INFO] [youtube-dl] downloading: 2024-04-27 - .ogg - 100.0% of 23.22MiB at 22.98MiB/s, 00:00 remaining
2024-07-29 09:56:23 tubesync  | 2024-07-29 09:56:23,289 [tubesync/INFO] [youtube-dl] finished downloading: 2024-04-27 - .ogg - 23.22MiB in 00:00:01
2024-07-29 09:56:24 tubesync  | ERROR: Conversion failed!
2024-07-29 09:56:24 tubesync  | 2024-07-29 09:56:24,176 [tubesync/INFO] Successfully downloaded media: D__D6nqc3kY (UUID: 9d21e0c7-0709-4f1b-b460-7004adf61770) to: "/downloads/audio/サイコパスおじさん【岡田斗司夫 切り抜き】/2024-04-27 - .ogg"
2024-07-29 09:56:24 tubesync  | 2024-07-29 09:56:24,197 [tubesync/INFO] Copying media thumbnail from: /config/media/thumbs/9d/9d21e0c7-0709-4f1b-b460-7004adf61770.jpg to: /downloads/audio/サイコパスおじさん【岡田斗司夫 切り抜き】/2024-04-27 - .jpg
172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /media/9d21e0c7-0709-4f1b-b460-7004adf61770 HTTP/1.1" 200 4266 "https://ts.something.xyz/tasks" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/styles/tubesync.css HTTP/1.1" 200 34554 "https://ts.something.xyz/media/9d21e0c7-0709-4f1b-b460-7004adf61770" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /media-content/9d21e0c7-0709-4f1b-b460-7004adf61770 HTTP/1.1" 206 524096 "https://ts.something.xyz/media/9d21e0c7-0709-4f1b-b460-7004adf61770" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/fontawesome/fa-regular-400.woff2 HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/fontawesome/fa-solid-900.woff2 HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/roboto/roboto-regular.woff HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/roboto/roboto-bold.woff HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/fontawesome/fa-brands-400.woff2 HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
2024-07-29 09:56:45 tubesync  | 172.30.0.10 - kanjieater [29/Jul/2024:09:56:45 -0500] "GET /static/fonts/roboto/roboto-light.woff HTTP/1.1" 304 0 "https://ts.something.xyz/static/styles/tubesync.css" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"

Sure, It's reproducible.

meeb commented 3 months ago

Can you paste what's in your "media format" field for this source?

kanjieater commented 3 months ago

image {yyyy_mm_dd} - {title}.{ext}

Edit: I am using title, not title_full to reproduce this error. I want to use title, but can't because of the other issue we discussed, 255 ch limit

meeb commented 3 months ago

Can you screenshot the media item page for D__D6nqc3kY as well?

kanjieater commented 3 months ago

image

EDIT: Sorry, i had to change my settings back to {title} for this to show as pictured (which is what causes the issue), original screenshot has been corrected

meeb commented 3 months ago

And finally, what file system are you saving the media to?

kanjieater commented 3 months ago

I'm using Docker. It's in WSL2, and mounts to a local windows drive.

meeb commented 3 months ago

So local non-networked NTFS?

kanjieater commented 3 months ago

yeah should be (assuming docker implementation doesn't do something crazy - can't say i've looked into it)

meeb commented 3 months ago

OK thanks. There's no logical reason this shouldn't work, everything in tubesync does fully support unicode so I'll see if I can replicate your issue.