search

andrey-yantsen / plexiglas

Sync plex content across multiple servers
MIT License
21 stars 3 forks source link

Crash on downloading #3

Closed Xalaxis closed 5 years ago

Xalaxis commented 5 years ago

When using the following --simple-sync-url, a crash occurs:

PS C:\Users\Wolfie\Desktop\Plexiglas> plexiglas --simple-sync-url "https://app.plex.tv/desktop#!/server/<redacted>?key=%2Flibrary%2Fsections%2F21&typeKey=%2Flibrary%2Fsections%2F21%2Fall%3Ftype%3D1&filters=unwatched%3D1&save=1&limit=&sort=originallyAvailableAt%3Adesc" 1


[01/12/2018 17:27:24] [INFO] [plexiglas] Downloading Split (2016).mkv to C:\Users\Wolfie\Desktop\Plexiglas\Movies, file size is 31.37 GiB
Split (2016).mkv.part:   0%|                                                                         | 0.00/33.7G [00:00<?, ?B/s]Traceback (most recent call last):
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Wolfie\AppData\Local\Programs\Python\Python37-32\Scripts\plexiglas.exe\__main__.py", line 9, in <module>
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\cli.py", line 242, in main
    required_media = plugin.sync(plex, opts)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\simple_sync.py", line 118, in sync
    download_media(plex, section.title, item, part, opts, mark_downloaded)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\content.py", line 217, in download_media
    savepath=savepath, showstatus=True, rate_limit=opts.rate_limit)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\content.py", line 165, in download
    if bar:
OverflowError: cannot fit 'int' into an index-sized integer```
andrey-yantsen commented 5 years ago

Checking it, thank you. As far as I can see there is something wrong with progress bar... As for dirty workaround for now you can redirect stdout to a file, and it should go well.

andrey-yantsen commented 5 years ago

I've just pushed new version, v0.3.9, can you please give it a try?

andrey-yantsen commented 5 years ago

@Xalaxis ping

Xalaxis commented 5 years ago

@andrey-yantsen Thanks for the ping, I missed the first email. Downloading now works, but with a new error:


[05/12/2018 13:44:18] [INFO] [plexiglas] Downloading deadmau5 - For Lack Of A Better Name Strobe.mp3 to C:\Users\Wolfie\Desktop\Plexiglas\Music, file size is 24.14 MiB
deadmau5 - For Lack Of A Better Name Strobe.mp3.part: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 25.3M/25.3M [00:11<00:00, 2.27MB/s]
[05/12/2018 13:44:29] [ERROR] [plexiglas] File "C:\Users\Wolfie\Desktop\Plexiglas\Music\deadmau5 - For Lack Of A Better Name Strobe.mp3.part" has an unexpected size (actual: 25307561, expected: 25309609)
Traceback (most recent call last):
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Wolfie\AppData\Local\Programs\Python\Python37-32\Scripts\plexiglas.exe\__main__.py", line 9, in <module>
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\cli.py", line 242, in main
    required_media = plugin.sync(plex, opts)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\simple_sync.py", line 118, in sync
    download_media(plex, section.title, item, part, opts, mark_downloaded)
  File "c:\users\wolfie\appdata\local\programs\python\python37-32\lib\site-packages\plexiglas\content.py", line 226, in download_media
    raise ValueError('Downloaded file size is not the same as expected')
ValueError: Downloaded file size is not the same as expected```
andrey-yantsen commented 5 years ago

@Xalaxis The last error was due to big file size (nearly 32 gigs), now you have completely new one :) Somehow the file size by announced Plex is different from the size of downloaded file... If you retry the download, would it fail again?

Xalaxis commented 5 years ago

@andrey-yantsen Sorry, I didn't have time to test downloading the full 32GB file, but at least the download now starts :).

Retrying the music file download fails with exactly the same error message. It's also worth noting that the incomplete .part file isn't automatically deleted (which might be worth doing if there's no point keeping it around)

Xalaxis commented 5 years ago

Trying the download with a different file works.

andrey-yantsen commented 5 years ago

Sorry, I didn't have time to test downloading the full 32GB file, but at least the download now starts :).

No worries, it's ok. At least it starts, right :)

Trying the download with a different file works.

What a... if you'd manually remove .part-file will it fix the issue? Otherwise — can you send me the "broken" file somehow (e.g. using something like https://uploadfiles.io/)?

Xalaxis commented 5 years ago

I tried manually removing the part file, but no luck unfortunately. I'd guess it's just overwritten right now anyway. Here's the partial file.

andrey-yantsen commented 5 years ago

@Xalaxis can you please also share the original mp3?

andrey-yantsen commented 5 years ago

And for the full picture — can you please send me the entire command (stripping server name, if you wish) of how you're executing the script?

Xalaxis commented 5 years ago

Here's the original file

And the command line is PS C:\Users\Wolfie\Desktop\Plexiglas> plexiglas --simple-sync-url "https://app.plex.tv/desktop#!/server/<redacted>?key=%2Flibrary%2Fsections%2F23&typeKey=%2Flibrary%2Fsections%2F23%2Fall%3Ftype%3D10&save=1&sort =viewCount%3Adesc" 1

andrey-yantsen commented 5 years ago

Okaaay, I still can't reproduce it :) Can you send me an XML with "media info" on this file? image

Honestly — I haven't tested "simple-sync" with photos/music/playlists, so at current stage I don't recommend to use it for anything except Movies or TV-Shows...

andrey-yantsen commented 5 years ago

If you'll suddenly find out that size attribute within Part tag is different from 25307561 (it's the size of file you've sent me) — try to refresh the music library (or at least the album, not sure if it's possible).

andrey-yantsen commented 5 years ago

@Xalaxis ping :)

Xalaxis commented 5 years ago

XML


<Track ratingKey="12025" key="/library/metadata/12025" parentRatingKey="12015" grandparentRatingKey="11824" guid="com.plexapp.agents.plexmusic://gracenote/track/199300679-0AF28A1ADD59F930781ECB4EA5AF61B2/199300689-4FAAA30A0CA1FAA2E5829C9AAF0B32CA?lang=en" librarySectionTitle="Music" librarySectionID="23" librarySectionKey="/library/sections/23" type="track" title="Strobe" grandparentKey="/library/metadata/11824" parentKey="/library/metadata/12015" grandparentTitle="deadmau5" parentTitle="For Lack Of A Better Name" summary="" index="10" parentIndex="1" ratingCount="1617586" viewCount="37" lastViewedAt="1543248901" year="2010" thumb="/library/metadata/12015/thumb/1523290397" art="/library/metadata/11824/art/1544670796" parentThumb="/library/metadata/12015/thumb/1523290397" grandparentThumb="/library/metadata/11824/thumb/1544670796" grandparentArt="/library/metadata/11824/art/1544670796" duration="637100" addedAt="1399227615" updatedAt="1535995826">
<Media id="12357" duration="637100" bitrate="318" audioChannels="2" audioCodec="mp3" container="mp3">
<Part accessible="1" exists="1" id="241242" key="/library/parts/241242/1399227615/file.mp3" duration="637100" file="/mnt/MMusic/Electronic Artists/Deadmau5/For Lack Of A Better Name/10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3" size="25309609" container="mp3">
<Stream id="144854" streamType="2" selected="1" codec="mp3" index="0" channels="2" bitrate="318" albumGain="-4.73" albumPeak="1.000000" albumRange="6.201957" audioChannelLayout="stereo" gain="-3.78" loudness="-14.22" lra="6.77" peak="0.676849" samplingRate="44100" displayTitle="Unknown (MP3 Stereo)"/>
</Part>
</Media>
<Mood id="2026" filter="mood=2026" tag="Dreamy Pulse"/>
<Extras size="0"> </Extras>
</Track>
</MediaContainer>
Xalaxis commented 5 years ago

The size is still the same after a refresh + analyze

andrey-yantsen commented 5 years ago

@Xalaxis can you check the file size on your computer? Because both files I have are 25307561 bytes. Something is extreeemely wrong. Somewhere... :)

Xalaxis commented 5 years ago

-rwxrwxrwx. 1 nobody nobody 25309609 May 4 2014 '10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3' 🤔

andrey-yantsen commented 5 years ago

waaaaaat

Xalaxis commented 5 years ago

This is weird

andrey-yantsen commented 5 years ago

And if you download the file from https://github.com/andrey-yantsen/plexiglas/issues/3#issuecomment-445030602 — what would be it's size?

andrey-yantsen commented 5 years ago

Maybe it's time to run fsck? :)

Xalaxis commented 5 years ago

image Here's the disparity

andrey-yantsen commented 5 years ago

I've checked once again on my side:

-rwxr-xr-x    1 virus 25307561 Dec  6 21:08 '10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3'

and md5:

5070c5b898bd534117ca9d4f82eb28b2  10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3
andrey-yantsen commented 5 years ago

There is no 25309609 in the dialogue, anyway

Xalaxis commented 5 years ago

Ah, my bad. Hmm

Xalaxis commented 5 years ago

Server: 2a620ab6921f3483e97033f84c32f745 10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3 Downloaded via Plex Web: 5070c5b898bd534117ca9d4f82eb28b2 10 - Deadmau5 - For Lack Of A Better Name - Strobe.mp3

andrey-yantsen commented 5 years ago

Which file you've uploaded to me: from the server, or from Plex Web?

Xalaxis commented 5 years ago

Plex web

andrey-yantsen commented 5 years ago

Ok, I need the original one then, can you send it?

Xalaxis commented 5 years ago

10 - Deadmau5 - For Lack Of A Better Name - Strobe.zip

andrey-yantsen commented 5 years ago

Ok, now I can reproduce it, thank you! Plex removes some mp3-tags and therefore the file size changes. I'll try to find a solution for this, or, at least, will add an option to ignore relatively small size difference.

Xalaxis commented 5 years ago

Perhaps disabling file size checking is the right way to go? Shouldn't the transport take care of ensuring no data is lost in the transfer?

andrey-yantsen commented 5 years ago

Yep, transport definitely should check the data, but I simply don't trust Plex :D I mean — if it'd decide to drop connection in the middle of the file I wouldn't be sure if it's ok, or an error... Anyway, I have a reproducible case now, I'll check what I can do here in next couple days.

andrey-yantsen commented 5 years ago

There is a new version, v0.3.10, on its way, can you please give it a try?