alucryd / oxyromon

Rusty ROM OrgaNizer
Other
120 stars 13 forks source link

`sort-roms` is fighting with `generate-playlists` #107

Closed TylerVigario closed 9 months ago

TylerVigario commented 9 months ago

For some reason, generate-playlists and sort-roms are not playing nice. I'm glad that it doesn't affect GameCube as well.

I've reduced the output and split the commands to make it easier to read

[root@tylervigario Temp]# oxyromon generate-playlists -a
...
Processing "Nintendo - GameCube"
Creating "Medal of Honor - Rising Sun (USA).m3u"
Creating "Baten Kaitos Origins (USA).m3u"
Creating "Tom Clancy's Splinter Cell - Chaos Theory (USA) (En,Fr,Es).m3u"
Creating "Teenage Mutant Ninja Turtles 2 - Battle Nexus (USA).m3u"
Creating "Lord of the Rings, The - The Third Age (USA).m3u"
Creating "Freaky Flyers (USA).m3u"
Creating "Resident Evil Zero (USA).m3u"
Creating "Tom Clancy's Splinter Cell - Double Agent (USA) (En,Fr,Es).m3u"
Creating "Teenage Mutant Ninja Turtles 3 - Mutant Nightmare (USA).m3u"
Creating "Enter the Matrix (USA).m3u"
Creating "Metal Gear Solid - The Twin Snakes (USA).m3u"
Creating "GoldenEye - Rogue Agent (USA).m3u"
Creating "Tales of Symphonia (USA).m3u"
Creating "Resident Evil - Code - Veronica X (USA).m3u"
Creating "Baten Kaitos - Eternal Wings and the Lost Ocean (USA).m3u"
Creating "Resident Evil (USA).m3u"
Creating "Resident Evil 4 (USA).m3u"
Creating "Tiger Woods PGA Tour 2004 (USA).m3u"
Creating "Killer7 (USA).m3u"
Creating "Tiger Woods PGA Tour 2005 (USA).m3u"
...
Processing "Sony - PlayStation"
Creating "Armored Core - Master of Arena (USA).m3u"
Creating "Legend of Dragoon, The (USA).m3u"
Creating "Dragon Warrior VII (USA).m3u"
Creating "Galerians (USA).m3u"
Creating "Rival Schools - United by Fate (USA).m3u"
Creating "Wing Commander III - Heart of the Tiger (USA).m3u"
Creating "Final Fantasy VIII (USA).m3u"
Creating "Command & Conquer (USA).m3u"
Creating "Hive, The (USA).m3u"
Creating "Street Fighter Collection (USA).m3u"
Creating "Driver 2 (USA).m3u"
Creating "Final Fantasy VII (USA).m3u"
Creating "Dracula - The Last Sanctuary (USA).m3u"
Creating "PlayStation Underground 3.3 (USA).m3u"
Creating "Star Wars - Rebel Assault II - The Hidden Empire (USA).m3u"
Creating "Valkyrie Profile (USA).m3u"
Creating "Final Fantasy IX (USA).m3u"
Creating "Metal Gear Solid (USA).m3u"
Creating "Parasite Eve (USA).m3u"
Creating "Chrono Cross (USA).m3u"
Creating "Alone in the Dark - The New Nightmare (USA).m3u"
Creating "Dracula - The Resurrection (USA).m3u"
Creating "Golden Nugget (USA).m3u"
Creating "PlayStation Underground Number 1 (USA).m3u"
Creating "PlayStation Underground Volume 2.3 (USA).m3u"
Creating "Star Ocean - The Second Story (USA).m3u"
Creating "Dragon Valor (USA).m3u"
Creating "PlayStation Underground 4.2 (USA).m3u"
Creating "D (USA).m3u"
Creating "PlayStation Underground Volume 2 Issue 4 (USA).m3u"
Creating "Tales of Destiny II (USA).m3u"
Creating "Wild Arms 2 (USA).m3u"
Creating "Novastorm (USA).m3u"
Creating "Chronicles of the Sword (USA).m3u"
Creating "Resident Evil 2 - Dual Shock Ver. (USA).m3u"
Creating "Lunar - Silver Star Story Complete (USA).m3u"
Creating "PlayStation Underground Number 3 (USA).m3u"
Creating "PlayStation Underground 3.2 (USA).m3u"
Creating "Thousand Arms (USA).m3u"
Creating "PlayStation Underground 4.3 (USA).m3u"
Creating "PlayStation Underground 3.1 (USA).m3u"
Creating "Shadow Madness (USA).m3u"
Creating "In Cold Blood (USA).m3u"
Creating "Xenogears (USA).m3u"
Creating "You Don't Know Jack (USA).m3u"
Creating "PlayStation Underground Volume 2 Issue 1 (USA).m3u"
Creating "Psychic Detective (USA).m3u"
Creating "BrainDead 13 (USA).m3u"
Creating "Lunar 2 - Eternal Blue Complete (USA).m3u"
Creating "Grandia (USA).m3u"
Creating "Riven - The Sequel to Myst (USA).m3u"
Creating "PlayStation Underground Number 4 (USA).m3u"
Creating "Syphon Filter 2 (USA).m3u"
Creating "PlayStation Underground 3.4 (USA).m3u"
Creating "Evil Dead - Hail to the King (USA).m3u"
Creating "Countdown Vampires (USA).m3u"
Creating "Command & Conquer - Red Alert - Retaliation (USA).m3u"
Creating "Command & Conquer - Red Alert (USA).m3u"
Creating "PlayStation Underground Volume 2 Issue 2 (USA).m3u"
Creating "X-Files, The (USA).m3u"
Creating "Covert Ops - Nuclear Dawn (USA).m3u"
Creating "G-Police (USA).m3u"
Creating "Shockwave Assault (USA).m3u"
Creating "Wing Commander IV - The Price of Freedom (USA).m3u"
Creating "Colony Wars (USA).m3u"
Creating "Fox Hunt (USA).m3u"
Creating "Fear Effect 2 - Retro Helix (USA).m3u"
Creating "PlayStation Underground Number 2 (USA).m3u"
Creating "Fear Effect (USA).m3u"
Creating "Parasite Eve II (USA, Canada).m3u"
Creating "Juggernaut (USA).m3u"
Creating "Arc the Lad Collection - Arc the Lad III (USA).m3u"
Creating "Oddworld - Abe's Exoddus (USA).m3u"
Creating "PlayStation Underground 4.1 (USA).m3u"
Creating "PlayStation Underground 4.4 (USA).m3u"
Creating "Heart of Darkness (USA).m3u"
Creating "Koudelka (USA).m3u"
Creating "Resident Evil 2 (USA).m3u"
...
[root@tylervigario Temp]# oxyromon sort-roms -a
...
Processing "Nintendo - GameCube"
Nothing to do
...
Processing "Sony - PlayStation"
Summary:
"Metal Gear Solid (USA).m3u" -> "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sony - PlayStation/Metal Gear Solid (USA).m3u"
"Resident Evil 2 - Dual Shock Ver. (USA).m3u" -> "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sony - PlayStation/Resident Evi"Rival Schools - United by Fate (USA).m3u" -> "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sony - PlayStation/Rival Schools - United by Fate (USA).m3u"
⠈                                                                                                                             Proceed? yes
thread 'main' panicked at src/database.rs:2330:25:
Error while updating romfile with id 201571
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
alucryd commented 9 months ago

I'll try to reproduce, haven't experienced that issue before, then again I don't have a lot of CD games but I believe I do have the RE and MGS games.

alucryd commented 9 months ago

BTW, the head of the develop branch is now using tokio instead of async-std as its async runtime, the former is actually maintained and seems a lot more mature. I was hoping it would help with some of the hard to reproduce bugs I can't explain.

TylerVigario commented 9 months ago

I previously used the develop branch, but the web portion stopped functioning recently. I didn't want to start an issue for something that might be in progress or my own doing, so I reverted to main.

EDIT 2: I confirmed this is present in the latest develop.

EDIT:

No matter how often I (begrudgingly) agree to sort a different .m3u, it keeps wanting to do it. If I get some time, I'll look over the logic for sorting and see if I spot any apparent issues.

[root@tylervigario Temp]# oxyromon purge-roms -t -m -o
Processing missing ROM files
Processing trashed ROM files
Processing orphan ROM files
[root@tylervigario Temp]# oxyromon sort-roms -a
Processing "Atari - 2600"
Nothing to do

Processing "Atari - 5200"
Nothing to do

Processing "Atari - 7800"
Nothing to do

Processing "Atari - Jaguar (J64)"
Nothing to do

Processing "Atari - Lynx"
Nothing to do

Processing "MAME"
Nothing to do

Processing "Microsoft - Xbox"
Nothing to do

Processing "Microsoft - Xbox 360"
Nothing to do

Processing "Nintendo - Game Boy"
Nothing to do

Processing "Nintendo - Game Boy Advance"
Nothing to do

Processing "Nintendo - Game Boy Color"
Nothing to do

Processing "Nintendo - GameCube"
Nothing to do

Processing "Nintendo - Nintendo 3DS (Encrypted)"
Nothing to do

Processing "Nintendo - Nintendo 64 (BigEndian)"
Nothing to do

Processing "Nintendo - Nintendo DS (Decrypted)"
Nothing to do

Processing "Nintendo - Nintendo Entertainment System (Headered)"
Nothing to do

Processing "Nintendo - Super Nintendo Entertainment System"
Nothing to do

Processing "Nintendo - Virtual Boy"
Nothing to do

Processing "Nintendo - Wii"
Nothing to do

Processing "Sega - 32X"
Nothing to do

Processing "Sega - Game Gear"
Nothing to do

Processing "Sega - Master System - Mark III"
Nothing to do

Processing "Sega - Mega Drive - Genesis"
Nothing to do

Processing "Sony - PlayStation"
Summary:
"Rival Schools - United by Fate (USA).m3u" -> "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sony - PlayStation/Rival Schools - United by Fate (USA).m3u"
⠐                                                                                                                             
Processing "Sony - PlayStation 2"
Nothing to do

Processing "Sony - PlayStation 3"
Nothing to do

Processing "Sony - PlayStation Portable"
Nothing to do

[root@tylervigario Temp]# oxyromon sort-roms -a
Processing "Atari - 2600"
Nothing to do

Processing "Atari - 5200"
Nothing to do

Processing "Atari - 7800"
Nothing to do

Processing "Atari - Jaguar (J64)"
Nothing to do

Processing "Atari - Lynx"
Nothing to do

Processing "MAME"
Nothing to do

Processing "Microsoft - Xbox"
Nothing to do

Processing "Microsoft - Xbox 360"
Nothing to do

Processing "Nintendo - Game Boy"
Nothing to do

Processing "Nintendo - Game Boy Advance"
Nothing to do

Processing "Nintendo - Game Boy Color"
Nothing to do

Processing "Nintendo - GameCube"
Nothing to do

Processing "Nintendo - Nintendo 3DS (Encrypted)"
Nothing to do

Processing "Nintendo - Nintendo 64 (BigEndian)"
Nothing to do

Processing "Nintendo - Nintendo DS (Decrypted)"
Nothing to do

Processing "Nintendo - Nintendo Entertainment System (Headered)"
Nothing to do

Processing "Nintendo - Super Nintendo Entertainment System"
Nothing to do

Processing "Nintendo - Virtual Boy"
Nothing to do

Processing "Nintendo - Wii"
Nothing to do

Processing "Sega - 32X"
Nothing to do

Processing "Sega - Game Gear"
Nothing to do

Processing "Sega - Master System - Mark III"
Nothing to do

Processing "Sega - Mega Drive - Genesis"
Nothing to do

Processing "Sony - PlayStation"
Summary:
"Rival Schools - United by Fate (USA).m3u" -> "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sony - PlayStation/Rival Schools - United by Fate (USA).m3u"
⠒
Proceed? [Y/n]
`` 
alucryd commented 9 months ago

I'm having trouble trying to reproduce, I even have a unit test covering that particular use case (import multi-disc game, generate playlist, then call sort-roms to move to 1G1R).

Could you quickly check where the file currently is, its permissions as well as its parent folder, and possibly try purge-roms -m -o -t?

TylerVigario commented 9 months ago

This is happening to CUE files! I dumped all the Redump CUE files into the import directory (obviously running import-roms) and then ran generate-playlists, which creates m3u files for DAT entries that only have the CUE file. Maybe check for ROM completeness before generating playlist files?

alucryd commented 9 months ago

Ah! I don't have orphan CUE files, that explains it. Yes, I'll change it so it only generates playlists when you have all the required roms.

alucryd commented 9 months ago

https://github.com/alucryd/oxyromon/commit/c71b720fbed54aca4f883f76d3d6af7aa8c1eae6 should do the trick. I went a step further and made sure all the games in a set are complete before generating the playlist. For example if you've got the full BIN+CUE FF7 disc 1 and 2 but are missing the 3rd one, the playlist isn't generated.

And thanks a lot for the gift, that's very kind of you. I wish you a happy new year :)

TylerVigario commented 9 months ago

@alucryd I appreciate your help updating this; it appears to be working great! I wish you a great new year as well!