Universal-Rom-Tools / Universal-XML-Scraper

Scraper de Rom
195 stars 46 forks source link

UXS crashes all the time :( #236

Closed verybadsoldier closed 6 years ago

verybadsoldier commented 7 years ago

After half a year I came back to Retropie and also to Universal XML scraper. So I downloaded the latest version today and tried to re-scrape my ROMs. Unfortunately UXS crahses regularly and I cannot finish a single session. I tried to scrape my Mega Drive ROMs (1700 ROMs). But after a some time (sometimes sooner, sometimes later) UXS just crashes while scraping.

Please find the logfiles here: LOGs.zip

Anything else I should try or provide to help fixing this? Thanks!

verybadsoldier commented 7 years ago

I think the reason that my RetroPie-HTPC is going to suspend (energy saving) after a while. I increased the inactivity timeout to 5 hours and UXS did not crashed and scraped all games. So I think that is the reason.

Maybe UXS could show a message box "Could not read ROMs" or something.

Universal-Rom-Tools commented 7 years ago

I need to add a test so :S

Or maybe something can be done with SSH (sending a wake up or a false activity to the PI ?)

verybadsoldier commented 7 years ago

Hm I was wrong it seems :( Also just crashed without HTPC going to suspend ...

Universal-Rom-Tools commented 7 years ago

arg :( it is always on the same "rom" ? or is it random ?

verybadsoldier commented 7 years ago

It is random. Last night I scraped all MegaDrive ROMs (>3000) but now I am trying N64 and it crashed after 231 ROMs. I started again and now it is running at 324.

Universal-Rom-Tools commented 7 years ago

Arg :( Random bug :(

I hate them...

By luck, have you a "process" to create the bug ? If it's really random, it will be a pain in the a... to find why :'(

Drovitch commented 7 years ago

Hi all,

Same on my side. UXS crash randomly...

verybadsoldier commented 7 years ago

@Universal-Rom-Tools Really nothing special: I scrape a ROM set and it will crash sooner or later.

I will describe my environment a bit, maybe it can help:

Universal-Rom-Tools commented 7 years ago

Nothing really different from my configuration :( (except I'm on Recalbox and My roms are on a NAS accessed via Samba Share) I use 13/5/1 threads on my test and it never happens to me :'(

This sort of bug are really insane... And apparently you are not alone :'( Maybe a Memory leak somewhere on a function... I'll try to investigate... The aim is to reproduce the bug on my computer... I'll try more test (scraping very big folder for exemple)

I keep you in touch...

verybadsoldier commented 7 years ago

Also happens with just 1 thread :(

verybadsoldier commented 7 years ago

I checked out the latest source code and ran directly the au3 files to get maybe a better error message. This is what appears when using au3 instead of the exe. I hope it helps: image

verybadsoldier commented 7 years ago

I think it has something to do with the generation of the images. The above error message seems to indicate that as the source code line calls a function of the gdiplus.dll which contains a set of image functions.

Also after a crash when I start the scraping process again then I think it skips the images it has generated already on the first run and it runs longer this time. I think after 2 or 3 runs I can get a whole set scraped this way.

Jaxel commented 7 years ago

I noticed it only crashes if I have Marquees enabled.

verybadsoldier commented 7 years ago

For me it crashes also without Marquees.

If nothing helps: I can offer to just trial & error different versions to find out which version introduced the crash. In Nov 2016 I used UXS without problems. So if you want, feel free to provide different versions that make sense to test and I will do that and report which ones worked and which ones crashed. Maybe there were changes to the image generation code at some point?

Warfare360 commented 7 years ago

Same issue here, impossible to finish a local scrap of my fba reference set (~4400 roms in 7z). I'm new with this software, so I'm 'glad' to find other guys with the same problem ^^ UXS v2.2.0.2 64 bits OS Windows 10 / AMD Using Mix Arcade Zoom Moon Retropie with Marquee Only one thread The duration before crash is variable ...

Any news ?

PS : Rolled back to version 2.2.0.0 this night, and this one is working without any crash.

rhoover87 commented 7 years ago

can confirm above post by Warfare360-- sporadic crashes fixed by 2.2.0.0 rollback.

Jaxel commented 7 years ago

I think I figured it out... UXS crashes if it is NOT the active window on your computer.

If it is the active window, it doesn't crash... Basically, every 60 seconds or so, I tab back to it to momentarily make it the active window... hasn't crashed since.

Universal-Rom-Tools commented 7 years ago

wow... thank you so much guy to help me tracking this ^^

I'm really sorry, I haven't so much time right now :(

But I see some interresting way : GDI+ error ? not sure... but the $@result array is a good way to check ;) I have made some "modification" in the last version to try out some 4 points distortions in the MIX. Normally they aren't active, but maybe there is a "side effect" :S

confirmed by the fact 2.2.0.0 haven't this bug...

But the "active" windows tips ??? If it's right, I have no clue on the problem... I hope it's just a coincidence ^^

Promise, I'll try to correct this quickly after my hard period of 'real' work :(

rhoover87 commented 7 years ago

Thanks for the constant support!

Skeptical about it being an "active" window, as UXS was the only application I had running (Windos 10) and it still suffered a crash.

verybadsoldier commented 7 years ago

I can confirm that 2.2.0.0 works fine! Just scraped >4000 ROMs without crash.

But I cannot confirm the foreground window stuff. Still crashes for me.

Universal-Rom-Tools commented 7 years ago

Just make some correction (commenting all line refered to the 4 points distortions + a little bug I found on an _GDIPlus_ImageDispose function in _myfunction library).

Just testing with about 1300 Roms (FBA) with the most complicated MIX I have (Bigshot) I can tell in 25 minutes if it's ok without error on my computer ^^

If yes, I will post here a link to a version to test ;) So if some of you guys (who often have the error) can make some test, I will greatly thank you...

ghost commented 7 years ago

I would be happy to test it out. Thanks!

Universal-Rom-Tools commented 7 years ago

Here we go !!!

No error.

Just relaunch a Mame Scrape (2555 Roms) ;)

ghost commented 7 years ago

I downloaded it and am trying it out now with one library of around 800 games for one system. How do I test it with all systems? If I choose scrape all systems, I get a popup with an array table of all the systems with the options to copy or exit the script. If I click the copy options, nothing happens. If I click the exit script, UXL quits.

ghost commented 7 years ago

Only 36 files done and I get a "Scraper XML Universel has stopped working" error with a Close program button. Trying again. It did not prompt me to recover the files it had already done, even though I had a bunch of stuff in the temp folder. The last version asked me if I wanted to write out the files to an xml file from the previous attempt after it had crashed.

EDIT: Crashed again. This time at 106 files completed. What information or logs would you like?

Universal-Rom-Tools commented 7 years ago

Arg :(

Can someone test the 2.2.0.1 ? from 2.2.0.0 to 2.2.0.1 I add the new progress bar (and it use GDI+) If it crash on 2.2.0.1 and not on 2.2.0.0 Then it's that... it's the only solution :(

ghost commented 7 years ago

https://1drv.ms/u/s!ApLfgMfQ7OL9gbMuzP3j3Rm9JM_RYQ = "Scraper XML Universel has stopped working" 2.2.0.2 = "AutoIt Error - Subscript used on non-accessible variable" 2.2.0.1 = Good! No crashes! Completed every time. Scrapped all systems and all libraries.

Sined66 commented 7 years ago

https://1drv.ms/u/s!ApLfgMfQ7OL9gbMuzP3j3Rm9JM_RYQ = same issue for me (tested in background and in foreground). I confirm the 2.2.0.1 was runnning perfectly (except for the name of roms not found, which leads me to download the 2.2.0.2...).

Dragontorc commented 7 years ago

Same issue here. Tried to scrape GB (1600 files). Crashes within 5 minutes from beginning of scraping. Tested on 3 different machines running W7 x64 (AMD) and W10 x64 (AMD and Intel CPUs).

Universal-Rom-Tools commented 7 years ago

OK... Can't find where is the problem on 2.2.0.2....

I rallback to 2.2.0.1 and make correction little by little on it...

Universal-Rom-Tools commented 7 years ago

Someone to test this version ?

I hope it's good. Scraped several time 2000+ roms (even Iso)

And no problem...

https://1drv.ms/u/s!ApLfgMfQ7OL9gbMuzP3j3Rm9JM_RYQ

Please tell me before I release ;)

Sined66 commented 7 years ago

Testing against 11000 Amstrad CPC roms... Running for 10 minutes (300 roms made)... Results in 3 hours (or before if crash ;-))

ghost commented 7 years ago

Downloaded it and testing against a smaller library of 600 roms.

ghost commented 7 years ago

First run of Atari 2600 completed. No crash. Trying a larger scrape now. Looks good so far!

EDIT: The Scrape All is working great now. It is switching systems and not crashing. Notice one bug. The game console logo on the program is not changing to match the console it is currently scraping when doing a scrape all. The logo is just staying on the first system logo.

ghost commented 7 years ago

Over 2000 roms so far! No crashing! It is finding most roms except for the ones that have multiple periods, hyphens, or apostrophes.

EDIT: Over 5000 files and no crashing!

Dragontorc commented 7 years ago

233 roms Atari 5200. No crash.

Sined66 commented 7 years ago

Running perfectly foreground during this night (over 20000 files...) This morning, put the running session in background -> BSOD few minutes after... I'll retry this ASAP to confirm... Nothing else was running (except a firefox instance).

Universal-Rom-Tools commented 7 years ago

Just pushed the new version ;)

Normally it will be OK...

Let me know ;)

ghost commented 7 years ago

It is working good so far! No crashes! I really like the apostrophe update!!!!

For example, Game Boy library has 565 roms. 2.2.0.2 reported 101 missing. 2.2.0.3 reported 51 missing.

A very big improvement. If you could fix the following symbols it would really clean up a bunch of them , - & . !

Thanks!

Sined66 commented 7 years ago

More than 41000 files with no problem (no bsod...) in update mode. Maybe if you could display the system being scrapped (if you process in alphabetical order).

Dragontorc commented 7 years ago

Tested 2.2.0.3 with several different systems (various sets between 100 and 6000 roms). All good. Really nice update.

Chinagonuk commented 6 years ago

I’ve just encountered this error for the first time, but i think it was caused when my SD card run out of space mid scrape.

The problem i have now is my SD card is maxed out but i have no images or gameslist file so i have no idea what junk has filled up the space that was there before i tried scraping.

I assume there is a tempory folder somewhere that needs clearing out, else i’ll be wasting valuable space on my card. If anyone could point me to this folder, i would very much appreciate it.

Universal-Rom-Tools commented 6 years ago

Chinagonuk Is UXS on your SD card ??? If yes, you should put it on your Windows Computer. If not, so the temporary files are on your computer, not on the PI... (in the UXS folder exactly ;) )

UXS put on the PI only what is needed (gamelist.xml and Picture)

Please open a new issue for that if needed ;)