Error that wipes gameslist.xml files

[zandrillian]

I've been having a re-occurring error when I try to scrape images to Retropie v4.1 using both the pi2 and pi3.

The error is as follows:

 Line 22076 (File "Universal_XML_Scraper64.exe"): Error: Subscript used on non-accessible variable.

Once this error occurs it wipes the gameslist.xml file for the current system I'm scraping and also starts corrupting all the other gameslist.xml files that have already been scraped successfully. I have tried this three times on the pi2 and once on the pi3 each time starting with a fresh Retropie, adding roms and then scraping. I've even tried multiple micro SD cards and same error.

I'm hoping this is a known issue or someone can suggest a fix.

[Universal-Rom-Tools]

Can you send me your logs files ?

First time I see that : ( I need to check in logs...

[zandrillian]

Here you go, thank you for looking into this.

Log1 http://pastebin.com/VaSrdQRP

Log http://pastebin.com/GDGqt295

[Universal-Rom-Tools]

Are they Log just after a "crash" ?

Because I can't see any error ><

All seem to be fine... Till the End : [10:45:38] - Universal XML Scraper Closed

[zandrillian]

Perhaps the error isn't being logged. The program scrapes about 90% of the images then a window pops up that says "Line 22076 (File "Universal_XML_Scraper64.exe"): Error: Subscript used on non-accessible variable." I hit the okay button and the program closes, when I explore the pi the images are in the proper folder but the xml file is blank. There is an additional error that pops up when I go to shut down the computer that has to do with Universal XML Scraper which I haven't written down but I will try and recreate these errors and provide more information.

I just noticed a new version of the scraper was released a few hours ago, I'm just trying to scrape right now and see what happens.

I also noticed I wasn't logged in with my screenscreeper.fr login, would that be causing an issue?

[Universal-Rom-Tools]

Logging Screenscraper isn't an obligation (even if you can scrape "faster" with contributing in DB information ;) )

If the Error come again (with the new version) Can you PasteBin the last logs file (just after the bug ;) ) because I think the one you send is "clean" ;)

[zandrillian]

Just happened again and I believe the error was logged this time. I had to split the log1.txt file into two pastebin logs as it was too big.

Log: http://pastebin.com/GDGqt295 Log1_01: http://pastebin.com/VaSrdQRP Log1_02: http://pastebin.com/DBA4Q5YH

[Universal-Rom-Tools]

I don't understand !!

Log said you are scraping Atari2600 And log1 say you are scraping Lynx

< what the hell ?

[zandrillian]

Very weird, I downloaded the new version of the scraper, put it in a brand new folder away from the previous version. I scraped Atari2600 not Lynx and I made sure to send you the only log files that showed up. I'm stumped as well.

The problem usually goes away when I do a fresh install of retropie. I can scrape maybe 6 or 7 systems correctly then suddenly the error occurs. The error doesn't always occur on the same system which makes it even more difficult to troubleshoot.

[zandrillian]

I'm going to try with a fresh copy of retropie and send you the log when it first happens.

[zandrillian]

The other error that I get when I go to either restart or shut down my computer is as follows:

AutoIt v3: Scraper64.exe - Application Error The instruction at 0x00110000 reference memory at 0x00110000. The memory could not be written. Click on OK to terminate the program.

[zandrillian]

Should I be restarting the pi or emulation station between scrapes? I just successfully scraped Atari2600 now I've moved to Atari7800 without restarting anything.

[zandrillian]

I think it may be the japanese titles I have in SNES that is causing the issue. I was only able to paste a small portion of the log due to size restriction but here you go.

http://pastebin.com/pD0CiZ7D

[Universal-Rom-Tools]

very strange...

The Empty XML is 'normal' because UXS generate it at the end. (so if he crash he don't generate anything)

The error about memory is pretty strange too...

Some thing to check :

Have you enough space on your drive (the one where UXS is. C: I think) ? Have you an Antivirus ? Try to launch it in Admin mode (SHIFT+Right CLic launch as administrator) If you scrape "small" amount of rom did it work ? (Error only occur on big romset ?) Can you try on an other computer ?

I see "no" error on logs.. even on jap roms... Sometime 1 rom with a "special" caractere can make strange things.. But you say it's random.

I'm really lost... Don't know why you have these "crash" :(

[zandrillian]

Hard drive has 545GB free, no antivirus installed, I have not tried to launch in Admin mode but I will today. I have been making disk images as I go to save my current state so when the error happens I can rollback. After the SNES error yesterday I rolled back and added MAME & FBA and they seem to be working fine. I'm going to make another save state and try again with SNES, NES and Genesis as they seem to be the problem rom sets.

Thank you for all the help so far, I will let you know what happens today.

[Universal-Rom-Tools]

save state

Are you on a VM ???

< I think it may be the problem ;) (I heard about other people having problem on VM :( )

[zandrillian]

No, not on a VM. I meant that I've been making back up images of the pi sd card as a way to back up my successful progress.

[retroted]

hi! i'm having HUGE problems with the whole emustation (i believe because of this tool).. can you please tell me what error 31 is? i get this error line: /!\ Error Downloading File : C:\Users\admin\Desktop\Universal XML Scraper\TEMP\example.xml /!\ Bytes read: 0 /!\ Size: 41 /!\ Complete: True /!\ successful: False /!\ @error: 31 /!\ @extended: 3

some covers and descriptions do download but they don'r appear in retropie... the last time i did a scrape my whole emustation menu got corrupted, the menus are pure black with some buttons hardly showing...

please help :)

(i will upload log file when i'ts done scraping)

[Universal-Rom-Tools]

I think the problem is different. First, in no way UXS can generate "problem" in emulationstation (it's only an XML file :S so even if it's bad formated, don't think it will crash ES...)

For the @error = 31 it normally mean : "Impossible to save the file"... Did UXS have the good right to write to the folder ? (Try launch it in admin mode to be sure...)

[zandrillian]

I think I realized what my issue is, memory card seems to be full. Size: 59G, Used: 57G, Avail: 85M, Use%: 100%. I'm going to delete some PSX roms to free up space and try to scrape again to confirm.

EDIT - YES! That was the issue, the card filled up but I hadn't realized it. I don't know if you can have Universal XML Scraper display when your card is full and stop scraping so it doesn't clear all the other gameslist.xml files. I know it was my error for not checking the available space on the card but a notification within the program would help the user out should that situation arise.

[retroted]

so i updated emustation, all retropie an other packets but i still get the black screen on main screen, when i press F4 and it takes me to the command line, there are lines that say "lvl0: Could not find node in gamelist "/home/pi/.....ALL FOLDERS) today i will do a total reinstall... :/ damn

EDIT: so the last thing i did before formatting.. i got into the es_settings.cfg and changed the theme from simple-dark to pixel (the only one i knew by memory:)), and now the system works again... so the theme that i was using when doing the scrape got corrupted in the process... i can't think of anything else, the only other thing i did before the problems started, i did move the raspberry to another TV an used it there for a game or two, nothing else... Yesterday when i was posting here i was doing another total scrape, the files were downloaded and everything, but now i see, that most of them arent scsraped, the ones that are include only 1 image, but i was using Standard 3img.. log.txt log1.txt

As it seems this scraper does not work for me.. :(

will try to setup a virtual machine on pc and try to scrape those...

[Universal-Rom-Tools]

@zandrillian Great ^^ so it's not a big bug ^^ For information it's hard to know when a distant linux drive is full :S I don't know if I can manage this sort of information without putting lot's of Test (like SSH interrogation) who will decrease speed an efficacity to the scraper... (so my Advice : always check your free space ^^)

@retroted I already have an issue about "Theme"... Don't know why but it seems some theme don't like Scraped data (Sselph had the same issue)... Changing theme is the only solution we have for now :(

[zandrillian]

Thank you for all your help, yes it was definitely my fault for not checking the space on the SD card.

[retroted]

thanks for reply! it seems that the pictures get downloaded and assembled correctly, they are stored on my SD, but won't get inserted to the games... can you find sothing wrong? i was trying to scrape only NES with img4 option, the images are there but the xml files weren't updated?

[Universal-Rom-Tools]

Don't see anything wrong... Did you kill your Emulationstation first ? (before scrape) ?

[Universal-Rom-Tools]

No news... hope it works...