Closed vanativo closed 5 years ago
Thank you for reporting this. 3.2.7 now released where this is fixed. It will now adhere to the --nosubdirs
option as expected. And of course you can also use the -i <FOLDER>
option in addition to this, if you just want to check other folders than the base platform rom folder. But I assume you are aware of that option already, just wanted to mention it.
Have fun!
@muldjord , I updated to 3.2.7 to give it a try but I'm not getting the results I'm expecting, unless I'm misreading them or not using the option correctly of course.
So if I run:
/opt/retropie/supplementary/skyscraper/Skyscraper -p c64 --cache report:missing=description,publisher,developer,releasedate,marquee,video --nosubdirs
I get:
Reading and parsing resource cache, please wait...
Successfully parsed 4499 resources!
Creating report(s) for resource type(s):
description
publisher
developer
releasedate
marquee
video
Input folder returned no entries...
0 compatible files found for the 'c64' platform!
Creating file id list for all files, please wait...
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_description-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'description' resource.
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_publisher-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'publisher' resource.
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_developer-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'developer' resource.
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_releasedate-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'releasedate' resource.
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_marquee-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'marquee' resource.
Report filename: '/opt/retropie/configs/all/skyscraper/reports/report-c64-missing_video-20190901.txt'
Assembling report, please wait... Done!
0 file(s) is/are missing the 'video' resource.
All done!
But I happen to know this is not the case, there are plenty of missing resources in my C64 cache. For example, look at this entry from my current game list:
<game>
<path>/home/pi/RetroPie/roms/c64/Alternative World Games.d64</path>
<name>Alternative World Games</name>
<cover />
<image>/home/pi/RetroPie/roms/c64/media/screenshots/Alternative World Games.png</image>
<marquee>/home/pi/RetroPie/roms/c64/media/marquees/Alternative World Games.png</marquee>
<video />
<rating />
<desc />
<releasedate>19870101T000000</releasedate>
<developer />
<publisher>Gremlin Graphics</publisher>
<genre />
<players />
</game>
It says that it found 0 compatible files. I need more info. You are using the --nosubdirs
option. This means it will only look for files in the c64 rom folder (probably /home/pi/RetroPie/roms/c64) and NOT any subfolders. So if all your roms aren't directly in that folder, it won't find them. If you want it to look into a specific subfolder you need to use -i <FOLDER>
. I've tested it and it works as expected on my end.
Hmmm. Well, the interesting thing here is that I haven't changed my files from when I did the resource gathering which worked. So something seems inconsistent between the resource gathering and the cache reporting.
My setup looks like this: At the top level which is /home/pi/RetroPie/roms/c64 (the default directory for c64 roms) I have symbolic links to the actual rom files that are in subdirectories. For example:
ls -l
drwxr-xr-x 2 pi pi 4096 Aug 23 20:45 Dragon's Lair
lrwxrwxrwx 1 pi pi 84 Aug 23 20:46 Dragon's Lair.d64 -> /home/pi/RetroPie/roms/c64/Dragon's Lair/Dragon's Lair (1986)(Software Projects).d64
The following is successful:
pi@retropie:~ $ /opt/retropie/supplementary/skyscraper/Skyscraper -p c64 -s screenscraper -u XXXX:XXXX --videos --nosubdirs "Dragon's Lair.d64"
------------------------------------------
Running Skyscraper v3.2.7 by Lars Muldjord
------------------------------------------
Platform: 'c64'
Scraping module: 'screenscraper'
Input folder: '/home/pi/RetroPie/roms/c64'
Game list folder: '/home/pi/RetroPie/roms/c64'
Covers folder: '/home/pi/RetroPie/roms/c64/media/covers'
Screenshots folder: '/home/pi/RetroPie/roms/c64/media/screenshots'
Wheels folder: '/home/pi/RetroPie/roms/c64/media/wheels'
Marquees folder: '/home/pi/RetroPie/roms/c64/media/marquees'
Videos folder: '/home/pi/RetroPie/roms/c64/media/videos'
Cache folder: 'cache/c64'
DID YOU KNOW: You can scrape a single game by adding it to the command line either with partial or full path. You can also scrape a span of games with the '--startat' and '--endat' command line options.
Fetching limits for user 'Vinny', just a sec...
Setting threads to 5 as allowed for the supplied user credentials.
Reading and parsing resource cache, please wait...
Successfully parsed 4500 resources!
Looking for optional 'priorities.xml' file in cache folder... Not found, skipping...
Trying to parse and load existing game list metadata... Success!
Starting scraping run on 1 files using 5 threads.
Sit back, relax and let me do the work! :)
#1/1 (T1) Pass 1 ---- Game 'Dragon's Lair' found! :) ----
Scraper: screenscraper
From cache: NO
Search match: 100 %
Compare title: 'Dragon's Lair'
Result title: 'Dragon's Lair' ()
Platform: 'Commodore 64' ()
Release Date: '1986-01-01' ()
Developer: 'Cinematronics' ()
Publisher: 'Software Projects' ()
Players: '1' ()
Ages: '' ()
Tags: 'Action' ()
Rating (0-1): '0.65' ()
Cover: YES ()
Screenshot: YES ()
Wheel: YES ()
Marquee: YES ()
Video: YES ()
Description: ()
'The first port of the popular arcade game Dragon's Lair to a home computer, featuring the infamous Dirk the Daring.
The game, as one would expect from the long-running Dragon's Lair series, has mixed gameplay, including platform, puzzle and arcade (in the old-fashioned choose-your-path sort of way).
This first port includes two adventures: the first is to raid the lair of Singe the Dragon (hence the classic name) and rescuing Princess Daphne, and the second, in a sort of instant sequel of the first chapter, features Dirk trying to escape from the castle, now overrun by the Lizard King and its minions.'
'screenscraper' requests remaining: 19969
1/0/1
Elapsed time : 00:00:17
Est. time left : 00:00:00
---- Resource gathering run completed! YAY! ----
Writing 4500 (0 new) resources to cache, please wait... Success!
---- And here are some neat stats :) ----
Total completion time: 00:00:17
Average search match: 100%
Average entry completeness: 92%
Total number of games: 1
Successfully processed games: 1
Skipped games: 0 (Filenames saved to '~/.skyscraper/skipped-screenscraper.txt')
So that implies it found the file I specified as being a valid target in the default C64 directory since I didn't provide a path. However the cache command is not identifying the same file as being valid. Is it possible that the cache command is not recognising symbolic links while the normal gathering command is?
That's the reason. I will make it include symlinks as well. Stay tuned.
3.2.8 out now which supports symlinks for the cache features.
Wow fast work!!! I will try it now and let you know the results.
That's got it!!! Thank you again! Great work.
Awesome! You're welcome and thank you :)
Describe the feature Currently when using the --cache report:missing command line option it reports missing resources for ALL roms in the path including sub-directories. This means the output includes many files which I don't care about since I originally gathered the resources using that option, so the only files I care about are at the top level. This makes it more difficult to isolate the real problems / missing resources.
Contact mails and documentation NA
Additional context Example, I have my amiga roms organized into one directory per game in order to hold multi-disk games. I then create a symbolic link to the first disk of each game at the parent level. Eg:
drwxr-xr-x 2 pi pi 4096 Aug 22 20:40 Dragon's Lair lrwxrwxrwx 1 pi pi 109 Aug 22 22:04 Dragon's Lair.adf -> /home/pi/RetroPie/roms/amiga/Dragon's Lair/Dragon's Lair (1987)(ReadySoft)[cr ACU][t +2 ACU](Disk 1 of 8).adf
When I then run the following command: /opt/retropie/supplementary/skyscraper/Skyscraper -p amiga -s screenscraper -u XXXX:XXXX --videos --nosubdirs --refresh
I only get matches attempted on the symbolic links which is what I want.
If I then run:
/opt/retropie/supplementary/skyscraper/Skyscraper -p amiga --nosubdirs --cache report:missing=all
I get results for all files and not just the ones in the target directory. It would be really useful if I could limit the report down to only files that I want to target for gathering.