Snatcher, Popful Mail, and others will not import to playlist properly.

[Khronikos]

I can't get this to pop on any SegaCD playlist, and believe me I have tried. I am curious as to who is actually getting these to show on playlists and what their source is. I have tried Snatcher and the rest mentioned from Internet Archives, guaranteed to be accurate and clean, and they still will not pull when scanned. These are full multi-bin roms that play perfectly when selected in Load Content, but I can't get them into a playlist.

This is my last bastion of hope here. I have no idea what RA considers a good burn of Snatcher, but it simply can't be realistic. When Internet Archive sets of the highest quality are not working, then something is wrong.

On PS2 this is definitely a major issue as well. I even got my own Maximo working, but it says beta for some reason even though it's from a legit copy. Maybe I am doing something wrong, but so far few games have worked for many PS2 games and definitely the Sega CD games outlined here. A couple others don't work, but these are the ones that really grind my gears. Any help or direction is appreciated. I don't make this thread lightly. I've tried 6 different versions of Snatcher.

Win 7 x64/RA latest stable version.

[Khronikos]

Alright, I have managed to get this working through a Manual Scan. I feel like RA needs to tune its scanning code possibly. Why would a manual scan work with cue and then a single file or directory scan does not pick up any of them? Again, it's only a few of them that don't pull, but they don't at all. Same with PSP stuff, and I have mostly been getting No-Intro stuff or verified dumps.

Anyway, you can close this I guess if there are no plans to improve how Retroarch sees a rom dump and imports to playlist. I guess Manual scan will just have to do.

[Sanaki]

Manual scan works with anything. You could scan cat photos into a playlist if you wanted. Improving the auto-scanner is a constant task, and one doomed to never be perfect. But there are plenty of people working on different aspects of it despite that.

[i30817]

Lots of people have outdated segacd dumps. Redump always works but redump is rare or incomplete for segacd (and saturn and dreamcast).

[Khronikos]

Lots of people have outdated segacd dumps. Redump always works but redump is rare or incomplete for segacd (and saturn and dreamcast).

I've never gotten one copy of Snatcher to work literally anywhere.

Confirmed to not work, which is why I made the thread. Sol-Feace and Popful Mail also do not work. I just manually scanned them, and luckily the art and everything is there just needed some tweaking. I understand it is a long process of getting things right.

[inactive123]

Please don't post links like that again.

[Khronikos]

Please don't post links like that again.

Sorry, was that a link to no-intro? I can't remember, I was just trying to help people understand. I forgot it links to things. But yeah, that was the issue and dumps seem fine. I'll refrain from that.

[inactive123]

Yeah, it was a link to it. Anyway, as long as you understood the reasoning for its removal, we can move on and it's no problem.

[i30817]

Confirmed to not work, which is why I made the thread. Sol-Feace and Popful Mail also do not work. I just manually scanned them, and luckily the art and everything is there just needed some tweaking. I understand it is a long process of getting things right.

Try a manual scan or a rename. It will work if and only if the playlist name and game name is on the thumbnail server: http://thumbnails.libretro.com/

The thumbnail server has 'problems' because the libretro-database repository doesn't create enough copies for all possible names (which they could and should do with a script instead of depending on contributors imo, since the redump/tosec/m3u names are predictable).

Especially if retroarch wants everyone to use m3us, the least they can do is make the m3u name (name without the disc part of the name) available for cd dumps, which is actually rare.

They also could use symlinks to save space with the right kind of download tool, which would work for both games with more than 1 cd and 'alternative names'.

[Sanaki]

They also could use symlinks to save space with the right kind of download tool, which would work for both games with more than 1 cd and 'alternative names'.

I did open #10665 regarding that. If that gets fixed, I'll personally go through and clean up a pretty big chunk of the thumbnail db. First using something akin to fslint, then by hand.

[i30817]

git knows how to clone symlinks instead of following them during clone, so making the thumbnail server 'get them' seems straightforward, the trouble is whatever download tool that RA uses (that isn't git), deal with symlinks, preferably without 2 https requests.

I think the best strategy in platforms with symlink capability might be something custom like 'check the existence of the file in the thumbnail server, download the file if a regular file but download the symlink target if not, then create a symlink to the target locally if the name requested is different from the file got'.

Problem with that idea is that if the 'backing file' already exists, it's a useless download too, but to prevent that you have to make two http requests (first to check the symlink target and if it already exists locally, second to actually get it). Maybe someone smarter than me can do this without wasting space either in the server or client in 1, probably by sending sibling file information in the http request.

Meh, this all sounds more complicated than a simple wget, so i completely understand why RA might not want to make the step of helping the client save space (most people don't have multiple copies of the same game anyway). The git strategy of copying the target bytes instead of the symlink is maybe for the best for the client.

But a pre-commit/script step that creates the copies or symlinks to have the right dump names in libretro-database would help a lot with missing covers and is a orthogonal concern.

[Sanaki]

Regardless, we're going out of scope for this issue.

[Khronikos]

Confirmed to not work, which is why I made the thread. Sol-Feace and Popful Mail also do not work. I just manually scanned them, and luckily the art and everything is there just needed some tweaking. I understand it is a long process of getting things right.

Try a manual scan or a rename. It will work if and only if the playlist name and game name is on the thumbnail server: http://thumbnails.libretro.com/

The thumbnail server has 'problems' because the libretro-database repository doesn't create enough copies for all possible names (which they could and should do with a script instead of depending on contributors imo, since the redump/tosec/m3u names are predictable).

Especially if retroarch wants everyone to use m3us, the least they can do is make the m3u name (name without the disc part of the name) available for cd dumps, which is actually rare.

They also could use symlinks to save space with the right kind of download tool, which would work for both games with more than 1 cd and 'alternative names'.

Well they have all scanned. I guess it's not a huge deal. I definitely went through LOADS of work doing my playlists though. Countless new box arts for missing ones across many different emus. I'd like to submit some for the PSP, but I am not even sure how to do that efficiently with Git. I know I can make a fork or whatever and then propose changes, but not sure how much work it is to submit a bunch of stuff.

I see some progress now and then though, and it seems like the database gets updated, just not at times for the particular rom, so that means renaming it and/or the thumbnails. It's been a wild ride. For instance, there isn't even a correct version of Sonic 1 on Genesis. It's the horrible white art that bears no relation to the main art most of us saw on that system on the box. Those are some of the things I would like to update, but I am not sure how to contribute efficiently, because I can't be on here a lot. It seems like only a few people do the box art stuff. I've made the highest quality PNGs I can find of a bunch of stuff box and screen. Also would like to contribute better screens to some games as they get redundant or some of them are just bad shots.

[i30817]

You can do it from the GUI interface, but it's not quite 'smooth' and you can gotcha yourself because you won't be able to update the PR without the command line git so if the project maintainer says he wants some adjustments, you'll have to close down the PR, create a new commit with another branch and reopen the PR, which is annoying.

You go to libretro-thumbnails project, read the readme at the bottom of the page, create your images as the readme requests; in the right directories, of correct name, of the right size without borders if possible (retroarch screenshots have the bad habit of including borders, so be careful if you're screenshooting the menu or ingame), find the console on the filelisting, for instance ps1, click on it to go to the subproject, fork it with the button in the upper right side, then on the fork click the button 'add file -> upload files' and 'drag and drop files'.

Drag and drop because that way you can drop three directories with the 3 game thumbnails inside, which would take multiple commits otherwise. Then instead of 'create directly in the master branch', choose 'create a new branch'. If done correctly your new thumbnails will be on the right directories among the others and the web GUI will be on the new branch. Then you'll see github showing a notification near the commit message to open a PR in the parent project. And you do that.

As mentioned, if you screw up, you'll have to make another branch with another drag and drop on 'master', so change to master in your fork in the web gui and redo the commit to a new branch and PR.

I did this this way myself because i had 0 interest in cloning locally all of the ps1 thumbnails just to add a new one. Doing it like this, it's all on github-side and decently fast. Unlike cloning locally. I also suggest doing 1 commit/PR per game, in case something has to be reverted upstream later, though it isn't very necessary in this case, it's a good habit.

[Khronikos]

Alright, thank you very much. So I can just submit the box and screen folders themselves in one go for one system correct? As in say 20 pngs in each of these for one system?

I will get to that at some point. I did open a fork, but hadn't done anything yet. I have to gather all the ones I have changed or added to and compare them to the directory. Does RA accept box and screen art for common translations, e.g., SNES RPGs? I had to do about ten of those. I don't use a hack playlist, and they come up in my normal playlists when scanned mostly. Mostly, they are just dupes of the ones already there under the name of the translation, but it would save anybody with that translation time.

---

From: i30817 notifications@github.com Sent: Friday, July 10, 2020 4:46 AM To: libretro/RetroArch RetroArch@noreply.github.com Cc: Khronikos orbitingcow@hotmail.com; Author author@noreply.github.com Subject: Re: [libretro/RetroArch] Snatcher, Popful Mail, and others will not import to playlist properly. (#10976)

You can do it from the GUI interface, but it's not quite 'smooth' and you can gotcha yourself because you won't be able to update the PR without the command line git so if the project maintainer says he wants some adjustments, you'll have to close down the PR, create a new commit with another branch and reopen the PR, which is annoying.

You go to libretro-thumbnails projecthttps://github.com/libretro-thumbnails, read the readme at the bottom of the page, create your images as the readme requests; of the right size without borders if possible (retroarch screenshots have the bad habit of including borders, so be careful if you're screenshooting the menu or ingame), find the console on the filelisting, for instance ps1https://github.com/libretro-thumbnails/Sony_-_PlayStation, click on it to go to the subproject, fork it with the button in the upper right side, then on the fork click the button 'add file -> upload files' and *drag and drop files'.

Drag and drop because that way you can drop three directories with the 3 game thumbnails inside, which would take multiple commits otherwise. Then instead of 'creating directly in the master branch', choose 'create a new branch'. If done correctly your new thumbnails will be on the right directories among the others. Then you'll see github showing a notification near the commit message to open a PR in the parent project. And you do that.

As mentioned, if you screw up, you'll have to make another branch with another drag and drop on 'master', so change to master in your fork in the web gui and redo the commit to a new branch and PR that.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/libretro/RetroArch/issues/10976#issuecomment-656588567, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2EAMQ5EO64PNNPSFZE2MLR23PPDANCNFSM4ORPCFGQ.

[i30817]

Alright, thank you very much. So I can just submit the box and screen folders themselves in one go for one system correct? As in say 20 pngs in each of these for one system?

Yes if you want to.

I don't know about the translations/hacks. The problem about them is that they have no standard name (or rather it's easy to do a standard name, by following the name on the hack database - I even made a tool to help with updating the database and hacks (linux only for updating the .dat file, but usable in windows to check for updates to patches with a extra file on the game dir) - but since libretro-database is not accepting SNES and NES and a few systems hacks checksums (they complained about 'too many') and because retroarch has no way to use checksums for scanner anymore i just don't bother.

Which means that since i was the only one using the tool, the names on the database are getting more and more distant from the predictable name the tool produces. Crap like FFVI_whatever_by_zeromus_v1.2(varies).fig or something. I'd be very mad if i had to name my hacks that way to get the image, especially if the version number was included and i updated the hack.

But sure, you can try asking. Open a issue. But be ready to 'decide on a standard' and renaming a lot of hack entries on libretro-database if you want the names to look good.

I'd prefer if you used the format my tool produces, which uses the original redump/nointro name of the games for translations main name, and the first hack name in romhacking.net for hacks main name and adds metadata on a [] group at the end of the name with T-Language if translation, Hack if they're a hack, and T-Language if they're both, and the main authors romhacking names/group (if translation or mixed the last translation authors, if hack only the first hack authors) plus the number of extra patches applied if > 1 and crucially not the version for the image not to disappear when i update the patch.

A complicated system to support multiple patches 'composition' without making awful names by ordering what's most important. Translations more important than hacks (translations do not need or have names on romhacking.net so all of them use the original game main name and are differentiated by the author in the metadata), the last translation more important than previous translations (addendum), and the opposite for hacks, where first hack is the most important and the addendum often is just bug fixes. The description field contains all of the hacks applied with all the authors and versions.

But this is probably arrogant, do what you will, i quit expecting sanity from RA romhack support long ago, much less ambitious ideas to systematize hack names or to collect their checksums and keep them updated (which is what that tool was for, besides warning me that i should update the hack). Just the fact that the automatic scanner uses serials sabotages all hacks to use the metadata of the original game if you use it, and i'm certainly not going to use libretro-database names + manual scanner if they're essentially non-standardized and ugly.

[i30817]

Alright, thank you very much. So I can just submit the box and screen folders themselves in one go for one system correct? As in say 20 pngs in each of these for one system?

Yes this is correct, and you probably understood the first time, but just to be obvious: the fork before you upload and choose the directories must be a fork of the subproject (they are directories with the playlist console name). If you try to commit directly to libretro-thumbnails, it won't work because you're going to end up trying to commit to a subproject, that repo is just a handy way to get to the subprojects.

[Khronikos]

Yes, directly to the sub project console or whatever. So I submit an entire clone as a fork of say PS1. And then I upload my part into that, and they can see all the changes correct?

AFAIK the translation roms are usually pretty samey from good sources, as in (T-En...) and then their project, but yeah those might not get included for whatever reason. It's still a huge hassle having to rename all of these and clone screens just to get it in there but whatever. I assume this all takes a long time lol to sort our rationally. But I think I will just start a little PSP and PS1 to try and fill out the gaps a little for major games and provide better boxart when necessary with a few sprinkled in from other systems like Sonic 1 Genesis, which needs to be changed.

What I am seeing is that they have a lot of (Disc 1 of 2) but no matching (Disc 1) for a lot of them, which I see a lot of roms name themselves, and I prefer as well this (Disc 1) type notation. I think those should be okay since their database already has met the paradigm of including both, just not consistently anywhere.

Anyway, thanks for your help man. I'll try and submit sometime here in the next few weeks.

---

From: i30817 notifications@github.com Sent: Friday, July 10, 2020 2:48 PM To: libretro/RetroArch RetroArch@noreply.github.com Cc: Khronikos orbitingcow@hotmail.com; Author author@noreply.github.com Subject: Re: [libretro/RetroArch] Snatcher, Popful Mail, and others will not import to playlist properly. (#10976)

Alright, thank you very much. So I can just submit the box and screen folders themselves in one go for one system correct? As in say 20 pngs in each of these for one system?

BTW, just to be clear since you might have misunderstood; the fork before you upload and choose the directories must be a clone of the subproject (they are directories with the playlist console name). If you try to commit directly to libretro-thumbnails, it won't work because you're going to end up trying to commit to a subproject, that's just a handy way to get to the subprojects.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/libretro/RetroArch/issues/10976#issuecomment-656859671, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2EAMSZZGDXEZLVUTPSMZDR25V7ZANCNFSM4ORPCFGQ.

[i30817]

Honestly a script that could copy and rename a standard name to dump variant names would be better utility. You could fix everything at once and fix it in the future without extra work.

This issue is about it: https://github.com/libretro-thumbnails/libretro-thumbnails/issues/77 though i just mention taking the name from the databases, which might be incomplete so it's not a perfect idea there.

[Khronikos]

Oh for sure, a script would purely be awesome for some of this stuff. And it should be done at some point for the easier things at least.

---

From: i30817 notifications@github.com Sent: Saturday, July 11, 2020 2:30 AM To: libretro/RetroArch RetroArch@noreply.github.com Cc: Khronikos orbitingcow@hotmail.com; Author author@noreply.github.com Subject: Re: [libretro/RetroArch] Snatcher, Popful Mail, and others will not import to playlist properly. (#10976)

Honestly a script that could rename a standard name to dump variant names would be better utility.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/libretro/RetroArch/issues/10976#issuecomment-657008136, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2EAMSCQWJZJFOCSURKR4TR3AII3ANCNFSM4ORPCFGQ.

[ghost]

I use Lakka OS and i noticed that Popful Mail does not add properly as well. I did do a manual scan of it but its not the only title and core that has a database issue. I end up having to add thumbnails manually for it but gets really out of hand when needing to do it when you have a larger amount of games.

[Khronikos]

I use Lakka OS and i noticed that Popful Mail does not add properly as well. I did do a manual scan of it but its not the only title and core that has a database issue. I end up having to add thumbnails manually for it but gets really out of hand when needing to do it when you have a larger amount of games.

Yeah, I just manually scan most folders now. It just doesn't do it right even on some games straight from No-Intro. Whatever I guess.

[LibretroAdmin]

Retest against latest version.