[Feature Request] Sublists based on folder structure

[chinagreenelvis]

I keep my ROMs sorted into subdirectories based on region, license, genre, etc. Rather than manually populating RetroArch with tons of custom playlists based on my categorizations, it would be much more convenient (and easier to browse) if the files added to any given playlist from a directory had the option of being sub-listed within that playlist, reflecting the folder structure.

[i30817]

What would be more convenient yet (if i wasn't convinced it was impossible in general because of lacking metadata and accurate scanning for hacks) would be if you could save metadata searches (from 'explore') as playlists, with some kind of more robust search and more robust metadata (not just 'plain and uninformative genre, company and language').

Then, with a similar mechanism to how you can 'refresh' manual scanner playlists today - thanks to the hard work of jdgleaver - you could do the same when RA updates metadata. First you'd 'update' the manual scan playlists to add or remove games, then you'd update the genre lists which take from those playlists.

If the query language gets 'negative' and better genres and characteristics you could even do things like '2d-beatemup && !contemporary' and other things like that. Unfortunately, game metadata from hobbyists (you know what? In general, all game metadata is poor) is completely and utterly pathetic, mainly because it's complicated to fill and they get into the rut of pretending a 'general genre' actually means anything instead being the pre-history of the web of things. What does 'adventure' actually means, or even 'action-adventure'. What does that tell me about the themes, plot, setting or characters? Nothing.

RA could make a big feature about allowing filling metadata 'approved' tags from retroarch itself and upload those to their servers then rollback and ip ban the inevitable trolls trying to turn every sega or nintendo games into a +18, but honestly, i think it never will. Not enough incentive when you can just use the flawed and painfully uninformative current external databases.

[i30817]

That said your feature is a nice idea but you can kind of do it today with the manual scanner - it can autoupdate as you move games around because it saves the scanned directory. If you really really want it.

I wouldn't have the patience because RA playlists should be organized by console first and foremost because of the core association and of the 'problem' of duplicates with scanning, even with a the manual scanner using filenames (there are many ports and no intro/redump kind of reuse names). So you'd endup with 'gamecube - rpg' and 'ps1 - rpg' or more arbitrarily specific divisions etc.

But i suppose if you already organized your folder structure, you have that patience.

A warning: neither scanner is perfect, and several games will not scan whatever you do in large collections and have to be added manually or have to have 'invented metadata' added (scummvm core did this but i think they gave up on it, because it was hell for users).

In the 'automatic' scanner because of bugs, or the serial is fucked from origin, or misleading false positives that look like duplicates (hacks) or if it's one of the systems still using CRCs, misleading crcs because the file used isn't actually readonly.

In the manual scanner because the name of the game it's searching for is either missing from the database, a hack name or something in a format you forgot to accept, or because of false positives from 'games' you need for the emulator to run, but don't actually want (for instance, mame bioses). The option to accept a ctrlmame dat file to filter is useful then cause because i think the scanner ignores files marked as 'system' files. And if you can make a script to transform xml you can use xslt to 'make your own filters' from the dumper dat file (and any self respectable romset has a a dat file, for instance, i could semi-easily make a 'whdload english only playlist' for puae like this. Again, lack of metadata hurts this feature - i had to string process it from the filename itself instead of setting up something in the menu to say 'scan this dir but exclude all non-english' - but it's only really often used for 'filter MAME bios' for 99%.

Or because if there is no way to associate a 'sane' filename to the the entry and use that as id (the best feature of the manual scanner) you get a bunch of 'agssetup.cfg' entries or similar (agssetup.cfg is a configuration file from a engine with about 1 thousand+ games and as a configuration file, it's not unusual to be edited by users, so the CRC is out too, if you did edit it). To fix that last i think it would be nice if the manual scanner had yet another option to say 'find the files but take the name to match medata from the file parent directory, not the file'.

[chinagreenelvis]

A feature for saving explore filters as custom playlists would be nice. In the meantime, allowing playlists to have the option of sublist structures based on the organization of the files themselves would solve my problems entirely since I don't have to wait on a community-driven metadata solution to organize my own ROMs.

[i30817]

The problem here is that this solution was already rejected for the manual scanner, a solution that is frankly superior. I even think, we can just create a empty playlist with a prototype of what the manual scanner wants and the path to your 'sublist' and 'refresh' the playlist to get get what you want, with the gotchas i mentioned.

For instance, this is a empty version of my whdload playlist:

{
  "version": "1.5",
  "default_core_path": "/home/i3/.config/retroarch/cores/puae_libretro.so",
  "default_core_name": "Commodore - Amiga (PUAE)",
  "base_content_directory": "/home/i3/Documents/Games",
  "label_display_mode": 0,
  "right_thumbnail_mode": 0,
  "left_thumbnail_mode": 0,
  "sort_mode": 0,
  "scan_content_dir": "/home/i3/Documents/Games/Commodore - Amiga/Amiga WHDLoad",
  "scan_file_exts": "lha",
  "scan_dat_file_path": "/home/i3/Documents/Games/Commodore - Amiga/Amiga WHDLoad/games.dat",
  "scan_search_recursively": true,
  "scan_search_archives": false,
  "scan_filter_dat_content": true,
  "items": [
]
}

And i'm pretty sure that if i hit 'refresh' in the manage playlist menu it would fill it all again. The name of the playlist file is the name of the playlist in RA since it's not a config item there. And there is a separator to have multiple types of file to scan i don't remember (probably ";" but it could be ',' or space even).

[chinagreenelvis]

Why would the option for subfolder respect on a manual scan - and the ability to navigate playlists according to that folder structure - be rejected? I would think it would be a desirable option for many users.

The most important thing for me, besides organization, is the navigation. I'm sure I could automate a script to write some manual playlists according to my categories and sub-categories, but the result would be an ugly mess since each playlist would wind up taking up a primary spot in the navigational menu - I would have to scroll through each one for every system, rather than just changing systems and then navigating by region/genre/playercount, etc.

Allowing for subfolder recognition on a scan, again - not by default but as an option - seems like a much more elegant way of addressing the problem, especially since metadata for games is, as you've pointed out, terribly inconsistent and for some systems functionally non-existent and will probably continue to be for many years to come. (Not to mention that the creation of playlists by metadata would probably result in the same navigational problem, which is a lack of heirarchy, and would also take unnecessary time if the user has already gone through the process of organizing by subfolders.)

To make it even better, I can imagine binding a key to switch between the folder heirarchy and the "all games" list. This is more or less how I've had it set up in Maximus Arcade for the last ten years (which I may continue using in spite of all of the things I love about RetroArch... the ability to create sublists in a sensible navigation is more or less a dealbreaker for me, I think.)

[i30817]

Ah you want subfolders in the actual playlist not just playlists per directory. I was misunderstanding almost the whole thing myself.

Good luck with your FR then.

[chinagreenelvis]

Right; an optional subfolder structure within a primary playlist.

[agluck91]

Just want to plus one this one