[Feature Request] Add an auxiliary directory where to search for games

[pocomane]

I propose to add in file_io.cpp, as the first scanned game directory (line 854 right now), something like:

#define GAME_OVERLAY_DIR "/run/MiSTer/games"
if (isPathDirectory(GAME_OVERLAY_DIR)) {
  printf("Found Overlay dir: " GAME_OVERLAY_DIR "\n");
  strncpy(dir, GAME_OVERLAY_DIR, dir_len);
  return 1;
}

You can change the specific directory to anything in a tmpfs file system like /run, /tmp, /var, etc. (Alternatively, a per-game subdirectory could be used, e.g. snprintf(temp_dir, 1024, "%s/%s", GAME_OVERLAY_DIR, dir);)

This feature let's us to write scripts and utilities that can easly make links to specific roms. For example, in mbc currently I have to work with temporary directories in /media/fat/games/NES (or similar) to archieve the same result, and the final functionality still have issues when external usb storage is used.

[sorgelig]

I don't get the idea of overlay and temp dir in general. Dirs are already too complex sine you can have games folder on SD card, on USB and even on root.. I see no reason to confuse users more. As for some specific use i recommend to make a symlink to temp dir (or whatever you want) from original game folder. For example inside games/nes you can create a symlink to some folder on usb and thus expand existing folder, so games/nes/usb will point to some USB storage.

[pocomane]

The issue is that there is no unique "games/nes". I can make a link in /media/fat/games/nes/my_link_dir, then one day I attach an usb storage, and I do not see my_link_dir anymore. [1]

But the most important point is that a folder in a tmpfs results in a more robust system. As I said, this is intended to support scripts and utilities, so consider for example an utility that fails for some reason, leaving the file system dirty. Then, if it was using the tmpfs directory, the user just have to restart the MiSTer to clean up. If instead the utility is forced to work with regular directory making links and similar things, sooner or later the user will see all these auxiliary files and folders populating his/her SD/HD. Edward Murphy docet.

About the user confusion: this is intended as an "API" [2], not something that the regular user must care about. Instead, I think that the complexity increase for an "Advanced user" is minimal; where "Advanced user" means one that already know that there is a sequence of scanned directories with priority: usb, usb/games, cifs, cifs/games, fat, fat/games. He/she must just add one item to the not-so-long mental list [3].

---

[1] Furthermore, from a script you can not know if the user is using /media/fat/games/nes or /media/fat/nes or CIFS mount or usb HD. And, it is not theoretical, all these are issues opened to mbc. Obviously the logic of Main_MiSTer directory selection can be duplicated in the utilities; but this should be continuously synchronized with Main_MiSTer. E.g. if one day the order of the searched directory changes, all the utilities must be modified.

[2] Please note that such need came to me while working on the mbc utility, that lets you to launch roms from the command line. But I tried to make a proposal that will be useful in other scenarios too. For example, a script can be made that launch a core with a selection of "Favorites" games. Or one that shows the game organized in some smart way without touching the file system (by year? by game name? by region? by developer?)

[3] Well, actually I have a terrible memory, so I just remember that there is a list. When I have to know the exact items, I open file_io.cpp and search for order:.

[sorgelig]

The key point here is that you are making something not intended for MiSTer and regular users. Linux doesn't mean that MiSTer has to many different tasks including brewing the coffee... just because it can.

[pocomane]

No, the key point is that we want to use the MiSTer for a lot of things you are not interested to. And these are about to enjoy our retro-games, not making coffee. Given that the proposal is very simple, in code and in the concept, I hoped it could be added.

However, obviously, the last word is yours, since you are maintaining it. The most I can do will be to find some ugly and fragile workaround if you reject it.

[sorgelig]

According to your wish you want to hijack core's home dir. While it will work in your script, if user would like to open OSD to load a file, he will find a strange set of files, not what he expected to see. More over, if some file will be loaded, then it will be put in recent list, so resent list will be messed with files from different folders and probably next time file won't be in that temp folder. This is quite disturbing case when different kind scripts will do what they want instead of user.

This is definitely against the user experience.

[pocomane]

I wish to add a directory to an already present list of possible directories; you are free to call this "hijack", but it seems misused in this context to me.

They seem misused too "Strange" or "Not what he expected to see": if an user launch a script that changes the file list, he/she EXPECTS to see a new list. And the files are not "Strange", are the response to his/her action.

It is exactly what happen inserting/removing/switching an usb storage. File list changes. Recent list got invalidated [1]. Are the usb storage "Definitely against the user experience"?

Indeed, what I am proposing behaves exactly like /media/usb0 (that is the highest priority folder, currently). The only functional difference is that no one is trying to mount anything by default in /run/games, and that no disaster can be made acting on a tmpfs. [2]

---

[1] Actually, if the /run/games directory contains just symbolic links, I think the recent list should continue to be valid (supposing it is built through a realpath-like functions)

[2] In fact, a workaround I am considering, is just to bind-mounting anything I found on /media/usb0 in some other usbX directory, then mounting a tmpfs on usb0, then using that one to make the dirty stuff. Still better would be to use UnionFS, but I do not know if it works on MiSTer without modification to kernel or modules.

[sorgelig]

Some time ago i had an idea to have a virtual home dir in tmpfs, but i found that sometimes i need to navigate outside of home dir. Some cores may use compatible files so can navigate (temporarily) outsied of home dir. With virtual home it it will be impossible as it will be "sandboxed". Some day i will solve this puzzle.

[pocomane]

Maybe, union mounts could be a good solution also for your puzzle. I just compiled mergefs for MiSTer, and it seems basically to work. I will make some experiment in the next days.

Actually, I think that the in-kernel overlayfs would be a better choice, but it is not currently enabled, and last time I tried, I had some issue compiling the MiSTer kernel.