Possibility to re-run `igir` commands, e.g. as in `igir_pocket_sync.sh`, to update collections❓

[fooness]

Maybe this feature is already built-in, please consider this to be a question instead.

Based on your igir_pocket_sync.sh script, I just created my first collection, if that’s the correct term we’d like to use here.

#!/bin/bash

set -euo pipefail

trap "printf \"\n-- %s -- \n\n\" \"rsync: interrupted\" >&2; exit 2;" INT TERM

dir="/Volumes/MISTERFPGA"

src="/Volumes/ARCHIVE"

if command -v igir >/dev/null; then
  igir copy extract test clean \
    --dat "${src}/No-Intro*.zip" \
    --input "${src}/No-Intro/" \
    --output "${dir}/games/{mister}/" \
    --dir-letter \
    --clean-exclude "${dir}/games/*/*.*" \
    --clean-exclude "${dir}/games/*/Palettes" \
    --no-bios \
    --no-unlicensed \
    --no-demo \
    --no-beta \
    --no-sample \
    --no-prototype \
    --no-test-roms \
    --no-aftermarket \
    --no-homebrew \
    --no-unverified \
    --no-bad \
    --single \
    --prefer-language EN \
    --prefer-region USA,WORLD,EUR,JPN \
    --prefer-revision-newer \
    --prefer-retail
fi

Would I be able to basically re-run that script, updating the created collection without again copying/extracting all files which are already present? I’m having something like rsync in mind, which basically compares the source and destination and only makes changes if something is either missing or different in the destination.

Note: I’m also very new to MiSTer FPGA, so maybe the arguments/parameters in my script could be improved.

--

Edit: It seems I’m having problems with the following exludes; using the first or second result in an error, using the last keeps some files and folders, but deletes other below games/*/Palettes/*; maybe my globbing is wrong in each and every one of those three lines?

--clean-exclude "${dir}/games/*/Palettes" \
--clean-exclude "${dir}/games/*/Palettes/" \
--clean-exclude "${dir}/games/*/Palettes/*" \

[emmercm]

Thank you for the feedback!

For your script, files in the --output directory won't be overwritten unless --overwrite is provided, so what you have should work as you expect. There isn't an rsync-like flag that only overwrites if different (e.g. filesize), but if you specify the igir test command then files that are not being overwritten will still be checked for accuracy.

Hmm, your globs look alright there, what error does it give? Could you give a sample file tree (e.g. tree -L 3 /Volumes/MISTERFPGA/games or similar). I'm wondering if the * actually needs to be ** or something like that.

[fooness]

For your script, files in the --output directory won't be overwritten unless --overwrite is provided, so what you have should work as you expect. There isn't an rsync-like flag that only overwrites if different (e.g. filesize), but if you specify the igir check command then files that are not being overwritten will still be checked for accuracy.

Thank you! I will study and try the igir check command.

Hmm, your globs look alright there, what error does it give? Could you give a sample file tree (e.g. tree -L 3 /Volumes/MISTERFPGA/games or similar).

The Palettes/ folder and its contents are downloaded via the update_all.sh script on MiSTer FPGA; please find below some sample content of some games/*/Palettes/ folders.

$ ls -F MISTERFPGA/games/*/Palettes

MISTERFPGA/games/Atari2600/Palettes:
'Atari 2600 Only'/   Trebor/

MISTERFPGA/games/Atari7800/Palettes:
'Atari 2600 Only'/   Trebor/

MISTERFPGA/games/Gameboy/Palettes:
Default/  Extra/  SGB/

MISTERFPGA/games/NES/Palettes:
 Composite.pal*   Emulator/    Grayscale.pal*  'Kitrinx34 High Saturation.pal'*  'NES Classic.pal'*   Nintendulator.pal*   PC-10.pal*   Real.pal*         Unsaturated/       YUV.pal*
'Console & PC'/   FCEUX.pal*   Kitrinx34.pal*   Miscellaneous/                    Nintendo/          'PC10 Better.pal'*    PVM.pal*    'Rockman 9.pal'*   Unsaturated.pal*

I'm wondering if the * actually needs to be ** or something like that.

Yeah, it seems that the proposed --clean-exclude "${dir}/games/*/Palettes/**" \—which I just tried—works indeed, i.e. none of the files listed above are deleted anymore; even though I’m not completely understanding the reasoning here; on the other hand, I also did not quite understand why with the other approaches, only some of those files or folders were deleted, but not all of them. Maybe some bash(?) weirdness.

[emmercm]

I completely misspoke above, I meant igir test and not igir check.

For the Bash weirdness - if the string is wrapped in double quotes then your shell won't do any glob expansion. Instead, igir will use micromatch to expand the globs. ${dir}/games/*/Palettes/** is what you want in order to select all files recursively, ${dir}/games/*/Palettes/* will only select files within the Palettes folder.

[fooness]

For the Bash weirdness - if the string is wrapped in double quotes then your shell won't do any glob expansion. Instead, igir will use micromatch to expand the globs. ${dir}/games/*/Palettes/** is what you want in order to select all files recursively, ${dir}/games/*/Palettes/* will only select files within the Palettes folder.

Thank you very much for elaborating this in detail!

[github-actions[bot]]

:lock: Inactive issue lock

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.