Run packs through fvtt-cli - [merged] #387

Closed asacolips closed 7 months ago

In GitLab by @ben on Feb 7, 2024, 12:56

Merges ben/fvtt-cli -> master

(This is based on !373, so there's an extra commit in the stack. I'm happy to rebase if/when that gets merged or rejected.)

This pulls in @foundryvtt/foundryvtt-cli as a dependency so it could be run as part of a release pipeline, and converts the extractPacks task to use it.

Here's the process to get it to work, because fvtt relies on a complete v11-style database to work:

npm run build and make sure your symlinks are in place.
Open Foundry, unlock one of the system compendia, make a change and then undo it. This triggers Foundry to do the database conversion to all the packs.
In foundry, return to setup.
npm run extractPacks

The result is what you see. Looks like the YAML is mostly the same, except for filenames, some _key entries, and string delimiters. It looks like the YAML is compatible with the current build pipeline. Gitlab makes it hard to see the code changes with all the YAML diff noise, so it's probably useful to look at the commits one by one.

In GitLab by @ben on Feb 7, 2024, 13:57

added 2 commits

d94def17 - Use gulp-shell to extract packs
130c2753 - Use gulp task to extract the rest

Compare with previous version

In GitLab by @ben on Feb 7, 2024, 14:01

added 2 commits

1c5331ef - Use gulp-shell to extract packs
646aa7ae - Convert YAML using gulp task

Compare with previous version

In GitLab by @ben on Feb 7, 2024, 16:56

added 1 commit

68bc18ca - Add documentation for setup steps

Compare with previous version

In GitLab by @Asacolips on Feb 7, 2024, 19:02

Commented on gulpfile.js line 69

We'll also need to update the compilePacks() task as well given that the formats on extracted packs have changed. I took a pass at it on my local but couldn't quite it to work. Here's a quick snippet of where I made it to:

/**
 * Gulp Task: Compile packs from the yaml source content to .db files.
 */
function compilePacks() {
  return gulp.src(`${PACK_SRC}/*/`)
    .pipe(shell([
      `fvtt package pack <%= file.stem %> --in=${PACK_SRC}/<%= file.stem %> --out=${PACK_DEST} --id archmage --type System`
    ]));
}

That successfully compiles the compendium databases, but when I open them in Foundry, they all appear to be blank. Not sure what I'm missing unless something didn't come through correctly during the extract process.

In GitLab by @ben on Feb 7, 2024, 19:20

Commented on gulpfile.js line 69

I think this is because after the extractPacks step, you've got two copies of all the database things - one in the *.db files, one in the v11-style directories. If those directories are around, they'll take precedence.

So if you do a rm -rf dist/packs/* before npm run compilePacks, everything seems to come out okay. I can replicate that here.

In GitLab by @ben on Feb 7, 2024, 19:22

Commented on gulpfile.js line 69

There's a bit more work to be done to replace mentions of the *.db files with the v11-style directories. I didn't want to get into that rabbit hole here, probably better in a separate PR.

In GitLab by @Asacolips on Feb 8, 2024, 13:57

Commented on gulpfile.js line 69

Hmm, that's not the case for me. I was running npm run compilePacks, which also runs the cleanPacks gulp task and clears out the compiled packs directory before recompiling. I did update my manifest as well to point to the directory versions of the compendiums, but no luck there.

I'll leave this open for now, as IMO both steps should be included in the same MR. The existing commands are capable of both compiling YAML into databases and extracting nedb databases back into YAML, and once we've swapped to using foundry cli, I'd like to use it for both steps for consistency.

I'll make another attempt at getting the packs to recompile and load in Foundry on my end tonight.

In GitLab by @ben on Feb 8, 2024, 15:40

added 3 commits

8183773b - Use fvtt to compile packs also
ce3d08b7 - Update system.json to refer to v11-style packs
d671a4b3 - Clean up gulpfile

Compare with previous version

In GitLab by @ben on Feb 8, 2024, 15:51

@Asacolips I pushed a few more commits that use fvtt for both directions. Again, inspect commit-by-commit to make GitLab's diff viewer not choke.

❯ npm run compilePacks

> archmage@1.0.0 compilePacks
> gulp compilePacks

[13:37:34] Using gulpfile ~/Code/archmage/gulpfile.js
[13:37:34] Starting 'compilePacks'...
[13:37:34] Starting 'cleanPacks'...
[13:37:34] Finished 'cleanPacks' after 72 ms
[13:37:34] Starting 'compilePacks'...
[classic-level] Packing "/Users/bstraub/Code/archmage/src/packs/src/bard" into "/Users/bstraub/Library/Application Support/FoundryVTT_11/Data/systems/archmage/packs/bard"
Packed hl4HScjo9l85WYiV (Arrow of Verse)
Packed bPFcpjwikdFWTEJf (Balladeer)
Packed N6tQPIG5SlFCboph (Bardic Songs)
Packed lSLxn7FjgFd21MGs (Bardic Spells)
Packed jSUHFluSNawW6S6y (Battle Chant)
Packed auCqz5aw5hblXJGW (Battle Cries)
Packed 2xR0LumEmGZIjcS7 (Battle Skald)
Packed qsrPWt4K6pFPbXDU (Befuddle)
Packed TzwirbOiXCeUMhzd (Charm Person)
Packed ZCqkhUNXPz8uG3VQ (Discombobulate)
Packed a03ky67mIaBVFnJT (Hang Tough!)
Packed rn9E7G8rJ5I2hKVd (Inspire Legends)
[…]
Packed YWUMZ29u82zpOHMi (Zealous Warbanner)
Packed gHpvxvGb5DIUPhgE (Zombie Beast)
Packed 0pKCCcW92wqXpUJP (Zombie Shuffler)
Packed luBNIwOxHVc0GsOZ (Zombie of the Silver Rose)
[13:37:36] Finished 'compilePacks' after 1.7 s
[13:37:36] Finished 'compilePacks' after 1.78 s

asacolips-projects / 13th-age

Run packs through fvtt-cli - [merged] #387