search

Mutagen-Modding / Spriggit

A tool to facilitate converting Bethesda plugin files to a text based format that can be stored in Git
GNU General Public License v3.0
93 stars 11 forks source link

Preprocessor "pipeline" #48

Open Noggog opened 6 months ago

Noggog commented 6 months ago

For example, CK randomizes script properties. Some people might want that ordered, and so a preprocessor cmd could be added that did the job before flowing into the spriggit call.

Would be nice to have this in the UI somehow to where it could run these for the user as part of the single click

Noggog commented 3 months ago

Not yet, but I'll see if i can whip up something soon

Noggog commented 3 months ago

Alright, did some work on a property sorter in 2c2548ffc4863eca6be78c1ece593a5c9ee52a6d

Still need to finish the "pipeline" itself

Noggog commented 3 months ago

Is there an example on how to use it?

I'll have to push a new version of spriggit for it to hit, but it would be something like: .\Spriggit-CLI.exe sort-script-properties -i "Path/To/Mod.esp" -o "Temp/Path/To/Mod.esp" -g SkyrimSE Where it would output the sorted mod to the temp path, which you could then pass to the serialize command as a followup

the 2nd half of the ticket mentions a nicety to have this in the GUI, and it seems much more complicated

Yeah, the pipeline would be a convenience feature for the UI to abstract away running all the commands one by one like you would in a bash script or whatever

Why not just emplace them into a sorted set or something? That way when they're deserialized, they will be alphabetized

Spriggit serializes into/from the Mutagen record definitions, which try to allow a 1:1 representation of what's in the binary file. If the mutagen definitions were to be a Set, then it would be impossible for it to represent some binary files that had duplicate names, and be impossible to represent a file that was in an odd order. These definitions are used for a lot more than just Spriggit, so they need the ability/power to have that granularity.

So the idea of there being a sorted set is something we need to add on top of the more "raw" definitions. Lots of approaches, but the cleanest imo is to just process the raw files to be how spriggit would prefer them. Making a whole 2nd wrapper of new definitions, /w just one field being a set instead of a list seems unnecessary. There's gonna be a lot of small tweaks spriggit users will want, so just toss it on the pile of something to process before doing a seriaization

Noggog commented 3 months ago

@gir489returns sorry about that! Pull dev and try again

Noggog commented 3 months ago

It wants a "normal" mod name that can be parsed into a ModKey. Eg XYZ.esp. Add a .esp to your output path

Noggog commented 3 months ago

it's sorting it and then I desearlize the sorted ESP?

yeah, the idea is you have an unsorted esp, you run this to get a sorted one, and serialize that instead

Ah, i can probably improve the logic to be less picky for the next version, but for now, let's do this: -o D:\Backup\Stuff\Starfield\mods\Cheat Terminal\Sorted\CheatTerminal.esp Where Sorted is a folder now, and the name of the mod remains the same

Noggog commented 3 months ago

Aight, pull dev and try again:

Noggog commented 3 months ago

Cool! I'll be working to get this more integrated overall, but hopefully that keeps you going in the meantime

Noggog commented 3 months ago

Are these within the VirtualMachineAdapter of the terminal menu? Edit: Maybe reproduced, give me a few

Noggog commented 3 months ago

Didn't reproduce. Red herring.

Here's a unit test flexing the sort of Terminal Menu in starfield: https://github.com/Mutagen-Modding/Spriggit/commit/efb70b19ab595fafe6de169ea973df7e8b87d913#diff-c4956256eb7a305e70915a92051ac80b6bb4e1f5aacc43e9af84263b0e58e87fR107

Is it perhaps something else at play?
The left side doesn't look sorted (Weapon should be after Outpost), so it makes sense that a sort pass would change it from there? Maybe the test battery got crossed somehow. I'd maybe try again and see if it keeps failing to sort?

Noggog commented 2 months ago

If I sort the ESP, it claims those same Objects now circularly belong to my mod (CheatTerminal.ESP).

What command are you running for the sort CLI? Are you including the DataFolder parameter? That'll be required for Starfield (almost for every command nowadays, until #81 is done)

If omitted, the formID indices can be scrambled (One of starfield's major design flaws)

Noggog commented 2 months ago

alright, add -d "Path/To/Data/Folder" and see if it does better. I'll adjust the script to scream a bit more if missing for starfield

Noggog commented 2 months ago

Yeah, UI needs a repass before 1.0 #85

The CLI and/or logs will give the detail you're hoping for, at the moment. Can you upload Current.log with the failure included?

Noggog commented 2 months ago

Opening Spriggit v0.26.0+a3c9c97e9

Let's grab the 0.27 versions of the CLI/UI and give it another spin

Noggog commented 2 months ago

Can you upload your latest CheatTerminal.esp for me to look at? I've only got the one /w Starfield.esm as a master

Noggog commented 2 months ago

Alright, pushed a fix to the sort CLI on dev branch. Was missing a parameter pass, so the data folder wasn't getting considered in one of the steps

Noggog commented 2 months ago

Cool. Yeah, i think we just need to get in the weeds on the details.. as the sorting code is hand crafted, so could be areas that I missed.

I think i got the fragments that you were showing in your previous pictures. Grab the latest cli from dev and give it another spin. Let me know if there's any more to look at

Noggog commented 2 months ago

Thanks for the back and forth!