Closed Hejsil closed 5 years ago
I've read more about the "unix philosophy" and I think it might be a good way to take things. The api was getting too complicated, and I think this is the way to keep complexity down while pursuing #19.
Let's say that we have the program mloadgen3
, which will output a special Record-Jar Format that looks something like this:
# This is a comment. Comments start with '#' and take up the whole line. You
# cannot have comment in the middel of a line:
# `field: value # This is not a valid comment`
# Each file will start with a global header that lists metadata about the rom
# which the file was generated from.
gen: 3
gamecode: BPEE
%%
# After the global header comes to a list of sections. Each section has its
# own header which will have two fields.
# The "kind" field gives information about the type of items the section
# contains. It is up to each tool that consumes the header to interpret
# each item based on this field.
kind: Species
# The "len" field specifies how many items this section contains.
len: 2
%%
# After the section header all of the sections items will be listed
name: "Bulbasaur"
hp: 45
attack: 49
defense: 49
sp.attack: 65
sp.def: 65
speed: 45
%%
name: "Ivysaur"
hp: 60
attack: 62
defense: 63
sp.attack: 80
sp.def: 80
speed: 60
%%
# After all items have been listed a new section can begin
All loadgenx
programs will follow this format. We can then have a consumer program called mrandstats
which would read this format and output a modified version:
gen: 3
gamecode: BPEE
%%
kind: Species
len: 2
%%
name: "Bulbasaur"
hp: 54
attack: 66
defense: 92
sp.attack: 22
sp.def: 41
speed: 167
%%
name: "Ivysaur"
hp: 111
attack: 29
defense: 84
sp.attack: 65
sp.def: 37
speed: 59
%%
The only thing mrandstats
expects, is that the input contains a section called "Species", and that each item in this section have ["hp", "attack", "defense", "sp.attack", "sp.def", "speed"] fields. Everything else is just written to stdout without modification. This allows different programs to output new fields to section items if they desire.
Then, when we want to make this back into a rom, we run a program called mapplygenx
, which will take a rom file as an argument an read our custom format from stdin. So we now have something like this:
mloadgen3 firered.gba | mrandstats --seed 200 | mapplygen3 firered.gba -o firered.randomized.gba
As for any binary data, we could do something like this:
%%
name: "Ivysaur"
sprite: /path/to/sprite
%%
And then mloadgenx
just have to output binary data to files somewhere on disk (probably in tmp
unless otherwise specified).
Also, #21 becomes trivial. All we have to do is write a GUI that gives you a list of all "consumers" inside same folder, and then the user can just list what they want to happen to the rom:
+---------------------------------------------------------+
| |
| +---------------------+--+ |
| |/path/to/rom | | |
| +---------------------+--+ |
| |
| +---------------------+--+ +----------------------+ |
| |/path/to/out | | | Randomize!!! | |
| +---------------------+--+ +----------------------+ |
| |
| |
| Options: |
| +------------------------+ +----------------------+ |
| | mrandstats | | mrandtypes | |
| | mrandtypes | | | |
| | mrandparties | | | |
| | | | | |
| | | +-+ | | |
| | | |+| | | |
| | | +-+ | | |
| | | | | |
| | | +-+ | | |
| | | |-| | | |
| | | +-+ | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| +------------------------+ +----------------------+ |
+---------------------------------------------------------+
This UI will, in theory, never have to be updated, as the logic of randomizing is all done by the underlying programs. When the user wants to update, they just download the new versions of the randomizer programs and throw it in the folder. The UI could even do this for the user.
The only thing that I need to decide on is the format. I realize that the format I mentioned above might actually not work out, as we have arrays of arrays data sometimes (zones, where each zone has wild pokemons). I'll list some formats here as I look more into this:
Hmmm, or maybe we should just give up on the line-oriented format, as our data is highly structured. We could probably do a json streaming format.
@tgschultz had this to say on the Zig IRC:
15:03 <MajorLag> why serialize to a text format instead of a custom binary format? You can
have a switch to render the output human readable. This would avoid the overhead cost of
parsing it and rendering it at every stage in normal operation.
15:06 <MajorLag> Actually, don't even do a switch, just make a program that renders the
input data and pipe the output to it.
The only benefit of a binary format is that we don't have to parse the input (some serialization and deserialization will always be needed).
Here are the things that a text format would allow:
bin -> text -> bin
could accomplish thisHere's another format that I dreamed up:
pokemons[0].name = "Bulbasaur"
pokemons[0].hp = 45
pokemons[0].attack = 49
pokemons[0].defense = 49
pokemons[0].sp_attack = 65
pokemons[0].sp_def = 65
pokemons[0].speed = 45
pokemons[1].name = "Ivysaur"
pokemons[1].hp = 60
pokemons[1].attack = 62
pokemons[1].defense = 63
pokemons[1].sp_attack = 80
pokemons[1].sp_def = 80
pokemons[1].speed = 60
zones[0].wild.land[0].species = 0
zones[0].wild.land[0].min_level = 2
zones[0].wild.land[0].max_level = 3
zones[0].wild.land[1].species = 1
zones[0].wild.land[1].min_level = 3
zones[0].wild.land[1].max_level = 4
Pros:
sed
and grep
:
❯ grep -E 'pokemons\[[0-9]+\]\.(name|hp)' test.t
pokemons[0].name = "Bulbasaur"
pokemons[0].hp = 45
pokemons[1].name = "Ivysaur"
pokemons[1].hp = 60
Cons:
I don't think parsing is gonna be a big deal. Here are 3 shell scripts and their times:
#sed1.sh
cat test.t \
| sed 's/pokemons\[\([0-9]*\)\]\.attack.*/pokemons[\1].attack = 2/' \
> /dev/null
#sed2.sh
cat test.t \
| sed 's/pokemons\[\([0-9]*\)\]\.attack.*/pokemons[\1].attack = 2/' \
| sed 's/pokemons\[\([0-9]*\)\]\.defense.*/pokemons[\1].defense = 3/' \
> /dev/null
#sed3.sh
cat test.t \
| sed 's/pokemons\[\([0-9]*\)\]\.attack.*/pokemons[\1].attack = 2/' \
| sed 's/pokemons\[\([0-9]*\)\]\.defense.*/pokemons[\1].defense = 3/' \
| sed 's/pokemons\[\([0-9]*\)\]\.sp_attack.*/pokemons[\1].sp_attack = 4/' \
> /dev/null
❯ time sh sed1.sh
sh sed1.sh 0,40s user 0,00s system 104% cpu 0,384 total
❯ time sh sed2.sh
sh sed2.sh 0,87s user 0,04s system 201% cpu 0,449 total
❯ time sh sed3.sh
sh sed3.sh 1,38s user 0,08s system 288% cpu 0,506 total
The test.t
file is 977761 lines (24M).
Started the effort. New repos are gonna be located at https://github.com/TM35-Metronome
It could be cool, if we could somehow have a bunch of smaller tools that we could pipe together to randomize (or just modify) a Pokémon rom:
The idea here is, that all roms have a lot in common in terms of data, so we could have our own IR of what a Pokemon game looks like, that we could then load and reapply to roms:
We then just pipe this IR between programs to modify roms.
Pros
Cons
Challenges