Feature-Request: add more information to the output of "tttool games"

[ToniMahagoni]

This is all I want for christmas... ;-)

At the moment "tttool games" prints only the known cough or rather implemented cough parts of a game. Looking at the same file with "tttool explain" may show some "-- unknown --" blocks related to the game. For analyzing of the game functionality it would be great to have a little indicator to see if there are unknown parts left. E.g. "unknown code detected" at the end of the game. Details would still be in the "tttool explain" output. This could also help for future use if more game types are added.

The definitely known parts should be listed with a useful description: u1 = rounds, the repeat-oid and bonus-game rounds of u2, u3 = the points needed for a rank in the main or bonus game, etc.

For the already discovered, but still unknown parts (e.g. u2 [or what is left] for games and u for subgames) it would help, to have the - as we assume - 16 bit values as single lines and number them from u1 to un. As some unknown variables are only available in gametype 6, the numbering may have holes for games of any other types. When the games have n unknown variables, the subgames should start with u(n+1), to have unique identifiers for every unknown element. This will blow up the output but makes it a lot easier to search a specific element in multiple files. Also numbering the playlists would help me a lot to have a quick overview. u3 should be renamed (maybe game-rank) and maybe could be grouped: for gametype 6: [[9,0][1,0,0,0,0,0,0,0]] or in two lines with different descriptions (main-rank, bonus-rank or whatever) for other gametypes: [9,8,7,6,5,4,3,2,1,0]

[nomeata]

E.g. "unknown code detected" at the end of the game.

this is actually tricky, because it is not clear when the game section actually ends. Sure, one can make an educated guess (i.e. everything until the next known thing), but that’s not easy. Is it too bad to just look at both the output of games and expain, i.e. in two windows?

The definitely known parts should be listed with a useful description:

Quite true! Remember: I have never myself looked into games. So you have to explain very carefully what output you want to see where. Assume some extra dumbness on my part while explaining, and I’ll implement it :-)

I’m having trouble following all your descriptions. Maybe you can paste the output as it is now for one good example, and then what you’d like to see instead?

[nomeata]

Or I’ll start to implement it based on what you wrote at https://github.com/ToniMahagoni/tip-toi-reveng/wiki/Games and you’ll test it and tell me if I did it wrong.

[ToniMahagoni]

Is it too bad to just look at both the output of games and expain, i.e. in two windows?

It is. I don't talk about single files. I talk about all files. I use a couple of batch files to print all explain/scripts/games into single textfiles. That makes searching for a special game-type very easy. And it would be easy to look for "unknown code detected" in all game-textfiles. A simple search in the explain-textfiles for "unknown code" would return all files because there's always something unknown in the header. I'll have a look, if I can manage it using some regex.

Maybe you can paste the output as it is now for one good example, and then what you’d like to see instead?

Or I’ll start to implement it based on what you wrote at https://github.com/ToniMahagoni/tip-toi-reveng/wiki/Games and you’ll test it and tell me if I did it wrong.

Which way do you prefer? I can live with both. I already added the pages to your project-wiki. Better take the information from there.

[nomeata]

I already added the pages to your project-wiki. Better take the information from there.

Thanks for the heads up. I think i can work with that, it’s quite precise (thanks!). I’m just not yet sure how best to map it to my data structures.

[ToniMahagoni]

I’m just not yet sure how best to map it to my data structures.

I don't know either and i don't really understand Haskell, so you're the better guy for the job. Maybe you should already think about how to export them into YAML. If there are no undiscovered (!=unknown) parts left for the games it should be possible to export them into YAML and repack the (untouched) file into a working GME-File. That would push me to the next level of game analyzing: Modify parameters and see what happens.

[nomeata]

Yes, roundtripping would be really nice! I’m optimistic that we’ll get there :-)

[ToniMahagoni]

:+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1:

[nomeata]

For Game 6 and 7, j points to a subgamelist or subgamelistlist. What is the precise format of these lists?

[ToniMahagoni]

For Game 6 and 7, j points to a subgamelist or subgamelistlist. What is the precise format of these lists?

AFAIR same as for playlist and playlistlist. The are already treated correctly. Just the naming is wrong.

[ToniMahagoni]

btw: thanks to @uli42 Without your great work I wouldn't even have an idea where to start. Maybe you could have a look at my additions?

[nomeata]

I’ve started to work on this on a new branch. I’m sure I have made various mistakes, so @ToniMahagoni, can you test it and report whatever is wrong or not as good as it could?

[ToniMahagoni]

"unkown" should be "unknown" :-) I'll run some tests later in the evening.

[uli42]

On Wed, Dec 23, 2015 at 7:15 PM, Toni Mahagoni notifications@github.com wrote:

btw: thanks to @uli42

You're welcome, but it was not me alone

Without your great work I wouldn't even have an idea where to start. Maybe you could have a look at my additions?

I have, and it looks good, but I cannot get into/check any details at the moment.

Uli

[uli42]

On Wed, Dec 23, 2015 at 6:39 PM, Joachim Breitner notifications@github.com wrote:

Yes, roundtripping would be really nice! I’m optimistic that we’ll get there :-)

At least for the non-binary games. OTOH hand: maybe the binary games are just blobs with a parameter block, so the could easily be preserved.

Uli

[ToniMahagoni]

Minor Bug: Some playlistlists are called playlists.

The output is much better, but "tttoool games" stops at gametype 16: tttool: No match in record selector gFinishPlayLists

Files: Englisch-Detektive Feuerwehr Flughafen

[nomeata]

Minor Bug: Some playlistlists are called playlists.

Not quite a bug. You can tell from the output whether its a list or a list of lists; I didn’t want to clutter the description with our feeble internal names :-)

tttoool games" stops at gametype 16:

fixed in 3345745.

[ToniMahagoni]

I didn’t want to clutter the description with our feeble internal names :-)

Well, there are still some "playlistlists" left in the output... :-D
In "tttool explain" replace "roud" by "round" and "specials" by "special".

I tried to fill some of the "unknown"-holes and found two more offsets to binaries: 0x00C8 and 0x00CC
The table right after the register inital values seems to belong to the binary games. Maybe you could mention the existance and the number of entries in the "tttool games" output.
I looked at "WWW Jahreszeiten" and there's only one game (type 0) contained, but the YAML-file shows G-commands for games 1,2,5,6,7,8,9,10. So it could be useful to have the binaries listed, too.
Not only game-related: There seem to be more than two special symbols. Right before every binary-table there are 4 unknown bytes and then again the 40 bytes list of special symbols. I didn't find any reference to this list and don't really know how to handle them.

[nomeata]

Thanks. Will have a look at it later. But don’t be shy to open new tickets even for very minor things; easier to keep track of than having one ticket with many different issues, some solved and some not.

[uli42]

On Mon, Dec 28, 2015 at 6:21 PM, Toni Mahagoni notifications@github.com wrote:

I tried to fill some of the "unknown"-holes and found two more offsets to binaries: 0x00C8 and 0x00CC

The table right after the register inital values seems to belong to the binary games. Maybe you could mention the existance and the number of entries in the "tttool games" output.

Yes, I vote for that, too.

I think the newer the version string in the gme file the more fields are filled/used in the header. We should document the version strings of the GMEs where a field has been seen, this might be helpful when generating own GMEs (although we have not found any evidence that the string is interpreted at all).

I looked at "WWW Jahreszeiten" and there's only one game (type 0) contained, but the YAML-file shows G-commands for games 1,2,5,6,7,8,9,10. So it could be useful to have the binaries listed, too.

Not only game-related: There seem to be more than two special symbols. Right before every binary-table there are 4 unknown bytes and then again the 40 bytes list of special symbols. I didn't find any reference to this list and don't really know how to handle them.

I suppose that is the input to the binary games.

Could these 4 bytes belong to the previous game? Maybe they are the checksum.

Also, while reading your enhanced doc I noticed that game types 3 and 5 are doing the same, are you aware of any difference between those two?

Uli

[ToniMahagoni]

I noticed that game types 3 and 5 are doing the same, are you aware of any difference between those two?

The difference between the two games in "WWW Bauernhof": On type 3 you are looking for n animal babies per round, on type 5 you are looking for 1 cereal per round. But if you look at the games 0 (Geräusche suchen) and 8 (Mäusesuchspiel), both should be possible with gametype 1. So I don't really know (yet!) what gametype 3 and 5 are for. But I'm pretty sure there are differences - we just have to find them.

[ToniMahagoni]

don’t be shy to open new tickets even for very minor things

Yup. My initial request has been adressed and if you correct the typos it could be merged with the master. Or do you want to keep the "issue-120" as an "experimental" branch? I can close this issue and open some more for the other little or bigger things.

How do you want me to handle this kind of stuff in the future? Add things to the wiki and then open an issue to get it build into the tttool?

[ToniMahagoni]

Could these 4 bytes belong to the previous game? Maybe they are the checksum. It appears before the first game. I guess it would help, to have all the binaries handlet correct in the first step. Not confirmed, but for 0x00A4 = 1 the specials symbols table has 4 unknown bytes in front and is followed by a binary table without another set of special symbols. Example from "WWW Komm mit auf die Baustelle":

At 0x0270DFE9 Size    32556: Header/Single binary 2/Main3202
   0x0270DFE0:                            CE 02 00 EA 41 50 50
   (skipping 2034 lines)
   0x02715F10: 6F 6E 00 00 00

At 0x02715F15 Size        4: -- unknown --
   0x02715F10:                B1 1D B9 2D

At 0x02715F19 Size        4: Header/specials symbols
   0x02715F10:                            8F 13 90 13

At 0x02715F1D Size       36: -- unknown --
   0x02715F10:                                        00 00 00
   0x02715F20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x02715F30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x02715F40: 00

At 0x02715F41 Size       32: Header/Single binary 1
   0x02715F40:    01 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x02715F50: 00 61 5F 71 02 30 7F 00 00 4D 61 69 6E 33 32 30
   0x02715F60: 31

[uli42]

On Mon, Dec 28, 2015 at 7:25 PM, Toni Mahagoni notifications@github.com wrote:

Could these 4 bytes belong to the previous game? Maybe they are the checksum. It appears before the first game.

Good point...

I guess it would help, to have all the binaries handlet correct in the first step.

of course...

Not confirmed, but for 0x00A4 = 1 the specials symbols table has 4 unknown bytes in front and is followed by a binary table without another set of special > At 0x02715F19 Size 4: Header/specials symbols 0x02715F10: 8F 13 90 13

Hmm, looks like two 16 bit values: 138f (5007) and 1390 (5008). These are probably not offsets as they would then point to single bytes. What happens when you manually change these values (hexeditor)

Uli

[nomeata]

Well, there are still some "playlistlists" left in the output... :-D

fixed in 8b239f7.

My initial request has been adressed and if you correct the typos it could be merged with the master. Or do you want to keep the "issue-120" as an "experimental" branch?

No, I’d rather have only one branch. I fixed the typos, so I’ll merge this and close this ticket right away.

How do you want me to handle this kind of stuff in the future? Add things to the wiki and then open an issue to get it build into the tttool?

That would be great. Ideally, I can use any short timestamp and immediatelly implement a bit of new information (or discuss one single bit), without reading through a large ticket like this one and trying to figure out what yet needs to be done.

[nomeata]

BTW, there are many playlistlsts, both in the games, but also in the welcome field. For the welcome field I mapped it to a flat list of media references when exporting, and when assembling I always generate a playlistlist with one playlist. Is there any reason to believe that this makes a difference?