search

zamaudio / ptformat

Free software file format parser for Avid ProTools sessions
GNU Lesser General Public License v2.1
72 stars 18 forks source link

ptf 7.x fixes #12

Closed mauritslamers closed 5 years ago

mauritslamers commented 5 years ago

Fixes for 7.x session compatibility

zamaudio commented 5 years ago

Interesting, does this allow the session(s) to open? I will check the sessions you uploaded soon.

mauritslamers commented 5 years ago

It turns out that many "flags" (such as 0x5a) in 7.1 sessions are 3 bytes (0x5a0002) where in 7.3 sessions they are down to two bytes (0x5a02). For example, commit 4bc5fa6 fixes reading the session sample rate. I have a few fixes coming up, which deal with the difference between 7.1 and 7.3 sessions where the type tag after a file name in the audio list is reversed (EVAW vs WAVE).

I do have still trouble finding the right metadata for the audio files and regions list.

Edit: I didn't check the header files before, just ran into a few structs. I will try to see whether to add a few, such as the audio files list.

Edit2: it is very possible that differences like the one described above are caused by one session being created on a PPC machine.

That is why I wonder why you didn't use C style structs to catch the different data structures. I would expect that to make reading and specifying the different session formats easier.

For example, I noticed that after the marker "Audio Files" there are 4 zero bytes, then a marker 0x02 after which there are 4 zero bytes, then a number of 4 bytes, then the string of the audio file. This pattern continues, only after the last audio file, the marker is 0x00 which marks the end of the audio files list. This kind of pattern would be rather easy to capture with a struct. The difference of the two bytes or three bytes marker (0x5a0002 vs 0x5a02) would then simply implemented through either an unsigned int or a unsigned byte. C++ isn't really my language though, so I don't know what the best way would be to implement this.

mauritslamers commented 5 years ago

I found some patterns while comparing the files I have (next to the examples I already posted) and the sessions provided in the bins folder.

The format seems to be rather straightforward: there is a header, which starts with this sort of bitmask and which seems to include a flag for endian-ness. After the header there is a series of blocks, each starting with a marker (0x5a) followed by something that looks like a block type (0x00 01 for example) which is then followed by the length of that block as a 32 bit integer. It looks like blocks can be nested. All strings (product name, file names, directories) are immediately preceded by a 32 bit integer indicating the length of the string that follows.

mauritslamers commented 5 years ago

Additional information: each block as described above starts with 0x5a, then a 16bit int which perhaps might describe a block layout type. Following is the length of the data segment (32bit int). The most important bit however is the next 16 bit number, as it seems to describe the actual content type. 0x2810 (LE) is the block describing the session sample rate for example.

zamaudio commented 5 years ago

I'm going to post your findings on the wiki here: https://github.com/zamaudio/ptformat/wiki

zamaudio commented 5 years ago

You're right, the code does need refactoring to use structs, but I didn't work out what the 0x5a etc markers were exactly except I use them to locate certain blocks.

zamaudio commented 5 years ago

@mauritslamers I'm glad you have taken up this challenge! Staring at hexdumps is more fun when more than one person is involved :+1:

zamaudio commented 5 years ago

By the way, Ardour already has this code built in and imports PT sessions. (Nightly builds)

mauritslamers commented 5 years ago

In order to make hexdumps even more palatable, I rewrote my NodeJS code to completely parse all nested blocks as well, in order to create a more readable representation which also shows the structure of the nesting. See https://github.com/zamaudio/ptformat/wiki/Blocks-structure-as-NodeJS

zamaudio commented 5 years ago

Woot!

mauritslamers commented 5 years ago

Different representation: https://github.com/zamaudio/ptformat/wiki/Block-structure-in-HexView

zamaudio commented 5 years ago

@mauritslamers I took your work on blocks and rewrote ptformat. See pt-blocks branch. Can you help me figure out why travis CI is failing on the regression tests, but they work locally when I run make && ./ptreg ? See https://travis-ci.org/zamaudio/ptformat/builds/547214200

zamaudio commented 5 years ago

Never mind, it was an uninitialised value bug...

zamaudio commented 5 years ago

@mauritslamers Thank you for your work on this!