Support PQT2

[mganss]

.EXT files have a section called PQT2. It sounds like it has to do with quantization, but from the data I can't tell what it does 🤷‍♂

[brunchboy]

Yes, I think mention this in the analysis article (although perhaps in dysentery’s—and discussing PQTZ—not crate digger’s). I believe it is an index supporting rapid jumping to a particular time in a variable-bitrate audio file. I needed to implement a similar index for MP3 files when I built my own high-performance home audio player around 2000. I haven’t seen any need to try to figure out the details, since I don’t plan to support playing audio (among other reasons, to avoid having Pioneer feel this software is a threat to their business model). But if you do figure it out, a pull request would be welcome. Please also update the analysis article as well as the Kaitai struct if you do.

For anyone tackling this, I would start by figuring out PQTZ, which will be a simpler/smaller format in some way, which worked with pre-nxs2 hardware, before moving on to PQT2.

[mganss]

PQTZ is the beat grid which is already supported.

[brunchboy]

Oh derp, I completely forgot we had figured that out! I was remembering the state of things before Crate Digger. Hmm, interesting… I wonder what new information is in PQT2? I am starting to really wish I had a newer player, but simply can’t justify spending the money, as we are not doing any paying shows any more.

[brunchboy]

It was the PVBR tag I was thinking of in my original comment, it turns out.

Say, @mganss how stable is the PSSI analysis looking at this point? I am preparing to cut a release so I can release Beat Link and Beat Link Trigger with the new nexus-2 enhanced cue support. Might you be willing to add a section describing the PSSI tag to the analysis article? Or, if you have no TeX setup and don’t want to dig into learning that, would you like me to make an attempt, and have you correct any mistakes I make?

Finally, I have been wondering how you find the right files to open when looking for these tags, given that you are working with performance mode. Does rekordbox keep a .pdb file with the entire collection on the computer somewhere that you can open and parse with Crate Digger?

[mganss]

I've tested with ~350 input files using C# code generated from the .ksy. This turned up the additional phrase style I added support for in the latest PR. So I'm pretty confident that it's stable, but in absence of a spec I guess you can never be 100% sure :)

It's been more than 20 years since I last touched LaTeX and I don't have a setup ready so I'd be glad if we could go with option B 😳

Rekordbox maintains a .DAT and a .EXT file for all tracks in the collection. On Windows, these are located at %AppData%\Pioneer\rekordbox\PIONEER\USBANLZ\Pxxx\xxxxxxxx\ANLZ0000.{DAT,EXT}, e.g. %AppData%\Pioneer\rekordbox\PIONEER\USBANLZ\P02B\00003DBF\ANLZ0000.EXT. I simply grep through a hexdump of all the files to find the corresponding files for a specific track.

[brunchboy]

Indeed, it is tragic that I am creating the closest things we have to specs for these file and network formats! 😢 But that is great, your testing sounds excellent, this will be more stable than I usually manage to achieve for a given release.

I’m a little surprised that you even had LaTeX experience 20 years ago… and that is probably how long it had been since I used LaTeX when I started working on the DJ Link packet analysis document, and found to my surprise that there was still nothing better for drawing diagrams of network packets and file structures, so I had to re-learn how to use it. I am trying to work on an extension to Asciidoctor that will let me port my diagrams to a more modern environment, so they can be integrated nicely with the attractive and cross-referenced documentation sites I build using Antora, but it’s been a long time since I used Ruby, and the Asciidoctor API is not yet fully documented. The community is great but extremely busy with more important tasks, so I do not pester them to help me, and I keep having other things to spend my open-source time on, like these new releases.

Thank you for offering to review my proposed documentation of the PSSI format, I hope to write something up Thursday evening or thereabouts.

[brunchboy]

@mganss I started working on the Song Structure description, but realized it will be difficult without an example .EXT file with a PSSI tag in it to look at while writing. Is it possible for you to share one with me? Also, for now I have used your Github profile URL to credit you, but is there a better URL that you would prefer that I use?