Reading/writing to BWF converted to FLAC?

[ktmf01]

Recently I've been busy improving the --keep-foreign-metadata feature of the flac command line tool. As I understand it, it is used by some people to compress BWF files including all metadata. As it seems BWF MetaEdit is a very valuable tool to work with BWF data, would it be considered valuable to work on adding support for reading/writing BWF metadata directly to FLAC?

I was thinking of implementing this (contributing a PR), but not without first seeking some approval. After all, the functionality of this tool suggest a high level of standardisation, and FLAC isn't a standard yet of course.

Considering the difficulty of implementing and maintaining this, I don't think FLAC's way of storing metadata is very complicated. In fact, it is quite close to how RIFF works, but the ordering is a little more strict. I think the biggest hurdle here is that while a new chunk can just be concatenated to a RIFF file, FLAC mandates all metadata to be at the start of a file, meaning all data has to be moved to make room for a new chunk (if no padding is available for use)

[dericed]

Hi @ktmf01, personally I'm interested in this and suggested adding it to the scope as well as one point, but the goal of the project was admittedly fully focused on BWF. Since BWF is lack on some of the chunk ordering, we could make the metadata edits go faster by voided chunks at the header when we had to write larger chunks at the end. But is there a tool already existing for editing metadata in FLAC (the flac tags plus foreign metadata)? I suspect that adding this feature in BWF MetaEdit may be limited if it was just a foreign metadata editor, while the FLAC metadata components would need to be accessed elsewhere?

[ktmf01]

To my knowledge there are currently no applications capable of working with the foreign metadata except the flac command-line tool, which can make and restore them. The metaflac command line tool can be used to inspect them, but the result of that is worse than simply using a hex editor.

I also see the problem with possible discrepancies in different kinds of metadata. However, this is a possibility with WAV too of course: some applications write ID3 data to WAV in a id3 chunk, which BWF MetaEdit doesn't read either I presume. Still, FLACs metadata system is much more widely accepted, so this problem will be more obvious.

Superficial (and easy to maintain) support could be achieved by simply bundling BWF MetaEdit with the latest flac tool and automatically decoding to a BWF file for editing. Editing can happen on the decoded file, and updating the FLAC file can happen through the metaflac tool without reencoding the whole file. metaflac doesn't have that capability yet, but it is already pretty high on my TODO list. [edit: The obvious disadvantage of that is that decoding might take some time. FLAC decodes extremely fast (in fact, 30% of decoding CPU load is MD5 sum calculation) but it creates a lot of harddrive activity]

[JeromeMartinez]

As it seems BWF MetaEdit is a very valuable tool to work with BWF data, would it be considered valuable to work on adding support for reading/writing BWF metadata directly to FLAC?

It is valuable but value is difficult to estimate due to chicken and egg issue, current BWF MetaEdit users like uncompressed content. But it is interesting to suggest to use FLAC without losing metadata, as we do with RAWcooked (FLAC by default, PCM in option), and showing that we can still edit them.

I was thinking of implementing this (contributing a PR), but not without first seeking some approval.

If you are not disappointed if it is not used immediately, go ahead :).

In fact, it is quite close to how RIFF works, but the ordering is a little more strict.

There may be an issue there, IIUC it is not possible to put metadata at the end of the file so any expansion of metadata requires full rewrite of the file. BWF MetaEdit supports both rewrite and append to the end, so with FLAC it would be only one method supported.

some applications write ID3 data to WAV in a id3 chunk, which BWF MetaEdit doesn't read either I presume

They are considered opaque and kept as is only.

Still, FLACs metadata system is much more widely accepted, so this problem will be more obvious.

Later it could be a FLAC metadata editor :), but as a first step just working on RIFF tags is fine.

Superficial (and easy to maintain) support could be achieved by simply bundling BWF MetaEdit with the latest flac tool and automatically decoding to a BWF file for editing.

It would help for some automation but slow, I was thinking more on implementing a basic reader/writer of FLAC header without decompression/compression. IMO it would be useful to have a FLAC support only if there is no decompression/compression, for arguing that editing a FLAC file is (nearly) same as editing a WAV file. But a lot more work, I know :(.

Note: I need to read RIFF tags in FLAC, in MediaInfo, so people can see that the FLAC file has the BWF metadata.

[ktmf01]

I've been studying the internals of BWFMetaEdit, and I think the best (least intrusive) way to do this is by adding a wrapper layer between ZenLib and BWFMetaEdit. If a plain BWF file is read, the wrapper will pass data through to/from ZenLib directly, if it reads FLAC it should decode the metadata to memory op opening the file and decode the audio on-the-fly (only if MD5 generation/verification is requested)

I have one question though: is adding a dependency on libFLAC possible/desirable/permitted, or should this feature be self contained?

[JeromeMartinez]

is adding a dependency on libFLAC possible/desirable/permitted, or should this feature be self contained?

it is acceptable to have the dependency. With dlopen if not too difficult, as it is not a feature everyone will use, no need to load the lib if not a FLAC file.

Thank you for your interest and good luck, the code is a bit messy (me hiding...)

[JeromeMartinez]

BTW, maybe longer term, but a button "Store in FLAC" (and also "Revert to WAV") as an advertisement for FLAC storage may be nice, for hinting people that they could save storage space and still get their exact WAV file if needed (similar to how RAWcooked does).

[ktmf01]

After more digging and a bit of planning, it seems better to not rely on integrating libraries but instead using the flac and metaflac binaries directly. So, not using dlopen or something similar, but calling the binaries. That would also make it much easier to add a 'convert to FLAC' and 'revert to WAV' feature. I'm currently working on adding features to metaflac specifically useful to modify stored RIFF chunks. The flow of editing would be as follows:

1. BWFMetaEdit is asked to open a FLAC file
2. metaflac is called to export all stored RIFF chunks, which it sends over a pipe to BWFMetaEdit
3. When BWFMetaEdit is asked to generate or verify a MD5 sum, it runs flac to test the file instead, and on OK runs metaflac --show-md5sum to get the MD5 sum (which has just been verified by flac). As far as I know the MD5 computation method of flac does not differ from BWFMetaEdit
4. Write back of the RIFF chunks is handled by metaflac

Would this pose problems with packaging? The binaries are licensed under GPL, I can't change that of course. The linux packages can simply add a 'recommend' to install flac and metaflac, for Windows and MacOS I would think the binaries should be included.

[JeromeMartinez]

The flow of editing would be as follows

Let's start with that.

Would this pose problems with packaging?

As there is no lib link, it is fine.