Native Instruments (Traktor) Format

[kartguru]

Hi, I found 'The Easton Ellises - Falcon 69.stem.mp4' didn't load up as a Stems file in Traktor. Was that an intended output for stempeg?

[faroit]

hi @kartguru thanks for this pointer. Actually I don't have access to Traktor so I can't test this. Your help would be appreciated here, as the intended output should be compatible with NI stems.

Do you get an error message? can you try to rename to The Easton Ellises - Falcon 69.stem.m4a ?

[kartguru]

No probs @faroit.

I can load it into Traktor as m4a or mp4 but am unable to load it as a stem track. It plays just doesn't allow the manipulating of the individual stems. I'm not sure if there are other uses for Stem files so just mentioning my experience with Traktor.

In the dev toolkit you can read about the requirements of the *.stem.mp4 file. "The Stem File format mostly relies upon already existing box types and introduces only one new box called stem".

https://github.com/axeldelafosse/stemgen uses the toolkit to create stems.

You can download the trial of Traktor pretty easily. The demo allows for 30 mins of play time before you need to exit out.

[faroit]

@kartguru yes I know about the NI "dev-kit" which is actually just a wrapper around mp4box but this would come with extensive additional requirements since mp4box is not easily installable with conda or other common package managers. Therefore I would like to stick with ffmpeg if possible.

The real issue here now is how to create compatible files with valid metadata just using ffmpeg.

Maybe @axeldelafosse can help here as he does seem to have more experience with NIs mp4box interface. I would love to join forces here ;-)

[axeldelafosse]

Hey @faroit -- definitely agree with you, let's find a way to make it work with ffmpeg. Love this approach. I'll have a look!

[faroit]

@axeldelafosse It would be interesting to know if its just a metadata issue. Because in that case we would just need mutagen which is easier to install. So we could use ffmpeg+mutagen to create the stem file.

Maybe just comment out the metadata writing and see if traktor can still open the file...

https://github.com/axeldelafosse/stemgen/blob/04801180a568daa9b68240a848a486b6730a2174/ni-stem/_internal.py#L154-L188

[axeldelafosse]

I did some testing. Yes, it is a metadata issue. But it's not the tags saved with mutagen. Seems obvious now, but it's the stems metadata that makes the file playable as a Stem file in Traktor, for example:

[
        {"name": "Drums" , "color" : "#FF0000"},
        {"name": "Bass"  , "color" : "#00FF00"},
        {"name": "Synths", "color" : "#FFFF00"},
        {"name": "Other" , "color" : "#0000FF"}
]

Here's the metadata I'm using (they are mapped with spleeter stems, respect the Stem color code and look nice in Traktor): https://github.com/axeldelafosse/stemgen/blob/master/metadata.json#L22-L27

They are passed with mp4box here: https://github.com/axeldelafosse/stemgen/blob/master/ni-stem/_internal.py#L150

[faroit]

interesting. So while the actual metadata is just a binary encoded json, I don't find any information if this is insertable into the UDTA with ffmpeg...

Maybe it would be easier to add mp4box to conda-forge?

[axeldelafosse]

Probably!

[faroit]

Probably!

or manually edit the binary header... but I'm not sure if python it the right language to do that ;-) https://github.com/axiomatic-systems/Bento4/issues/30

[axeldelafosse]

Haha! I'm not familiar with conda-forge, I'll dig into it.

[faroit]

Haha! I'm not familiar with conda-forge, I'll dig into it.

maybe this is out-of-scope and we should ask for this upstream

[kartguru]

Therefore I would like to stick with ffmpeg if possible.

I thought you would say that ... and I think longterm its a better approach as NI don't (seem to) support their dev kit anymore (ref: https://www.native-instruments.com/forum/threads/who-wants-to-generate-stems-files.260123/page-8#post-1867507).

For me ni-stem CLI doesn't work in Win 10 and NI said they will not support so I have their Stem Creator software (slow process) or to look at other options.

[faroit]

coming back to this. Would someone be able to convert my the Easton Ellises - Falcon 69.stem.m4a using NI stem creator and add this to this repo via PR so we have a better unit test to start with?

[kartguru]

sure i can give it a shot. do you have the separated tracks?

[kartguru]

For me ni-stem CLI doesn't work in Win 10 and NI said they will not support so I have their Stem Creator software (slow process) or to look at other options.

In the process of adding ffmpeg and trying the mp4box win install I can now run the ni-stem successfully BUT I still like the path you are headed down @faroit

[faroit]

sure i can give it a shot. do you have the separated tracks?

well you can just demux them with stempeg ;-)

[faroit]

In the process of adding ffmpeg and trying the mp4box win install I can now run the ni-stem successfully BUT I still like the path you are headed down @faroit

@kartguru sure, but the goal should be to be compatible with NI stems, so we should probably add this as a test file

[kartguru]

sure i can give it a shot. do you have the separated tracks? well you can just demux them with stempeg ;-)

hahaha, of course, i was always interested in the other way so forgot about this feature. it worked well i just had to run as stem2wav "The Easton Ellises - Falcon 69.stem.mp4" on my Win 10 machine.

Here's the Stem Creator details for your reference

Unfortunately I am not a coder nor github expert ... just a hack who sometimes reports bugs where I can and as best I can. I assume I just upload the file with the intention to overwrite. If so I need 'push access to this repository'. Let me know if I'm wrong.

I'm dream feeding a 6 month old child in a few minutes then off to bed so I'll pick up in the morning.

[faroit]

@kartguru thanks for your help. I will try on my machine in the next days and get back to you soon.

[kartguru]

sorry @faroit , I think you misunderstood. I cannot upload the file (assuming this is the best way to contribute/get the file to you). see below pic

and @faroit , thanks for your vision and hard work getting stempeg to here ;)

[axeldelafosse]

Hey guys! Sorry for the delay, I'm pretty busy during the week. I'm on it!

[faroit]

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

[axeldelafosse]

Here we go: https://github.com/faroit/stempeg/pull/32

[axeldelafosse]

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

Nice! Agree. Any news regarding mp4box on conda-forge? How can I help?

[faroit]

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

Nice! Agree. Any news regarding mp4box on conda-forge? How can I help?

Maybe we can together work on a conda recipe. Feel free to kick this off yourself, I won't have much time this week.

The build scripts seem quite simple. Maybe just start by looking at other package managers such as homebrew

[axeldelafosse]

Let's do it. I started to work on it, testing on Unix first.

Let's catch up when you have more time. Have a good one!

[faroit]

@axeldelafosse did you had some time to look into a mp4box conda package?

[axeldelafosse]

Not since last time, sorry :(

[faroit]

@axeldelafosse no worries. Will have a look next week.

[faroit]

back to this. I figured out the metadata. Its actually quite simple

the trick is that the json also needs to have the mastering_dsp tags to make it work in traktor.

meta = {
  "mastering_dsp": {
    "compressor": {
      "ratio": 3,
      "output_gain": 0.5,
      "enabled": False,
      "release": 0.300000011920929,
      "attack": 0.003000000026077032,
      "input_gain": 0.5,
      "threshold": 0,
      "hp_cutoff": 300,
      "dry_wet": 50
    },
    "limiter": {
      "release": 0.05000000074505806,
      "threshold": 0,
      "ceiling": -0.3499999940395355,
      "enabled": False
    }
  },
  "version": 1,
  "stems": [
    {"color": "#009E73", "name": "Drums"},
    {"color": "#D55E00", "name": "Bass"},
    {"color": "#CC79A7", "name": "Other"},
    {"color": "#56B4E9", "name": "Vocals"}
  ]
}

and voila...

this can be added to a file (in-place) using

mp4box stempeg/data/The\ Easton\ Ellises\ -\ Falcon\ 69.stem.mp4  -udta "0:type=stem:src=base64,eyJtYXN0ZXJpbmdfZHNwIjogeyJjb21wcmVzc29yIjogeyJyYXRpbyI6IDMsICJvdXRwdXRfZ2FpbiI6IDAuNSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbGVhc2UiOiAwLjMwMDAwMDAxMTkyMDkyOSwgImF0dGFjayI6IDAuMDAzMDAwMDAwMDI2MDc3MDMyLCAiaW5wdXRfZ2FpbiI6IDAuNSwgInRocmVzaG9sZCI6IDAsICJocF9jdXRvZmYiOiAzMDAsICJkcnlfd2V0IjogNTB9LCAibGltaXRlciI6IHsicmVsZWFzZSI6IDAuMDUwMDAwMDAwNzQ1MDU4MDYsICJ0aHJlc2hvbGQiOiAwLCAiY2VpbGluZyI6IC0wLjM0OTk5OTk5NDAzOTUzNTUsICJlbmFibGVkIjogZmFsc2V9fSwgInZlcnNpb24iOiAxLCAic3RlbXMiOiBbeyJjb2xvciI6ICIjMDA5RTczIiwgIm5hbWUiOiAiRHJ1bXMifSwgeyJjb2xvciI6ICIjRDU1RTAwIiwgIm5hbWUiOiAiQmFzcyJ9LCB7ImNvbG9yIjogIiNDQzc5QTciLCAibmFtZSI6ICJPdGhlciJ9LCB7ImNvbG9yIjogIiM1NkI0RTkiLCAibmFtZSI6ICJWb2NhbHMifV19"

I am now working on a PR

[axeldelafosse]

Cool!

[faroit]

@axeldelafosse next question: how does traktor handle the mix? The stems metadata only hold the four sources

  "stems": [
    {"color": "#009E73", "name": "Drums"},
    {"color": "#D55E00", "name": "Bass"},
    {"color": "#CC79A7", "name": "Other"},
    {"color": "#56B4E9", "name": "Vocals"}
  ]

That means the mix will be the first mp4 substream but doesn't get any metadata?

[axeldelafosse]

I think Traktor doesn't use the mix. Yes the mix is the first mp4 substream: https://github.com/axeldelafosse/stemgen/blob/909d9422af0738457303962262f99072a808d0c1/ni-stem/_internal.py#L141

And I'm not sure about the metadata but I guess you are right: https://github.com/axeldelafosse/stemgen/blob/909d9422af0738457303962262f99072a808d0c1/ni-stem/_internal.py#L150

[faroit]

@axeldelafosse alright. Then, I wonder about

#ID=Z for the mix and #ID=Z:disable for the sources. Any idea what this is for?

[axeldelafosse]

I have no idea, sorry...

[faroit]

fixed in #28 , documentation will be added asap