Handling directory with several subdirs as having "reels" from same content

[JeromeMartinez]

This kind of directory structure is expected to be handled:

        ├── 99075c82-2770-4020-a289-d191dab0b852_1
        │   ├── 99075c82-2770-4020-a289-d191dab0b852_1_00086400.dpx
        │   ├── (...)
        ├── 99075c82-2770-4020-a289-d191dab0b852_2
        │   ├── 99075c82-2770-4020-a289-d191dab0b852_2_00086400.dpx
        │   ├── (...)
        ├── 99075c82-2770-4020-a289-d191dab0b852_3
        │   ├── 99075c82-2770-4020-a289-d191dab0b852_300086400.dpx
        │   ├── (...)
        ├── (...)
        ├── 99075c82-2770-4020-a289-d191dab0b852_9
        │   ├── 99075c82-2770-4020-a289-d191dab0b852_900086400.dpx
        │   ├── (...)
        ├── 99075c82-2770-4020-a289-d191dab0b852_10
        │   ├── 99075c82-2770-4020-a289-d191dab0b852_1000086400.dpx
        │   ├── (...)
        ├── (...)

[JeromeMartinez]

@kieranjol and other users could you comment on such subdirectory structure? I am surprised about such structure, as it looks like there is a change of paradigm between reel 2 and reel 3: reel 1 & 2 have an underscore after the reel number in the DPX name, not reel 3 and above. I could still handle such structure, testing only the incrementing directory names, but I would have preferred an extra check (DPX name is reel name + underscore) for avoiding some false positive in reels detection.

[kieranjol]

Hi Jerome,

Did that example come from me? If so I'll have to check with the folks that created that example and get a sense of what exactly happened there.

On Sat, 28 Apr 2018, 14:02 Jérôme Martinez, notifications@github.com wrote:

@kieranjol https://github.com/kieranjol and other users could you comment on such subdirectory structure? I am surprised about such structure, as it looks like there is a change of paradigm between reel 2 and reel 3: reel 1 & 2 have an underscore after the reel number in the DPX name, not reel 3 and above. I could still handle such structure, testing only the incrementing directory names, but I would have preferred an extra check (DPX name is reel name + underscore) for avoiding some false positive in reels detection.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MediaArea/RAWcooked/issues/53#issuecomment-385174315, or mute the thread https://github.com/notifications/unsubscribe-auth/ABEyvl0iaASY1rDyJAGSOiG8g9MBL1bzks5ttGh0gaJpZM4TrZBN .

[stephenmcconnachie]

From BFI point of view we would enforce a strictly predictable folder / filename convention without anomalies from reel to reel, and wed police that convention in our creation / acquisition of DPX, so that the input to the tool would be standard, to avoid false positives.

[retokromer]

I would not recommend either to use such a structure.

Yet why does this hurt? I guess, the regex matching incremential numbers from timecode 01:00:00:00 on (at 24 fps) would work fine in any case, or am I missing something?

[JeromeMartinez]

I would not recommend either to use such a structure.

I don't recommend either (I don't get how this structure makes sense), but I have to handle real use cases :) and I guess this directory structure is playable by the tool having created that (the order is available for a tool developed for handling such directory and file naming structure).

Note that if RAWcooked missed the reels directory structure, the MKV is weird (1 video stream per reel) but reversibility is kept so nothing is lost (the MKV file is just not playable as expected).

[retokromer]

(the MKV file is just not playable as expected)

Not sure I understand why.

[kieranjol]

Btw I think the structure of that particular example was auto generated by davinci Resolve when scanning via the Cintel. I'll have more info Monday.

On Sat, 28 Apr 2018, 15:29 Reto Kromer, notifications@github.com wrote:

(the MKV file is just not playable as expected)

Not sure I understand why.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MediaArea/RAWcooked/issues/53#issuecomment-385180117, or mute the thread https://github.com/notifications/unsubscribe-auth/ABEyvnJZDMwIqidVdH3dCRTqLcWVwBUHks5ttHy2gaJpZM4TrZBN .

[JeromeMartinez]

(the MKV file is just not playable as expected)

Not sure I understand why.

When RAWcooked missed the reels, it considers each directory as a video stream (as with audio files for the moment, i.e. 1 WAV = 1 audio track), because other users have 1 directory per video track (1 track for "raw" content, 1 track for "with forced subtitles" content, both tracks are synchronized so you can switch from 1 video track to another video track like you do when you switch from 1 audio track to another one), so MKV playback duration will be the maximum duration of a reel and you will see 1 video track per reel in VLC (you can switch from one track to another one). I guess this is not the expected behavior, you an play it but you have to switch the video track and start again the playback for getting the next reel, so "not playable as expected".

When RAWcooked caught the reels, you get only 1 video stream with all reels concatenated and MKV duration is the sum of all reels duration, I guess this is what you are looking for.

[retokromer]

I think to have the reels playable one by one in VLC is a BIG step forward, in comparison to the current situation, where special software is needed (or the use of mpv or ffplay with the correct parameters to match the colours).

I don’t consider this an important issue, and should not block the 1^st release, in my opinion.

Yet I still not see why the regex should miss the reels.

[JeromeMartinez]

I think to have the reels playable one by one in VLC is a BIG step forward,

This is my medium term target.

Yet I still not see why the regex should miss the reels.

Someone may decide (imagination has no limit) to call directories ReelA, ReelB... or Reel1OfFile1, Reel2OfFile1... and I would not catch that (I plan to check only numbers at the end)

[JeromeMartinez]

While checking the RAWcooked file structure, and from https://github.com/MediaArea/RAWcooked/issues/54#issuecomment-385186145 indicating that there may be 1 WAV per reel, note for myself: right now the following configs are possible:

• 1 file per frame (e.g. 1 DPX per frame), with header and footer data per frame stored in the Frame part of RAWcooked file.
• 1 file per track (e.g. 1 WAV for the whole track), with header and footer data per track in the Track part of RAWcooked file.

It is not possible to store header and footer data for several source files in the same track if source file is not per frame, and this is needed if we want to concatenate the WAV files content in the resulting MKV (1 MKV file for several reels).

There is the need to add something in RAWcooked file format for handling that, keeping in mind that the muxer (e.g. FFmpeg) decides about the duration of an audio frame and could change the frame duration (not mandatory that the audio frame duration is same as the video frame duration), so maybe something indicating how many bytes are expected per WAV file.

[retokromer]

Someone may decide (imagination has no limit)

Many labs do use for reel 1 name_01nnnnnn.dpx, for reel 2 name_02nnnnnn.dpx, etc. therefore, I guess, it’s a kind of a de facto standard.