Making dataset for the stacked CAE

[mohammad-brdrn]

Thank you for your code and for sharing it. There is one point I wanted to share with you.

When preparing cubes of n images as input (for example [32,10,100,100]), the difference between folders are not taken into consideration. it means that this cube below (which is prepared) would not be a proper one:

[frame198-folder 2, frame 199-folder2, frame 200-folder2, frame1-folder 3, frame 2-folder3, .....]

as you can see in the list above, some frames from different adjacent folders would be mixed. As the folders are for different timing it would not be a continuous combination.

[Rangachary]

Dear Mohammad, can you please elaborate the issue faced? So that, we can suggest if we can.

[mohammad-brdrn]

Sure. Look in Ped1 /training, there are 34 folders. The contents of these folders are frames for video clips which ate taken from the same scene but at different time. for example, look at the last frame of folder 30 and the first frame of folder 31. it is clearly shown that they are from a different time. So folder 31 does not follow folder 30 temporally.

Now in section 2 of your code: you prepare stacked frames, to let the network capture the temporal evaluation. and you stack 10 consecutive frames for example. the problem lies here that in your code, you "glob.glob" all frames and attach 10 of them in each step. now one of your produced inputs to the network would be:

[f30-197, f30-198, f30-199, f30-200, f31-1, f31-2, f31-3, f31,4, f31-5, f31-6]. ---> f30-199 means frame 200 from folder 30.

now the network which wants to learn temporal evaluations (motions and frame changes) sees f30-200 and f31-1 consecutively and tries to learn it!! but it shouldn't.

if you put the frames of the example above in a folder and watch them in a row, you can see the sudden change.

I hope I could make it clear. By the way, appreciate your nice job, they are very helpful.