ERROR WHEN ATTEMPTING TO FAST TRAIN

[GitwellAnyohub]

Hi. I'm trying to get leap up and running and I'm having some errors when I try the fast train. Something is up with the h5files I think but I'm banging my head against the wall trying to figure out what it is. Perhaps you guys could assist me? It would be very appreciated.

[talmo]

Hi there,

That seems to be quite a strange issue. Would you mind running this command in the MATLAB console and pasting the output here: h5disp('/home/rockwell/leap/box.h5')

Thanks!

[GitwellAnyohub]

sure! @talmo

h5disp('/home/rockwell/leap/box.h5') HDF5 box.h5 Group '/' Attributes: 'boxPaths': '/home/rockwell/leap/box.h5' Dataset 'box' Size: 192x192x1x435 MaxSize: 192x192x1x435 Datatype: H5T_IEEE_F32LE (single) ChunkSize: 192x192x1x1 Filters: deflate(1) FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'dataIdx' Size: 435x1 MaxSize: 435x1 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'double' Dataset 'exptID' Size: 1x435 MaxSize: 1x435 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'double' Group '/clusters' Dataset 'G' Size: 435x1 MaxSize: 435x1 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'double' Dataset 'cluster_centroids' Size: 192x192x1x5 MaxSize: 192x192x1x5 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'k' Size: 1x1 MaxSize: 1x1 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'double' Group '/pca' Dataset 'coeff' Size: 36864x50 MaxSize: 36864x50 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'coeff_box' Size: 192x192x1x50 MaxSize: 192x192x1x50 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'explained' Size: 1042x1 MaxSize: 1042x1 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'latent' Size: 1042x1 MaxSize: 1042x1 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'mu' Size: 1x36864 MaxSize: 1x36864 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'score' Size: 435x50 MaxSize: 435x50 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single' Dataset 'tsquared' Size: 1043x1 MaxSize: 1043x1 Datatype: H5T_IEEE_F32LE (single) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'dtype': 'single'

[talmo]

Hmm, I'm not entirely clear on why this bug is happening, but try downloading the newest version of the repository (either via git pull or by downloading the master branch) and trying to fast train again.

It seems to be an issue with saving the skeleton, which is not really necessary, so if it's still not working for you, try just commenting out these lines in leap/generate_training_set.m:

h5savegroup(savePath,skeleton)
h5writeatt(savePath,'/skeleton','jointNames',strjoin(jointNames,'\n'))

Let me know if either of these work for you and I'll try to figure out how to fix it from there.

[GitwellAnyohub]

I commented out the code you suggested and it seems to be working (currently on epoch 6). I'll let you know if there are any other problems finishing the demo. Thanks a bunch!

[GitwellAnyohub]

Everything seemed to work correctly although when I run

imagesc(box(:,:,:,1)),axis image,hold on,plot_joints_single(preds.positions_pred(:,:,1),skeleton);

or

vplay(box, @(~,idx)plot_joints_single(preds.positions_pred(:,:,idx),skeleton))

The actual predictions are not showing up on the plot however when I run

figure,plot(squeeze(preds.positions_pred(3,1,:)),squeeze(preds.positions_pred(3,2,:)),'.-')

I get a nice trajectory of the head

[GitwellAnyohub]

Im guess it is because the skeleton wasn't saved due to commenting it out?

[GitwellAnyohub]

Also just in case you're interested, here is how I did the first figure

%Some data restructuring for plotting purposes dim=size(preds.positions_pred); number_labeled_joints=dim(1); %head body and tail number_of_frames=length(preds.positions_pred)/number_labeled_joints; half_the_joints=dim(3);

x_and_y_pos_pred=zeros((half_the_joints),2);

translation_factor=0;

for i=1:dim(3)*number_labeled_joints

x_and_y_pos_pred(i,1)=preds.positions_pred(i+translation_factor);
x_and_y_pos_pred(i,2)=preds.positions_pred(i+number_labeled_joints+translation_factor);

if mod(i,number_labeled_joints)==0
    translation_factor=translation_factor+number_labeled_joints;
end 

end

%%%% HOW TO PLOT

frame=185; dim=size(preds.positions_pred); nlj=dim(1); idx=frame*nlj-(nlj-1);

imshow(box(:,:,:,frame)) axis on hold on; scatter(x_and_y_pos_pred(idx:idx+nlj-1,1),x_and_y_pos_pred(idx:idx+nlj-1,2))

[talmo]

Hi @GitwellAnyohub,

That's right, I'm guessing you're missing the skeleton structure to pass in to the plotting function. You can try it by loading it from the MAT file for the skeleton you created or from the labels file since it's saved there too:

labels = load('/home/rockwell/leap/box.labels.mat');
vplay(box, @(~,idx)plot_joints_single(preds.positions_pred(:,:,idx), labels.skeleton))

I'm curious as to why the saving wasn't working though. Did you ever manage to update the code to see if it would fix it?

I'll leave this issue open for now until I get a chance to dig into it. Let me know if you run into any other problems!

[GitwellAnyohub]

So for the record (concerning the previous issue) the only thing that still works is commenting out that code!

[talmo]

Hi @GitwellAnyohub,

I'm still having a bit of a hard time reproducing this issue. Any chance you could send me a link to your box.h5 and box.labels.mat files?

I'm guessing they're just produced the example data we provide for the tutorial, correct?

Much appreciated!

[talmo]

Closing but feel free to re-open if you're still running into problems!

[talmo]

Quick update: a colleague managed to reproduce this bug and I pushed a fix. The issue was in saving the training dataset with a skeleton with no edges.

Try out the latest version if you're still running into this issue.