Closed yuan0821 closed 1 year ago
Can you plot the projected COM onto these images as well? Are you using a MAX model or an AVG model? If a MAX model, you may need to add the 3D COM coordinates to the predictions, depending on which version/branch you are using.
On Tue, Feb 7, 2023, 12:28 PM yuan0821 @.***> wrote:
hi @davidhildebrand https://github.com/davidhildebrand @spoonsso https://github.com/spoonsso @data-hound https://github.com/data-hound
The loss of the prediction result is very low, and the relative position of each label is correct, but the result seems to be shifted as a whole, which does not match the video image. But the result of com-predict looks normal. Thank you so much for your time... ` viewGui = View3D(params, videos, skeleton);
pts3d = load('F:\dannce\demo\mouse20230129\DANNCE\predict_results\save_data_MAX0.mat')
viewGui.loadFrom3D(pts3d.pred) `
[image: 截屏2023-02-08 上午1 12 23] https://user-images.githubusercontent.com/50692530/217318472-04e55bc1-0789-4a02-a367-40765b162316.png
— Reply to this email directly, view it on GitHub https://github.com/spoonsso/dannce/issues/138, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW2P4ZECDIFUDLUASIN6E3WWKA2VANCNFSM6AAAAAAUUH5AAE . You are receiving this because you were mentioned.Message ID: @.***>
Thank you @spoonsso I start with MAX and finetune with a 5-cam-pre-trained MAX network. I have four cameras, so I duplicate the camera1 video file as camera5. And also, copy the sync and label data in the .mat
net_type: MAX
train_mode: finetune
num_validation_per_exp: 0
max_num_samples: 1000
dannce_finetune_weights: 'F:\dannce\weight\MAX5\'
Please advise how to load com and dance predict results in the same image. I use the dannce_development branch. [https://github.com/spoonsso/dannce/tree/release_development]
And I also don't know how to add the 3d COM coordinates to the prediction; only use:
new_predict=pred(:, :,1:16)+com(:,:);
???
After the addition,
the result same a little better, but still shifted.
hi @spoonsso thank you for suggestion. the result is normal now after add COM corrdinates
for i=1:16
new_predict(:,:,i)=dannce_predict.pred(:,:,i)+com_predict.com;
end
skeleton = load('F:\dannce\Label3D\skeletons\rat16.mat');
close all
viewGui = View3D(params, videos, skeleton);
viewGui.loadFrom3D(new_predict(framesToLabel,:,:));
Perfect, glad it is working now. By the way, the reason why the end of your tail is not being tracked well is because it is most likely outside of the 3D extent of the image volume.
@data-hound there is an addCOM
argument in processing.savedata_tomat()
that would solve this problem during dannce prediction, but it looks like it is not being used. Can we add this to the release you are working on?
Yes, we can add this addCOM
argument to the savedata_tomat()
. This should be done for MAX model predictions, correct?
Yes, the AVG predictions already have the COM built in b/c they use the world grid coordinates in the calculation.
On Tue, Feb 7, 2023, 2:50 PM Anshuman Sabath @.***> wrote:
Yes, we can add this addCOM argument to the savedata_tomat(). This should be done for MAX model predictions, correct?
— Reply to this email directly, view it on GitHub https://github.com/spoonsso/dannce/issues/138#issuecomment-1421354161, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW2P446HYD233MBOOZJDFLWWKRQDANCNFSM6AAAAAAUUH5AAE . You are receiving this because you were mentioned.Message ID: @.***>
hi @davidhildebrand @spoonsso @data-hound
The loss of the prediction result is very low, and the relative position of each label is correct, but the result seems to be shifted as a whole, which does not match the video image. But the result of com-predict looks normal. Thank you so much for your time... ` viewGui = View3D(params, videos, skeleton);
pts3d = load('F:\dannce\demo\mouse20230129\DANNCE\predict_results\save_data_MAX0.mat')
viewGui.loadFrom3D(pts3d.pred) `