DANNCE support for different sized/proportioned animals and vibration canceling methodologies

[Vusmeree]

Hello! We want to migrate fro DLC to DANNCE. Before getting started, we wanted to verify a few things.

First, does DANNCE have any issues with different sized animals? We are dealing with marmosets and want to capture data from both small youths to fully grown adults. Would we have to make a different data set for the smaller animals vs the bigger ones or can we use the same data set for both? Since, DANNCE knows where the position of the markers relative to each other, would this cause a problem for different sized/proportioned animals?

Secondly, due to portability, our cameras are attached to the animal setup. As such there is some vibration transmitted to the cameras. Would it be a problem for DANNCE to track the corners of the enclosure so that we can take out vibration noise in post? I assume that if they aren't attached to the skeleton then there shouldn't be a problem but since DANNCE tracks where the points are relative to other points I don't know if this would be a problem later on.

If there is somewhere else I should ask these questions please let me know and I will be happy to move my questions there. Thank you for your help!

[spoonsso]

Sorry for the delay, I have been traveling. We have not tested scale invariance systematically, but if you design the 3d bounding box for each condition such that the animals are described by a similar number of boxes, then I would expect to see reasonable generalization across conditions. For instance, say the adults are 2x as big as the youth and you use a 3d bound box that is 600 mm per side for the adults (set via the vol_size variable, then if you set the box to 300 mm for the youth your image features will be the same size in terms of voxels (which is all that the network learns about).

The problem is that dannce currently doesn't support combining different sized volumes during training -- so if you wanted to combine labels across adults and youth you would either need to request an update (or help us implement it :-)) or get creative with your image resolution and/or calibration.

Dannce prediction does support using different volume sizes, however. So if you were to train over only the adults and wanted to test on the youths using a proportionally smaller volume, you could.

[spoonsso]

As for the vibration tracking, you would most likely need to train a separate dannce network to do this because the network tracking your animal assembles a 3d volume tightly enclosing your animal. Unless your arena is super tiny, the arena corners will typically not be inside this volume.