catalystneuro / datta-lab-to-nwb

MIT License
1 stars 2 forks source link

[Follow-up] Context of elliptical fit for CompassDirection #12

Closed CodyCBakerPhD closed 1 year ago

CodyCBakerPhD commented 1 year ago

For the behavior part of the Markowitz/Winthrop conversion, we have a CompassDirection container of a single radial SpatialSeries series named OrientationEllipse with description="Mouse orientation in radians estimated using an ellipse fit."

Overall this is a decent representation as far as NWB goes, but related to the experimental design/processing/analysis it is not super clear to me exactly what that means; @pauladkisson can you explain it to me more in this issue and we can discuss how to improve the description/name of the object to further clarify for future re-users?

pauladkisson commented 1 year ago

From the paper, MoSeq: Preprocessing

""" MoSeq consists of two essential workflows: one for pre-processing depth data and converting it into a low-dimensional time series that describes pose dynamics, and another for modelling the low-dimensional time-series data. As previously described, in order to focus on pose dynamics, raw depth frames were first background-subtracted to convert depth units from distance to height from the floor (in millimeters). Next, the location of the mouse was identified by finding the centroid of the contour with the largest area using the OpenCV findcontours function. An 80 × 80 pixel bounding box was drawn around the identified centroid, and the orientation was estimated using an ellipse fit (with a previously described correction for ±180-degree ambiguities4,27). The mouse was rotated in the bounding box to face the right side. The 80 × 80 pixel depth video of the centred, oriented mouse was then used to estimate pose dynamics. """

pauladkisson commented 1 year ago

Digging deeper into the methods of one of the cited papers, I found this description for a bit more detail: """ To subtract the background image of the arena, the median value of the first 30 seconds of data from any imaging stream were taken, and subtracted from all video frames; any spurious values less than zero were by default reset to zero. The image was then binarized and any objects that did not survive three iterations of morphological opening were eliminated. The mouse was defined as the largest object within the arena that survived the subtraction and masking procedures. The centroid of the mouse was identified as the center-of-mass of the preprocessed image; an ellipse was fit to its contour to detect its overall orientation. In order to orient the mouse along the axis of the spine, a random forest classifier was trained on a set of manually oriented mouse images to facilitate automated identification of the head and tail. Although this algorithm was nearly always correct, its output was manually supervised to ensure accuracy. Note that this procedure does not directly identify the spine, but that the ellipse-based alignment procedure has the effect of aligning the animal along this anatomic axis. During episodes when the mouse was reared directly upwards (and therefore appears as a circle against the sensor, which is orthogonal to the axis of the mouse’s spine under these conditions) neither a human nor our algorithm can identify the spinal axis; because we model pose dynamics based upon dimensionally-reduced representations of the pixels (rather than extracted scalar features, like orientation or length), the presence of this symmetry does not adversely affect model performance, and synthetic data generated from the model appropriately captures these rearing episodes without “flipping” the mouse along its spinal axis. """

CodyCBakerPhD commented 1 year ago

OK, I see now thank you

In light of this I'd request a small PR changing the name of that compass direction spatial series to just HeadOrientation and something in the description mentioning how it used the find contours and elliptical fit

Reason being, it being the name of the entire neurodata object makes it seem like the ellipse part is really important to the experiment, where from this its just an implementation detail about how they estimated the which direction the subject was facing