We need to fix the code so that subtypes are not used in softmax layer, or subtypes are used in training example. (In the latter case, we should instead take out S label)
Reproduction steps
Run SWT on a video, confirm the labelset in the TimePoint annotation objects are 22-way classification.
Bug Description
When we run the app, it returns classification results (on each timepoint) on 22 labels
(for example)
But the subtype-suffixed labels were never actually used when reading the annotations for training
https://github.com/clamsproject/app-swt-detection/blob/5925f029c6f13446a78e7144ae30f146354186a2/modeling/train.py#L136
(subtype labels are stored as
labels['frames'][i]['lsubtype_abel']
in the preprocessed annotation metadata file)But used in softmax space.
https://github.com/clamsproject/app-swt-detection/blob/5925f029c6f13446a78e7144ae30f146354186a2/modeling/train.py#L76-L80
and
https://github.com/clamsproject/app-swt-detection/blob/e6662c46aa19e4540eccf42c83ecbf07cf9397be/modeling/__init__.py#L5-L6
We need to fix the code so that subtypes are not used in softmax layer, or subtypes are used in training example. (In the latter case, we should instead take out
S
label)Reproduction steps
Run SWT on a video, confirm the labelset in the
TimePoint
annotation objects are 22-way classification.Expected behavior
No response
Log output
No response
Screenshots
No response
Additional context
(full label definition is in #1)