search

beeldengeluid / dane-visual-feature-extraction-worker

Uses the VizXP feature extraction to produce features/vector spaces
MIT License
0 stars 0 forks source link

Proper indexing of input #20

Closed Veldhoen closed 1 year ago

Veldhoen commented 1 year ago

This issue concerns the module data_handling.py In the current implementation, indexing of the input is left to a sort of the input paths, and it is implicitly assumed that every keyframe has a corresponding spectogram.

It would be nicer to have the VisXPData class have a dedicated list of timestamps (that have corresponding frames a/o spectograms) and use that for indexing (in__getitem__).

In theory, frame or spectogram may not exist and it is still possible to create features for the existing input. Proper handling of this scenario would need to be implemented in the feature_extraction.py.

Veldhoen commented 1 year ago
Veldhoen commented 1 year ago

Handling of non-existing spectograms or frames is out of scope for now. The current behaviour of data_handling in those cases, is to return a all-zeros tensor for those cases. However, this is obscured by model application (in feature_extraction > apply_model). If this (not existing sound a/o image) would become a realistic scenario, this needs to be dealt with properly.