search

TUMFARSynchrony / SynthARium

An online tool for researchers to host and conduct customizable remote experiments with adaptable vision and audio pipelines in a multiuser setting.
GNU General Public License v3.0
30 stars 4 forks source link

Synchrony Score Input Stream Pipeline Filter #29

Closed KuroiX closed 1 year ago

KuroiX commented 1 year ago

Synchrony score integration into the hub. Each relevant participant gets a filter applied that calculates the input of the synchrony score algorithm from the input frame and saves it over time (windowSize frames).

NCEghtebas commented 1 year ago

This issue is broken down into several smaller issues, i.e.

We have an AU extraction filter from PR #37

With the introduction of GroupFilters in PR #141 we can now apply time synchornized filters on more than 2 participants video/audio streams.

Issue #107 would set up the realtime BOW on the AU signals in order to compute the realtime OASIS score.

With the introduction of our OpenFace Forked Repo, we have an optimized build of openface that can help extract AU's closer to realtime: https://github.com/TUMFARSynchrony/experimental-hub-openface