search

BIDMCDigitalPsychiatry / LAMP-platform

The LAMP Platform (issues and documentation).
https://docs.lamp.digital/
Other
12 stars 10 forks source link

Backwards Compatibility for Cortex step_count #677

Closed AsherCohenBerkeley closed 1 year ago

AsherCohenBerkeley commented 1 year ago

Describe the bug

Lack of backwards compatibility for cortex.secondary.step_count.step_count

To Reproduce

Try running cortex.secondary.step_count.step_count on data earlier than March 2022 like a SHARP participant

Expected behavior

There should be

(1) backwards compatibility in this cortex function (2) a script for data migration of existing legacy data to the new format (3) some kind of documentation explaining the old format and what happened to it

dcurrey88 commented 1 year ago

I'm sorry we didn't document our reasoning (you guys can add docs on this) but to provide some insight, the reason we chose to not preserve backwards compatibility for step count is because the step data was not good. There were different data formats across Android / iOS and there were questions about whether steps were cumulative (and if so when they reset) (https://github.com/BIDMCDigitalPsychiatry/LAMP-platform/issues/503). Moreover, we were only getting data from a few devices (lots of missing Android data). Overall, the data we were generating was not useful so we decided it wasn't worth it.

That being said if you want to change this to have backward compatibility, go for it. While you make these cortex edits, in the short term you can either 1) Use the older version of cortex (2022.03.11) or 2) look in _data/danielle_data for already processed features.

AsherCohenBerkeley commented 1 year ago

Thank you so much for this, Danielle. Indeed, for SHARP, we decided to just use the old version of Cortex. I think going forward we should be ok w j the newer version.