Deleted Activities?

[Tuna9129]

Describe the bug

Tried to pull mindLAMP activity data for each participant in a study. Activity IDs are fine, but when I try to see more details using LAMP.Activity.view(), it returns an empty list as the dict value. (suggests that the activities got deleted)

To Reproduce

Redacted credentials so I could share the code here: Untitled (1).pdf

Expected behavior

When passing every activity id as an argument for LAMP.Activity.view()['data'], I was expecting a list that contained more info about the activity.

Example

LAMP.Activity.view('j50sd1t3f6mce33g7jvb')['data'] returns the following (and is expected):

[{'id': 'j50sd1t3f6mce33g7jvb',
  'schedule': [{'start_date': '2021-07-14T22:28:00.000Z',
    'time': '2021-07-13T14:00:15.878Z',
    'custom_time': None,
    'repeat_interval': 'daily',
    'notification_ids': [335963]}],
  'spec': 'lamp.journal',
  'name': 'Self-Affirmations',
  'settings': []}]

LAMP.Activity.view('sba4ampmycdvzqwwp2sf')['data'] returns:

[] 

[avaidyam]

@Tuna9129 Thanks for investigating this! As you suspected, it seems like the original Activity objects were deleted, which is why there isn't any name substitution going on, and thus the field is blank even though a score exists for the ActivityEvent. I'm not sure this is fixable without un-deleting the Activities from the database? (Which is possible using Cortex util functions, but risky/dangerous...)

@lukeoftheshire Thoughts? If I'm correct, then I think the next step is for @Tuna9129 to use the Cortex util function above and re-try.

[Tuna9129]

Why were they deleted? Was it just a bug? Ok I can try using the db functions if given the green light! Thanks again :D

[avaidyam]

@Tuna9129 I'm not sure! 😅 I suspect if this was the digital clinic or a similarly set up study, the older activities were no longer required for patients and thus deleted (to avoid cluttering the home UI). (FYI For future reference, the better way to handle this, though, is to hide the activity from the home tabs instead of deleting the activity entirely.)

[Tuna9129]

ok got it, thanks for explaining!

[lukeoftheshire]

Hey @Tuna9129 @avaidyam I might be a little late to the party but I think these explanations are exactly right - returning an empty array from LAMP.Activity.view basically means the activity was deleted. However, due to how LAMP stores data, the data is not really gone - instead, a Deleted tag is toggled in the database to True, and any data where this key is set to true is simply not returned from an API request (but still exists!). To my knowledge there's no way to fix this through the API alone since there isn't currently an undelete API method - although one has been proposed: see https://github.com/BIDMCDigitalPsychiatry/LAMP-platform/issues/263

This means it can be recovered from cortex using those util functions above - which should at least be a little safer than writing new code to interact directly with the DB. When I've had to do this in the past I've used the https://docs.lamp.digital/data_science/cortex/utils/database/#utilsdblist_deleted_activities function to get a list of deleted activities on a study by study basis, then https://docs.lamp.digital/data_science/cortex/utils/database/#utilsdbrestore_activities to restore them (in fact, the code example for restore_activities gives an example of that).

In terms of why these were deleted my recollection is that it was less about clearing up the Researcher home page and more about removing activities that were no longer desired by or necessary to show to study participants - now that might be more easily done by explicitly assigning the activity to not appear on any users page as outlined here: https://github.com/BIDMCDigitalPsychiatry/LAMP-platform/issues/165

^This functionality might need to be outlined on the LAMP docs as I couldn't find it.

[avaidyam]

now that might be more easily done by explicitly assigning the activity to not appear on any users page

I'll move this issue to more of a docs issue than a cortex issue as you're absolutely right, @lukeoftheshire. I'll work with @ertjlane & co to add some brief info on this feature to the docs soon.