Closed robertsamm closed 1 year ago
HI, everyone, I have double checked the data destructions in dev for this Connect_ID: all the data is deleted correctly as expected
Hi all - I checked SMDB and there was one issue for connect_id: 2361618927. I could not download the original HIPAA and consent agreement forms. Instead I received an error message that said "An error has occured generating the pdf please contact support". Otherwise, everything else looked good - I was able to download the data destruction and HIPAA revocation forms and the signatures were correct on SMDB, I was able to access all 4 forms in MyConnect, the notifications are gone from both the MyConnect and the SMDB, and the correct variables were the only ones remaining in SMDB.
@kmazzilli this issue will be fixed by this PR https://github.com/episphere/dashboard/pull/568
@jhflorey thanks! I was able to download the forms on SMDB without the error message just now.
@jhflorey We just noticed someone entered a data destruction request for a HFH participant in stage. We don't know who originated this one... what this one you initiated?
Just pushed a data destruction in stage for Connect ID: 2055639272 and signed the data destruction and revocation forms. @jhflorey we can check tomorrow to see if everything looks good for the stub record.
Hi, @robertsamm, may you please let me know where I can find the stub record for Connect ID:2055639272. It will be helpful for me to double check on my end for the data destruction. Thanks, Amelia
Addtl data destructions that were pushed to stage today, neither have study activity data in them. Connect IDs: 8553496854, 3800067442
@robertsamm, the data destructions for these two Connect IDs will be processed as same as the previous one for all the data in Stage, right?
@jhflorey We just noticed someone entered a data destruction request for a HFH participant in stage. We don't know who originated this one... what this one you initiated?
@brotzmanmj my test participant in stage
@jeannewu Yes, you'll want to check all 3 records. I confirmed w/ Michelle - the stub record is not a separate record. It is just a list of the only variables that should remain after everything else is destroyed. They will reside in the data structure the same place they always do.
We'll first need to have @jhflorey confirm the destruction was run overnight and that the 3 Connect IDs (CIDs: 2055639272, 8553496854, 3800067442) data was deleted and that they look correct in Firestore first. Then Jing you can confirm the same in BQ after that. If you have any questions about the stub record, please ask Jessica who would know best.
@robertsamm I am checking in the all the data in Dev, I don't see any data on these three Connect_IDs even in the participants table, right?
@jeannewu these are stage participants
@robertsamm Thanks. I am checking them now.
@robertsamm @jhflorey I have just checked the BQ data: right now the 2055639272 data are still shown in the stage, (seen the table)
data in stage | undeleted |
---|---|
bioSurvey_v1_JP | 2.06E+09 |
biospecimen_JP | 2.06E+09 |
clinicalBioSurvey_v1_JP | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
covid19Survey_v1_JP | 2.06E+09 |
module1_v1_JP | 2.06E+09 |
module1_v2_JP | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module2_v1_JP | 2.06E+09 |
module2_v2_JP | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module3_v1_JP | 2.06E+09 |
module4_v1_JP | 2.06E+09 |
participants_JP | 8553496854, 3800067442, 2055639272 |
participants_noPII_JP | 8553496854, 3800067442, 2055639272 |
@jeannewu @robertsamm The job was still working last night but it ran into a problem about the index key since we have not generated it in stage (firebase) yet. I asked for help from @anthonypetersen and will inform to you soon. Sorry for this inconvenience.
@jhflorey Thank you very much. Please let me know if anything is needed from me.
@jeannewu it's already worked, i have also checked my test participant (3800067442). It worked as my expectation. I suggest that you can check it on PWA and SMDB first then check data in BQ later.
Hi @jhflorey can you check the other two participants as well and then we'll check the SMDB and PWA?
@jhflorey I have just checked the stg-5519.Connect: all the data of these three participants are deleted as expected
data in stg-5519.Connect | data deletion |
---|---|
bioSurvey_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
biospecimen | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
clinicalBioSurvey_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
covid19Survey_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
menstrualSurvey_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module1_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module1_v2 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module2_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module2_v2 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module3_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
module4_v1 | No data on Connect_ID: 8553496854, 3800067442, 2055639272 |
participants | 2055639272|3800067442|8553496854 dim= 3 181 |
Maybe I will check their deletion in the BQ data next Monday when the BQ data is refreshed.
@jeannewu yup, thank you so much.
Maybe I will check their deletion in the BQ data next Monday when the BQ data is refreshed.
You can check on PWA and SMDB first.
Thanks all. Kaitlyn and I will check PWA and SMDB. @jeannewu Monday is a federal holiday, so check on Tuesday.
@robertsamm Yup, I will check the data of these three CID next Tuesday.
Kaitlyn and I checked the SMDB and PWA for all 3 participants and they looked good in stage. Destruction and stub record variables are as expected.
@robertsamm Yup, I will check the data of these three CID next Tuesday.
@jeannewu Have you checked it?
@jhflorey @robertsamm I've just checked the BQ data: all the data of these three CID are deleted as expected. Thanks for double checking with me.
@jeannewu thanks for your confirmation
Reviewed stub record in stage for connectID 1475895409 and found a few remaining issues. After the stub record was pushed, it did the same thing in the SMDB where I can't view the Participant Summary page for this participant anymore, I'm not sure if that is intentional or not but I don't see it in the SOP. So looking at this record in the SMDB and the CIDs in the array Jessica sent me here's the few remaining issues I found