CancerRegistryOfNorway / nordcancore

Other
0 stars 0 forks source link

Write a script to clean dir_result #21

Closed CotterpinDoozer closed 1 year ago

CotterpinDoozer commented 2 years ago

Joonas writes:

During running nordcan.R, sensitive data is saved into dir_result as before. It now occurs to me that those should probably be deleted at the end of the script when they are no longer needed. I would write a single function into perhaps "nordcancore" for this cleaning-up. Then at the end of the script there is simply this:

  # run this if you think you have finished here. this cleans up potentially
  # sensitive data in dir_result.
  nordcancore::clean_results(dir_result)

I think this is a good idea. The user should have the option to delete these data after having finished everything, and it is ok to leave it up to the user to do this or not.

HuidongTian commented 2 years ago

Can you give some clues which are sensitive data? I can

  1. delete all files which are sensitive. E.g. cancer_record_dataset.rds, and some files in folder "iarccrgtools" & "survival", etc.
  2. or, we delete all files except the zip files.

@WetRobot @CotterpinDoozer

WetRobot commented 2 years ago

I would delete cancer_record_dataset.rds and iarccrgtools and survival dirs entirely at the end of the script. There may be others, I don't remember.

In any case, the function that deletes those should ask the user whether to delete them or not. See readline R function.

Do you want to permanently delete all these files? They are not needed if you have successfully
finished the whole process. We recommend deleting these once you have submitted your data.
These files and directories contain sensitive data.
- cancer_record_dataset.rds
- iarccrgtools/
- survival/
1: yes, delete these permanently
2: no, don't delete anything yet
: 
CotterpinDoozer commented 1 year ago

Issue: Write a script to clean dir_result #21 Huidong comment: Huidong will take this task. Estimate time: 0.5 hour.

HuidongTian commented 1 year ago

Updated.

https://github.com/CancerRegistryOfNorway/nordcancore/blob/master/R/utils.R#L611 https://github.com/CancerRegistryOfNorway/NORDCAN/blob/master/nordcan_participant_instructions/nordcan.R#L322

AnnaSkog commented 1 year ago

Can I test this?

HuidongTian commented 1 year ago

Not now, I need to compile the source to a R package, then you need to remove the nordcansurvival packages installed already on your computer, and install the new version, then you can test.

From: AnnaSkog @.> Sent: 1. februar 2023 09:47 To: CancerRegistryOfNorway/nordcancore @.> Cc: Huidong Tian @.>; State change @.> Subject: Re: [CancerRegistryOfNorway/nordcancore] Write a script to clean dir_result (Issue #21)

Can I test this?

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_CancerRegistryOfNorway_nordcancore_issues_21-23issuecomment-2D1411673805&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0cSnZK5BHxe8AcnTZAvseb79v-Xr0zxliT3eeFBGEus&m=IW0KtpfM7uZbBJsaQunWjX1ayqvCIG9ojhHy93J81A8&s=7xA2oQF40pz44s-mDAzLkM88x7KHQYV0boAVEagwqgg&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AOMXZFVAKZL232X7U5CRDH3WVIPGNANCNFSM53KJTGDA&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0cSnZK5BHxe8AcnTZAvseb79v-Xr0zxliT3eeFBGEus&m=IW0KtpfM7uZbBJsaQunWjX1ayqvCIG9ojhHy93J81A8&s=EGzd_yIKxV1L6vXHbbGyx18KJsaEfrE0qtq2AF1envg&e=. You are receiving this because you modified the open/close state.Message ID: @.**@.>>