Open jcomedouteau opened 3 years ago
EDIT: This is not relevant anymore, we won't move anything using SQL scripts.
References to move with an SQL script:
| datasets |
center (??)
| import |
nifticonverter
| migrations |
migrations (la base sera à priori à jour)
| preclinical |
anesthetic
anesthetic_ingredient
contrast_agent
pathology
pathology_model
reference
therapy
| studies |
acquisition_equipment
center
coil
manufacturer
manufacturer_model
timepoint
| users | role
For the rest we need to move step by step all the data, that'll be complex as we need to keep a matching key between the old study and the new study.
Migration comptes GME:
The decision was made to make a full programatical solution to migrate data from a study to another server. The idea is tu use as much as possible the rest API to migrate the studies and their datas.
Elements that can we consider already existing
import
migrations
preclinical
studies
users
Users microservices
Users don't have to be migrated, they have to be re-created by the concerned users. Only one user (the one demanding the migration) will be added to the new study as admin
Study Microservice
Elements to be moved:
Subject
Study
Center
AcquisitionEquipement
Model
Manufacturer Preclinical microservice
Elements to be moved:
animal_subject -- reset ID -- Update subject ID with animalSubjectID
subject_pathology -- reset ID -- Update subject ID with animalSubjectID
subject_therapy -- reset ID -- Update subject ID with animalSubjectID
anesthetics
therapies
AFTER DATASETS
Dataset microservice
This is the most complicated microservice as it contains all the logic of datasets.
StudyCards
==> StudyCardAssignment
Examinations => DatasetAcquisitions
Related datasets (update datasetId and studyId)
Add extra-data files
Solr has to be declenched too (shanoir_metadata to be looked at ? I think it is useless but..?)
TECHNICAL DECISIONS:
To connect to a distant keycloak: 1) Declare local Shanoir as client in distant keycloak 2) Allow offline session and token settings in realm settings 3) Allow user to get offline_token for this specific client (user, Role mapping) 4) Export keycloack.json containing data about keycloak (see shanoit-uploader) -> Only name displayed in front 5) Prepare a link to be used in front to get this
The user will have to communicate its login/password/id of the distant Shanoir
To communicate between the Microservices
OLD Pre thinking. Moving data to GME, how to do this properly ? Many possibilities:
1) Copy all the database (dump) then remove one by on all the unneeded studies from interface
=> Can we bring a hard disk with all the content to the GME? / How to transfer the data ? => Can we (even temporarily) copy all the ‘private’ data of Neurinfo ? => How to copy data from the PACS ? Is it possible to make a dump too ? (or use a dcmtk tool ?) => If we delete a study, all data deleted ? (in PACS = NO I guess (this has to be corrected btw), but if the data is not here it’s not a pb)
2) Programatically, using the API/interfaces
=> Create a little program that interrogates shanoir-neurinfo on choosen study, then recreates the study in GME with all its elements.
=> Difficult to list ALL elements to recreate => Maybe use shanoir uploader? => Will we use import interface then ? What about re-anonymization ?
3) Use some kind of BIDS archive that we can export subject by subject then re-import directly into GME
=> Problem of re-anonymization as we redo an import ? => Not BIDS as we want to work with dicom essentially => Recreate manually all studies / study card / centers ? => How to do this automatically (export then re-import)
4) Create a huge SQL request / procedure to move data
=> How to move files ? => How to move PACS ? => The list of all data will be enormous
5) Mixed solution
Pertinent questions: => How do we move dicom in PACS ? Can we make PACS communicate => create an alternative PACS that’ll be dupliacted from neurinfo with choosen studies ? (or use a dcmtk tool ?) Possible to make a dump too ?
That would mean opening shanoir’s/GME's PACS to the world (temporarly)?
=> How do we move files on server if not programaticaly or with hard drive ?
Lots of questions here, not a lot of answers => to be discussed !