brain-bican / models

BICAN data models
https://brain-bican.github.io/models/
3 stars 3 forks source link

Csv convert for greentable format #21

Closed djarecka closed 2 months ago

djarecka commented 8 months ago

I've modified the converter from #19 to work for the examples of the Green Table (for now I only updated cvs2yaml.

I've added also an example of cvs files and the header that could be used to run it: python utils/csv2yaml.py --csv csv_test/GreenTable.csv --enum csv_test/GreenTable_ValueSet.csv --header csv_test/head.yaml --output csv_test/gt_test.yaml

The output should look like csv_test/GreenTable_model.yaml

I've slightly modified the GreenTable format, see here (note that I only worked on Generate Library for now). The changes are related to the Value Sets and I've created a new tab: Generate Library Value Sets (you can compare with Value Sets Orig). The original tab for Value Sets was hard to automatically convert to pandas.DF, so I've added column Field Name to remove the titles of previous multiple columns. In order to properly link ValueSets from the Generate Library to the values for specific Field Name, I assumed that it is either the original Field Name from the Generate Library tab or DataType has the name of the set values, e.g. ValueSet: barcode name.

Issues with Value Set that need feedback:

Other issues to discuss:

Other TODOs (that do not require feedback):

FYI. I will look into the code from the Yale team, I only opened the output quickly and I noticed that there are some things that we discussed done differently, so I decided to finish this PR to help with the discussion.

djarecka commented 8 months ago

@satra @lydiang @patrick-lloyd-ray - if you have time this week you can give me some feedbacks, especially on the issues listed in the PR description. If not we will discuss it later

djarecka commented 7 months ago

notes from the discussion with Pam at the SFN:

patrick-lloyd-ray commented 7 months ago

@djarecka could you check the integer data types here: https://github.com/brain-bican/metadata-schemas/pull/31 and make sure we are accurate?

djarecka commented 7 months ago

@patrick-lloyd-ray - yes I'm planning to check it, I forgot that this is in PR and that's probably better place to add more comments

patrick-lloyd-ray commented 7 months ago

@patrick-lloyd-ray - yes I'm planning to check it, I forgot that this is in PR and that's probably better place to add more comments

Great! Feel free to simply suggest changes directly on the PR.