Move config into data-models repo

adamjtaylor commented 9 months ago

We will move the config file looked at by DCA from the data_curator_config repo to the data-models repo.

@elv-sb and I discussed and agree that we will keep here under manual control and not implement its generation in CI.

adamjtaylor commented 9 months ago

Slack discussion here.

https://sagebionetworks.slack.com/archives/C01ANC02U59/p1698873125444109

Need confirmation that DCA will look at a URL in the template_menu_config_file field of https://github.com/Sage-Bionetworks/data_curator_config/blob/main/dcc_config.csv

aclayton555 commented 8 months ago

From Anthony on 2023.12.01 - Not urgent - only the staging DCA has been migrated to the new config setup, so we can check this out. Prioritize this (and providing any feedback to FAIR) through December.

FAIR expecting to migrate configs and update prod in January, so any feedback we provide through December, can be taken into consideration in this process.

aclayton555 commented 8 months ago

Actions:

[x] move config to our repo and updating link to new config file in data curator config staging branch
[x] perform testing (this sprint)
[x] decide how this config is updated. Could leave this as static, with no CI to generate config. If adding a new component, have author update manually.
[x] Longer term/TBD: Have separate issue for automating config generation via CI after JSON generation? Or, at least run a check to compare DIFF between JSON and config without pushing any changes

adamjtaylor commented 8 months ago

Reopening while waiting on this PR to be merged: https://github.com/Sage-Bionetworks/data_curator_config/pull/133

adamjtaylor commented 8 months ago

This is now deployed on the DCA staging instance. It seems to be working (I can select the expected templates in the drop downs)

adamjtaylor commented 8 months ago

decide how this config is updated. Could leave this as static, with no CI to generate config. If adding a new component, have author update manually.

I think we agreed to leave static for now

aclayton555 commented 7 months ago

Discuss during 2024.01.04 mid-sprint - what, if any, action remaining here (and file tickets accordingly). From Anthony on 2024.01.02:

Hi everybody, This Thursday, January 4th, DCA prod will switch to the new json config files. The main branch of data_curator_config will be changed to prod for consistency with DCA. No immediate action is required, though anyone contributing to their config files will need to use the prod branch. Let me know if you have any questions.

adamjtaylor commented 7 months ago

No change needed. The HTAN prod config is pointing towards the same data model version as the main branch

prod branch: https://github.com/Sage-Bionetworks/data_curator_config/blob/4cb12b0367885fd79649d872430d684ad1360494/HTAN/dca_config.json#L5

main branch: https://github.com/Sage-Bionetworks/data_curator_config/blob/ddd0ae98ce9cc38cdb1f24b5ade0ce60c610cabd/dcc_config.csv#L7

aclayton555 commented 7 months ago

Confirm that there are no breaking changes and everything looks okay when we make our next release

adamjtaylor commented 7 months ago

Waiting on https://github.com/Sage-Bionetworks/data_curator_config/pull/139 for our latest version to be deployed with the new prod branch in the data curator config repo

aclayton555 commented 6 months ago

Remaining actions:

is to update config to point to latest release. Filed here: https://github.com/Sage-Bionetworks/data_curator_config/issues/145
also deprecate this file: https://github.com/Sage-Bionetworks/data_curator_config/blob/prod/HTAN/dca-template-config.json

adamjtaylor commented 6 months ago

Waiting on https://github.com/Sage-Bionetworks/data_curator_config/pull/147 to be merged to close this

adamjtaylor commented 6 months ago

Merged!

ncihtan / data-models

Move config into data-models repo #320