non-default config projects are not read

[afwillia]

Only the default config.yml settings are being read, despite setting R_CONFIG_ACTIVE to "amp-ad".

[danlu1]

For my usecase of setting up 1kD_dccmonitor, this issue happened when I called dccvalidator::gather_template_ids. The get_golem_config in gather_template_ids refers to the config file of released dccvalidator app. I guess the reason is that the app_sys() reset the app's config file from dccmonitor to dccvalidator. See the following screenshots.

[danlu1]

This screenshot shows that in the config::get() used by get_golem_config, file value has been updated to dccvalidator/config.yml instead of the one in dccmonitor.

This shows the loaded config files. It only includes the config of the released version of dccvalidator (not includes the most recent changes in github repo)

Take pec for example.

[danlu1]

I believe we have 3 approaches to solve this problem. 1. (quickest fix) save the df_helpers script in dccmonitor repo and call gather_template_ids directly. By doing this, we can make sure the app calls correct config file and not rely on the synchrony of config files in dccvalidator and dccmonitor. 2. synchronize the config files between dccmonitor and released/deployed dccvalidator. 3. modify the app_sys and allow it to refer to the current app instead of a predefined app.

[ychae]

Will need to look into to see how much time it will take to solve - 1 day timebox AC: app reads config values from non-default projects

[afwillia]

I think updating the app_sys function like @danlu1 suggested will be the most reliable way forward. I can push a fix this sprint @ychae

[danlu1]

The same issue happened when running check_cols_complete from dccvalidator::check_all().

The get_golem_config get dccvalidator default config. The default required column (individualID) in individual file has been checked instead of what listed in the config for 1kD.

The dccmonitor is not generating the error that generated by dccvalidator: There is missing data in some columns that are required to be complete in the individual metadata.

[afwillia]

@ychae on second thought after trying to update the app_sys() function, this will require a bit more thought. @danlu1 Syncing the dccvalidator config file so it matches dccmonitor should fix this issue.

[danlu1]

@afwillia I think we might need to deploy the latest version of dccvalidator and update renv.lock in dccmonitor in order to get the most recent config file in dccvalidator including 1kD section. Also, I suggest we can have a regular CI/CD pipeline so we can include most recent changes in dccvalidator.