search

nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.7k stars 621 forks source link

Using AWS secrets in `includeConfig` doesn't work #5312

Open bentsherman opened 3 days ago

bentsherman commented 3 days ago

We added the ability to use secrets in the includeConfig path in #4840 . This works for local secrets, but not AWS secrets, because AWS secrets requires the xpack-amzn plugin and the config is resolved before plugins are loaded, since the config may include additional plugins.

To fix this I think we would need to load plugins in two phases, adding an "early" phase which happens before the config is resolved.

Not sure how much demand there is for this use case, but it would include any AWS Batch run in Seqera Platform that loads a private config using a secret credential.

cc @markpanganiban

pditommaso commented 3 days ago

Bit ugly, because plugins depends on the config 🤦