Open markpanganiban opened 1 month ago
As discussed with Ben, I tested this PR on Platform but the AWS Secrets are not being detected.
Using slurm and local nextflow CLI, the secrets/pipeline secrets are working.
Nextflow should be able to parse the config coming from AWS SecretsManager when used in IncludeConfig context.
On your repository's GitHub nextflow.config, add the following.
includeConfig "https://${secrets.PAT}@raw.githubusercontent.com/markpanganiban/nf-test/master/ubuntu.config"
Make sure that you create the PAT secret with your GitHub token. Select the secrets on Pipeline Secrets menu.
May-30 20:56:00.760 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /.nextflow/secrets/store.json May-30 20:56:00.762 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@664e848c] - activable => nextflow.secret.LocalSecretsProvider@664e848c May-30 20:56:00.771 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard` May-30 20:56:00.837 [main] TRACE nextflow.file.FileSystemPathFactory - File system path factories: [] May-30 20:56:00.838 [main] TRACE nextflow.file.FileSystemPathFactory - File system path factories: [] May-30 20:56:00.839 [main] TRACE nextflow.config.ConfigBase - Include config file: https://null@raw.githubusercontent.com/markpanganiban/nf-test/master/ubuntu.config [parent: /.nextflow/assets/markpanganiban/nf-test/nextflow.config] May-30 20:56:00.841 [main] TRACE n.file.http.XFileSystemProvider - File remote URL: https://null@raw.githubusercontent.com/markpanganiban/nf-test/master/ubuntu.config May-30 20:56:00.892 [main] DEBUG n.file.http.XFileSystemProvider - Got HTTP error=404 waiting for 250ms (attempt=1) May-30 20:56:01.158 [main] DEBUG n.file.http.XFileSystemProvider - Got HTTP error=404 waiting for 750ms (attempt=2) May-30 20:56:01.922 [main] DEBUG n.file.http.XFileSystemProvider - Got HTTP error=404 waiting for 2250ms (attempt=3) May-30 20:56:04.188 [main] TRACE nextflow.file.FileSystemPathFactory - File system path factories: [] May-30 20:56:04.194 [main] ERROR nextflow.cli.Launcher - Unable to parse config file: '/.nextflow/assets/markpanganiban/nf-test/nextflow.config' Config file does not exist: https://null@raw.githubusercontent.com/markpanganiban/nf-test/master/ubuntu.config java.nio.file.NoSuchFileException: Config file does not exist: https://null@raw.githubusercontent.com/markpanganiban/nf-test/master/ubuntu.config at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321) at nextflow.config.ConfigBase.memoizedMethodPriv$readConfigFilePath(ConfigBase.groovy:145) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
The log reports the local secret provider was used, not the amazon one. It may be caused by a configuration error
Bug report
As discussed with Ben, I tested this PR on Platform but the AWS Secrets are not being detected.
Using slurm and local nextflow CLI, the secrets/pipeline secrets are working.
Expected behavior and actual behavior
Nextflow should be able to parse the config coming from AWS SecretsManager when used in IncludeConfig context.
Steps to reproduce the problem
On your repository's GitHub nextflow.config, add the following.
Make sure that you create the PAT secret with your GitHub token. Select the secrets on Pipeline Secrets menu.
Program output
Environment