nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.61k stars 606 forks source link

Nested config parameters merge/overwrite each other #5006

Open kverstae opened 1 month ago

kverstae commented 1 month ago

Bug report

When defining the same parameter object structure multiple times in different scope, the values will overwrite (when keys match) or merge (when keys are unique for one of the 2 objects). This seems very similar to issues reported earlier and fixed in #2485 a while ago. It is indeed fixed for those reported cases, but if you nest one level deeper the issue seems to return.

Expected behavior and actual behavior

Given following config:

params {
  foo {
    bar = 'bar1'
    baz {
        wrong = 'root'
    }
  }
  nested {
    foo {
      bar = 'bar2'
      baz {
        wrong = 'nested'
        only_nested = 'value'
      }
    }
  }
}

Expected: the value foo.baz.wrong equals root and foo.baz only contains 1 key: wrong Actual: the value foo.baz.wrong equals nested and foo.baz contains 2 keys: wrong and only_nested

Steps to reproduce the problem

Store config from above in nextflow.config and create a main.nf:

workflow {
    main:
        println "$params"
}

Run nextflow run . and observe the wrong output

Program output

 N E X T F L O W   ~  version 24.04.0-edge

Launching `./main.nf` [sick_bohr] DSL2 - revision: d674ace2a4

[foo:[bar:bar1, baz:[wrong:nested, only_nested:value]], nested:[foo:[bar:bar2, baz:[wrong:nested, only_nested:value]]]]

Environment

Additional info

A clearer example of the merging to illustrate that it doesn't just overwrite:

params {
  foo {
    bar {
        key1 = 'value1'
    }
  }
  nested {
    foo {
      bar {
        key2 = 'value2'
      }
    }
  }
}

Gives as output:

 N E X T F L O W   ~  version 24.04.0-edge

Launching `./main.nf` [magical_rubens] DSL2 - revision: d674ace2a4

[foo:[bar:[key1:value1, key2:value2]], nested:[foo:[bar:[key1:value1, key2:value2]]]]
bentsherman commented 1 month ago

This should be solved by #4744