griddynamics / mpl

[IT-36925] Jenkins Shared Modular Pipeline Library
https://blog.griddynamics.com/developing-a-modular-pipeline-library-to-improve-devops-collaboration/
Apache License 2.0
158 stars 96 forks source link

Helper miss method make job go fail #66

Closed Cheava closed 4 years ago

Cheava commented 4 years ago

When I want to use Module's OUT to add param in CFG, like below,

        steps {
          MPLPipelineConfigMerge(MPLModule())
        }

jenkins's job go failure, the error message is attached below

groovy.lang.MissingMethodException: No signature of method: static com.griddynamics.devops.mpl.Helper.configEntrySet() is applicable for argument types: (java.util.LinkedHashMap) values: [[...]]]

I have view the source code, this error is caused by this line. https://github.com/griddynamics/mpl/blame/master/src/com/griddynamics/devops/mpl/MPLConfig.groovy#L89 The missing method was remove on this #46 . Because the method is missing, I have to uncomment the " Helper.configEntrySet(this.@config) ". Finally, the job run well . But CFG seems no change after merging with OUT.
Can you have a look on this code? And could you tell why CFG seems no change after merging with OUT?

sparshev commented 4 years ago

Hi @Cheava ,

Yeah, you can't merge the entire OUT to the existing pipeline config. Sorry for such mistake - it was not added to the documentation. The method MPLPipelineConfigMerge was designed to be used as:

MPLPipelineConfigMerge(MPLModule().key)

Means you have to put data to specific OUT.key value and after that use such key to access the data and merge it.

The MPLConfig object denies to use config object for iteration - you have to know the first key you need. After that first key will return you dict/array/value or null - which you will be able to use further in your logic. Otherwise the logic in your pipeline will miss the important information - what exactly you doing there.

Also I think I will add some documentation about that and some checks to tell what's exactly wrong.

sparshev commented 4 years ago

About the second part - when the CFG is not changed after merging - could you please describe steps to reproduce? It's quite hard to understand which CFG should be changed.

sparshev commented 4 years ago

Removing of the Helper.configEntrySet method looks wrong, but overall - you will face the same result, beacuse you trying to iterate over MPLConfig object - and such action was intentionally denied.

Cheava commented 4 years ago

Thank you for the answer. I will try the usage you mentioned later. The second question is based on my view of MPLPipelineConfigMerge, I think MPLPipelineConfigMerge is aim to merge OUT and CFG, to update/add some CFG propertity, from OUT. I don't know if I misunderstand the usage.

The code I write like below

### Build.groovy
......
//deliver cutomer  value
OUT.'project.type' = 'Docker'

### Deploy.groovy
......
//check if the key is exist
if(CFG.'project.type' == 'Docker'){
......
}

### NestedPipeline.groovy
.....
   stage( 'Build' ) {
        when { expression { MPLModuleEnabled() } }
        steps {
          MPLPipelineConfigMerge(MPLModule())
        }
      }
   stage( 'Deploy' ) {
        when { expression { MPLModuleEnabled() } }
        steps {
          MPLPipelineConfigMerge(MPLModule())
        }
      }
.....
Cheava commented 4 years ago

I just try the code 'MPLPipelineConfigMerge(MPLModule().key)' as below:

### Build.groovy
......
//deliver cutomer  value
OUT.'project.type' = 'Docker'
OUT.'project.home' = '/usr/local'

### Deploy.groovy
......
echo "${CFG.project}"
......
}

### NestedPipeline.groovy
.....
   stage( 'Build' ) {
        when { expression { MPLModuleEnabled() } }
        steps {
          MPLPipelineConfigMerge(MPLModule().project)
        }
      }
   stage( 'Deploy' ) {
        when { expression { MPLModuleEnabled() } }
        steps {
          MPLPipelineConfigMerge(MPLModule().project)
        }
      }
.....

According to the echo log, CFG.project is 'null' .

sparshev commented 4 years ago

Hi @Cheava

Right, it will be null, because only the value of OUT.project is merged. Means in the CFG you will see CFG.type and CFG.home. Hopefully you got the idea: if you want to see CFG.project in Deploy - you need to put project into OUT.build for example and use it to merge.

Cheava commented 4 years ago

Yeah, I have tried the way you said, It works.
Thank you.