Open FALLAI-Denis opened 1 year ago
It seems that the issue is related to the calculation of the access path to the impacting copybook. Have you checked the applicationSrcDirs configuration to ensure that it is correct and points to the correct locations of the copybooks? It is also possible that there is an issue with the impactSearch configuration.
You may want to double-check these configurations and see if they are correctly specified. It might also be helpful to compare the configurations between DBB 1.1.0 and DBB 1.1.3 to see if there are any changes that could be causing the issue.
I share your guess, @FALLAI-Denis, that the root cause is in the identification of the relative path of the modified file.
To calculate the correct offset is tricky - especially when you provide multiple applicationSrcDirs.
We had added another scenario about 10 months ago. Please review the git blame for the fixGitDiffPath
method within the ImpactUtilities - you might need to add some additional tracing, where it fails.
It can be a scripting issue with the new scenario or a change in the groovy runtime (DBB 1.1.3 includes Groovy 4.0).
https://github.com/IBM/dbb-zappbuild/blame/2.5.2/utilities/ImpactUtilities.groovy#L993
Hi,
Adding some "println" in fixGitDiffPath
:
*** Changed files for directory /SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230106-102324/master/ceab3-environn-outinfr-central-mvp2/.imports/ENVIRONN :
DEBUG>>> file=src/bloc/COPYCOB/S9FDI902.cpy, dir=/SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230106-102324/master/ceab3-environn-outinfr-central-mvp2/.imports/ENVIRONN, mustExist=true, mode=null
DEBUG>>> Scenario 1
DEBUG>>> relPath=ceab3-environn-outinfr-central-mvp2/.imports/ENVIRONN/
DEBUG>>> fixedFileName=src/bloc/COPYCOB/S9FDI902.cpy
DEBUG>>> defaultValue=src/bloc/COPYCOB/S9FDI902.cpy
DEBUG>>> props.workspace & fixedFileName=/SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230106-102324/master/src/bloc/COPYCOB/S9FDI902.cpy
DEBUG>>> Scenario 2
DEBUG>>> dirName=ENVIRONN
DEBUG>>> dir & file=/SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230106-102324/master/ceab3-environn-outinfr-central-mvp2/.imports/ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy
DEBUG>>> sortie Scenario 2 no mode en ligne 1038, result=[ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy, 2]
**** ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy
I think the case where external projects (External Git Repositories) are stored inside the main project tree (Main Git Repository) is not provided for in the xxxx routine. Scenario 2 is wrongly selected and does not correspond to our use case.
In terms of folder organization we have:
${workspace}/${application}
: Main Git repository (program)${workspace}/${application}/.imports/xxxx
: External Git repository (copybooks)This organization is similar to that of Git submodules (but with a proprietary implementation): Git Tools - Submodules
add a check at the beginning of scenario 2 to see if the .imports directory is present in the file path. If it is, you can continue with scenario 2 as it is implemented.
if (dir.contains(".imports")) {
// Continue with scenario 2 as it is implemented
} else {
return [props.workspace + fixedFileName, null]
}
If it is not, you can return the path constructed using props.workspace + fixedFileName instead, as this should be the correct path for a file in the main project tree.
def fixGitDiffPath(file, dir, mustExist, mode) {
// Check if the .imports directory is present in the file path
if (dir.contains(".imports")) {
// Scenario 1
def relPath = dir.substring(props.workspace.length())
def fixedFileName = relPath + file
def defaultValue = file
// Check if the file exists at the path props.workspace + fixedFileName
if (new File(props.workspace + fixedFileName).exists()) {
return [fixedFileName, null]
}
// Scenario 2
def dirName = dir.substring(props.workspace.length() + 1, dir.length() - 1)
def result = [dirName + '/' + file, 2]
if (mode != null) {
result << mode
}
return result
} else {
// Return the path constructed using props.workspace + fixedFileName
// This should be the correct path for a file in the main project tree
return [props.workspace + file, null]
}
}
Hi,
We applied a solution similar to the one proposed: hard test the presence of ".imports" in the path, which is specific to our use case... It worked.
// Scenario 2: Repository name is used as Application Root directory
// Relativisation du path par rapport au workspace
String dirName
if (dir.contains("/.imports/")) {
dirName = dir - (props.workspace + '/')
}else{
dirName = new File(dir).getName()
}
if (new File("${dir}/${file}").exists()) {
return [
"$dirName/$file" as String,
2
]
}
However, I think that there should be a thorough reflection and a solution that takes into account the use of several nested Git repositories (use case of submodules in Git).
@FALLAI-Denis, One approach you could take is to use the git ls-files command to list the files in the Git repository and then parse the output to determine the correct file path.
def fixGitDiffPath(file, dir, mustExist, mode) {
// Run the 'git ls-files' command to list the files in the repository
def command = "git ls-files --full-name"
def process = command.execute()
process.waitFor()
def output = process.in.text
// Parse the output to determine the correct file path
def lines = output.split("\n")
def filePath
for (line in lines) {
if (line.endsWith(file)) {
filePath = line
break
}
}
if (filePath) {
return [filePath, null]
} else {
return [file, mode]
}
}
@FALLAI-Denis You are correct, Scenario 2 is not correctly selected, but the criteria for this scenario is met.
Your scenario sound actually to be Scenario 4. I think, you would just need to reorder the scenarios and move Scenario 4 before Scenario 2. Would you mind to try this?
Hi,
I will try to make a diagram to explain the organization of our Workspace for DBB by positioning the different variables used by the zAppbuild scripts.
This will allow you to understand our expectation and possibly adapt the fixGitDiffPath
function accordingly.
Hi,
We have been running DBB 1.1.0 for a while and have just upgraded to DBB 1.1.3. zAppbuild has been upgraded with the 2.5.2 package.
fullBuild
andimpactBuild
worked fine in DBB 1.1.0 and zAppbuid before 2.5.2. With version DBB 1.1.3 and zAppbuid 2.5.2 thefullBuild
works, but theimpactBuild
triggers an error on the calculation of an access path to an impacting copybook.In a Repository to build, we have the following configuration:
The subfolders
ENVIRONN
andGCL
under${application}/.imports
are actually imported Git Repositories containing the copybooks used by the COBOL programs of the main Git Repository corresponding to${application}
. We are obliged to declare them inapplicationSrcDirs
so that theimpactBuild
detects the changes on the contained copybooks.A copybook located in a subfolder of
${application}/.imports/ENVIRONN
is modified. This copybook impacts a COBOL source located in a subfolder of${application}/src
and which by itself is not modified.Command executed:
groovyz /var/DBB/MAINT/SCRIPTS/buildBPCE.groovy -w /SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230105-181754/master -a ceab3-environn-outinfr-central-mvp2 -h GTEMP.JENKINS.CEAB3.MVP2.M -o /var/DBB/MAINT/LOGS/dbb-logs --impactBuild --verbose
zAppbuild detects the change on the copybook in
${application}/.imports/ENVIRONN
:At this stage, the path to the copybook should have been:
.imports/ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy
, notENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy
? Is this the cause of the error that occurs afterwards?A copybook usage impact search is triggered, but it throws an error in the routine
SearchPathImpactFinder
:In the detail of the stack, we see that the access path to the copybook has been incorrectly calculated:
/SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230105-181754/master/ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy
instead of:/SYSTEM/var/tmp/DBB/JENKINS/workspace/CEH49-CEAB3-20230105-181754/master/.imports/ENVIRONN/src/bloc/COPYCOB/S9FDI902.cpy