search

tianocore / edk2-pytool-extensions

Extensions to the edk2 build system allowing for a more robust and plugin based build system and tool execution environment
Other
60 stars 41 forks source link

Some cloning errors are still not being propagated during setup #300

Closed corthon closed 1 year ago

corthon commented 2 years ago

Describe the bug Sometimes errors when cloning a primary module or submodule fail to propagate to the top in the Stuart tools. This causes unexpected behavior later in pipelines and local dev environments when the setup looks successful but wasn't. I believe this is largely around the Repo class and stuart_ci_setup of submodules.

Expected behavior If any clone in a setup or ci_setup step fails, the error should be reported in that step so that the pipeline may halt rather than wasting resources.

Additional context PR #286 attempted to fix some of this, but there are some edge cases that are still missing.

corthon commented 2 years ago

Repro log included below....

INFO - Log Started: Thursday, April 07, 2022 09:52PM
SECTION - Init SDE
WARNING - Using Pip Tools based BaseTools
DEBUG - --- self_describing_environment.__init__()
DEBUG - Skipped directories specified = ()
DEBUG - --- self_describing_environment.load_workspace()
DEBUG - Loading workspace: D:\a\1\s
DEBUG -   Including scopes: cibuild, edk2-build, host-based-test, pipbuild-win, global-win, global
DEBUG - --- self_describing_environment._gather_env_files()
DEBUG - --- self_describing_environment.update_simple_paths()
DEBUG - --- self_describing_environment.update_extdep_paths()
DEBUG - --- self_describing_environment.report_extdep_version()
SECTION - Loading Plugins
SECTION - Start Invocable Tool
DEBUG - Dependencies list [{'Path': 'MU_BASECORE', 'Url': 'https://dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_basecore', 'Branch': 'release/202202'}]
PROGRESS - Syncing MU_BASECORE
INFO - Resolving at D:\a\1\s\MU_BASECORE
INFO - Cloning at D:\a\1\s\MU_BASECORE
PROGRESS - Cloning repo: https://dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_basecore
INFO - Cmd to run is: git clone --branch release/202202 --single-branch --depth=5 --recurse-submodules https://dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_basecore D:\a\1\s\MU_BASECORE
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - Cloning into 'D:\a\1\s\MU_BASECORE'...
INFO - fatal: Cannot prompt because user interactivity has been disabled.
INFO - fatal: Authentication failed for 'https://dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_basecore/'
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:03 ----------
INFO - ----------- Return Code: 0x00000080 ------------
INFO - ------------------------------------------------
ERROR - ERROR CLONING 
INFO - Cmd to run is: git rev-parse --abbrev-ref HEAD
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - HEAD
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git remote
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - origin
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --get remote.origin.url
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git rev-parse HEAD
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - 40cec8b7d35bff0488cc75ecf6f22cc8323efac5
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git status --short
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git log --branches --not --remotes --decorate --oneline
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --get remote.origin.url
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git rev-parse --is-bare-repository
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - false
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --file .gitmodules --get-regexp path
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000001 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git fetch origin
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - fatal: Cannot prompt because user interactivity has been disabled.
INFO - fatal: could not read Password for 'https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus': terminal prompts disabled
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:01 ----------
INFO - ----------- Return Code: 0x00000080 ------------
INFO - ------------------------------------------------
ERROR - fatal: Cannot prompt because user interactivity has been disabled.
fatal: could not read Password for 'https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus': terminal prompts disabled
INFO - Cmd to run is: git checkout release/202202
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - Branch 'release/202202' set up to track remote branch 'release/202202' from 'origin'.
INFO - Previous HEAD position was 40cec8b7d3 Propogate failures
INFO - Switched to a new branch 'release/202202'
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git submodule update --init --recursive
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:01 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git rev-parse --abbrev-ref HEAD
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - release/202202
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git remote
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - origin
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --get remote.origin.url
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git rev-parse HEAD
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - f9bafcbcf04d35b4c17d7d11d1df3f98551b7d21
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git status --short
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git log --branches --not --remotes --decorate --oneline
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --get remote.origin.url
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - https://projectmu@dev.azure.com/projectmu/Mu_Security_Private_Release/_git/mu_tiano_plus
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git rev-parse --is-bare-repository
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - false
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000000 ------------
INFO - ------------------------------------------------
INFO - Cmd to run is: git config --file .gitmodules --get-regexp path
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000001 ------------
INFO - ------------------------------------------------
INFO - Repo resolver resolved ['D:\\a\\1\\s\\MU_BASECORE']
SECTION - Summary
PROGRESS - Success
Javagedes commented 1 year ago

git functionality was overhauled in #407 so this is hopefully not applicable anymore. It can be re-opened or another issue raised if any additional issues are seen.