search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.5k stars 2.61k forks source link

[BUG]: PythonScript@0 is stuck during pipeline execution #19201

Closed KristofKuli closed 12 months ago

KristofKuli commented 1 year ago

Task name

PythonScript@0

Task version

0.229.3

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 20.04

Task log

2023-11-02T07:55:25.7654254Z ##[debug]Evaluating condition for step: 'PythonScript'
2023-11-02T07:55:25.7655134Z ##[debug]Evaluating: SucceededNode()
2023-11-02T07:55:25.7655539Z ##[debug]Evaluating SucceededNode:
2023-11-02T07:55:25.7656027Z ##[debug]=> True
2023-11-02T07:55:25.7656484Z ##[debug]Result: True
2023-11-02T07:55:25.7656781Z ##[section]Starting: PythonScript
2023-11-02T07:55:25.7678821Z ==============================================================================
2023-11-02T07:55:25.7679121Z Task         : Python script
2023-11-02T07:55:25.7679260Z Description  : Run a Python file or inline script
2023-11-02T07:55:25.7679467Z Version      : 0.229.3
2023-11-02T07:55:25.7679563Z Author       : Microsoft Corporation
2023-11-02T07:55:25.7679729Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/python-script
2023-11-02T07:55:25.7680106Z ==============================================================================
2023-11-02T07:55:26.6185152Z ##[debug]Using node path: /vsts/agent/externals/node16/bin/node
2023-11-02T07:55:27.5941432Z ##[debug]agent.TempDirectory=/workspace/_temp
2023-11-02T07:55:27.5955668Z ##[debug]loading inputs and endpoints
2023-11-02T07:55:27.5959307Z ##[debug]loading INPUT_SCRIPTSOURCE
2023-11-02T07:55:27.5968621Z ##[debug]loading INPUT_SCRIPTPATH
2023-11-02T07:55:27.5970293Z ##[debug]loading INPUT_WORKINGDIRECTORY
2023-11-02T07:55:27.5971443Z ##[debug]loading INPUT_FAILONSTDERR
2023-11-02T07:55:27.5972639Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2023-11-02T07:55:27.5973880Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2023-11-02T07:55:27.5975120Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2023-11-02T07:55:27.5979829Z ##[debug]loading SECRET_SV_NSSO_PWD
2023-11-02T07:55:27.5981551Z ##[debug]loading SECRET_SV_PWD
2023-11-02T07:55:27.5983651Z ##[debug]loading SECRET_SV_NSSO_UID
2023-11-02T07:55:27.5985921Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2023-11-02T07:55:27.5988703Z ##[debug]loading SECRET_SV_SRUID
2023-11-02T07:55:27.5991004Z ##[debug]loaded 12
2023-11-02T07:55:27.5994342Z ##[debug]Agent.ProxyUrl=http://***.***.com:8080
2023-11-02T07:55:27.5994871Z ##[debug]Agent.ProxyUsername=undefined
2023-11-02T07:55:27.5995285Z ##[debug]Agent.ProxyPassword=undefined
2023-11-02T07:55:27.5996287Z ##[debug]Agent.ProxyBypassList=["localhost","127\\.0\\.0\\.1","\\.***\\.com","\\.***\\.***\\.com","\\.corp\\.gwpnet\\.com","\\.***apps\\.com","\\.***apps-np\\.com","\\.***apps-ee\\.com"]
2023-11-02T07:55:27.5997768Z ##[debug]expose agent proxy configuration.
2023-11-02T07:55:27.5998203Z ##[debug]Agent.CAInfo=undefined
2023-11-02T07:55:27.5998605Z ##[debug]Agent.ClientCert=undefined
2023-11-02T07:55:27.5999015Z ##[debug]Agent.SkipCertValidation=undefined
2023-11-02T07:55:27.6079901Z ##[debug]check path : /workspace/_tasks/PythonScript_6392f95f-7e76-4a18-b3c7-7f078d2f7700/0.229.3/task.json
2023-11-02T07:55:27.6085535Z ##[debug]adding resource file: /workspace/_tasks/PythonScript_6392f95f-7e76-4a18-b3c7-7f078d2f7700/0.229.3/task.json
2023-11-02T07:55:27.6086354Z ##[debug]system.culture=en-US
2023-11-02T07:55:27.6106329Z ##[debug]scriptSource=filePath
2023-11-02T07:55:27.6108609Z ##[debug]scriptPath=/workspace/1/s/res-py-configurator/core/py/UtilsAdoDailyMasterDataSync.py
2023-11-02T07:55:27.6109285Z ##[debug]script=undefined
2023-11-02T07:55:27.6109640Z ##[debug]arguments=undefined
2023-11-02T07:55:27.6110336Z ##[debug]pythonInterpreter=undefined
2023-11-02T07:55:27.6112753Z ##[debug]workingDirectory=/workspace/1/s/res-py-configurator/core/py
2023-11-02T07:55:27.6113377Z ##[debug]failOnStderr=false
2023-11-02T07:55:27.6120798Z ##[debug]which 'python'
2023-11-02T07:55:27.6123950Z ##[debug]found: '/workspace/_tool/Python/3.10.12/x64/bin/python'
2023-11-02T07:55:27.6124684Z ##[debug]which '/workspace/_tool/Python/3.10.12/x64/bin/python'
2023-11-02T07:55:27.6125797Z ##[debug]found: '/workspace/_tool/Python/3.10.12/x64/bin/python'
2023-11-02T07:55:27.6127003Z ##[debug]/workspace/_tool/Python/3.10.12/x64/bin/python arg: /workspace/1/s/res-py-configurator/core/py/UtilsAdoDailyMasterDataSync.py
2023-11-02T07:55:27.6130022Z ##[debug]exec tool: /workspace/_tool/Python/3.10.12/x64/bin/python
2023-11-02T07:55:27.6130567Z ##[debug]arguments:
2023-11-02T07:55:27.6131080Z ##[debug]   /workspace/1/s/res-py-configurator/core/py/UtilsAdoDailyMasterDataSync.py
2023-11-02T07:55:27.6133377Z [command]/workspace/_tool/Python/3.10.12/x64/bin/python /workspace/1/s/res-py-configurator/core/py/UtilsAdoDailyMasterDataSync.py
2023-11-02T07:55:28.6769281Z environ({'dotnet_SDK': '/usr/bin/dotnet', 'az': '/usr/bin/az', 'REQUESTS_CA_BUNDLE': '/etc/ssl/certs/ca-certificates.crt', 'ant': '/usr/bin/ant', '_': './bin/Agent.Listener', 'kubectl': '/usr/bin/kubectl', 'HTTP_PROXY': 'http://***.***.com:8080', 'dotnet_SDK_6.0': '/usr/bin/dotnet', 'Cloud_Foundry_CLI': '/usr/bin/cf', 'dotnet_SDK_3.1': '/usr/bin/dotnet', 'IBM_JAVA_8_SDK': '/opt/sr-ibm-java-sdk-8.0.6.30_01', 'maven': '/usr/share/maven/bin/mvn', 'JAVA_HOME_8_X64': '/usr/lib/jvm/java-8-openjdk-amd64', 'NO_PROXY': 'localhost,127.0.0.1,.***.com,.***.***.com,.corp.gwpnet.com,.***apps.com,.***apps-np.com,.***apps-ee.com', 'VSTS_AGENT': '***', 'DOCKER_CERT_PATH': '/certs/client', 'PowerShell_v7': '/usr/bin/pwsh', 'java': '/usr/bin/java', 'MAVEN_HOME': '/usr/share/maven', 'VSO_AGENT_IGNORE': '_,MAIL,OLDPWD,PATH,PWD,VSTS_AGENT,VSTS_ACCOUNT,VSTS_TOKEN_FILE,VSTS_TOKEN,VSTS_POOL,VSTS_WORK,VSO_AGENT_IGNORE', 'JAVA_HOME_17_X64': '/usr/lib/jvm/java-17-openjdk-amd64', 'dotnet': '/usr/bin/dotnet', 'https_proxy': 'http://***.***.com:8080', 'PowerShellCore': '/usr/bin/pwsh', 'VSTS_WORK': '/workspace', 'M2_HOME': '/usr/share/maven', 'PWD': '/vsts/agent', 'DOCKER_HOST_IP': '192.168.0.1', 'NODE_EXTRA_CA_CERTS': '/usr/local/share/ca-certificates/npm-chain.pem', 'DOCKER_HOST': 'tcp://localhost:2376', 'ANT_HOME': '/opt/apache-ant-1.10.14', 'NODEJS': '/usr/bin/node', 'node': '/usr/bin/node', 'DEBIAN_FRONTEND': 'noninteractive', 'docker-compose': '/usr/local/bin/docker-compose', 'TF_BUILD': 'True', 'Agent.Type': 'OneTimeAgent', 'powershell': '/usr/bin/pwsh', 'VSTS_POOL': 'Default-Linux', 'PATH': '/workspace/_tool/Python/3.10.12/x64/bin:/workspace/_tool/Python/3.10.12/x64:/opt/oracle/instantclient_19_14:/opt/mssql-tools/bin:/opt/go1.17.6/bin/:/opt/mage/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HTTPS_PROXY': 'http://***.***.com:8080', 'SHLVL': '0', 'npm': '/usr/bin/npm', 'HOME': '/vsts', 'HOSTNAME': '3b16618b01c9', 'JAVA_HOME': '/usr/lib/jvm/java-8-openjdk-amd64', 'JAVA_HOME_11_X64': '/usr/lib/jvm/java-11-openjdk-amd64', 'VSTS_ACCOUNT': '***', 'http_proxy': 'http://***.***.com:8080', 'no_proxy': 'localhost,127.0.0.1,.***.com,.***.***.com,.corp.gwpnet.com,.***apps.com,.***apps-np.com,.***apps-ee.com', 'DOCKER_TLS_VERIFY': '1', 'SV_SRUID': '***', 'SV_PWD': '***', 'SV_NSSO_UID': '***', 'SV_NSSO_PWD': '***', 'INPUT_SCRIPT': '', 'INPUT_ARGUMENTS': '', 'INPUT_PYTHONINTERPRETER': '', 'ENDPOINT_URL_SYSTEMVSSCONNECTION': 'https://***.visualstudio.com/', 'AGENT_ENABLE_PIPELINEARTIFACT_LARGE_CHUNK_SIZE': 'true', 'SYSTEM_TEAMFOUNDATIONCOLLECTIONURI': 'https://***.visualstudio.com/', 'USE_MSAL': 'true', 'AGENT_VERSION': '3.227.2', 'BUILD_SOURCEBRANCHNAME': 'master', 'VSTS_PROCESS_LOOKUP_ID': 'vsts_2853cad9-1720-4bb4-80da-73465ddbe18c', 'SYSTEM_JOBPOSITIONINPHASE': '1', 'BUILD_REASON': 'Manual', 'AGENT_CLOUDID': '', 'SYSTEM_TEAMPROJECT': 'RESAPP-RESPlatform', 'AGENT_PROXYURL': 'http://***.***.com:8080', 'SYSTEM_JOBIDENTIFIER': 'Job.__default', 'SYSTEM_TIMELINEID': 'be313c2b-3473-436f-b401-122ade3dbc23', 'SYSTEM_TASKINSTANCENAME': 'PythonScript', 'BUILD_BUILDID': '7442040', 'SYSTEM_PHASEDISPLAYNAME': 'Job', 'BUILD_QUEUEDBYID': 'a84c2c70-e6c9-63f2-b20a-4a376eec0443', 'SYSTEM_PHASEID': '3a3a2a60-14c7-570b-14a4-fa42ad92f52a', 'AGENTCAPABILITYSKIPYAMLDECORATOR': '', 'VSTSAGENT_CONTINUE_AFTER_CANCEL_PROCESSTREEKILL_ATTEMPT': 'true', 'SYSTEM_COLLECTIONID': 'e99f0146-bd0f-443b-9865-7bc4fe807b64', 'SYSTEM_JOBID': '12f1170f-54f2-53f3-20dd-22fc7dff55f9', 'SYSTEM_JOBDISPLAYNAME': 'Job', 'RESOURCES_TRIGGERINGCATEGORY': '', 'BUILD_SOURCESDIRECTORY': '/workspace/1/s', 'AGENT_OS': 'Linux', 'SYSTEM_TASKINSTANCEID': '840007cc-f097-5b1c-e29a-3ed6decfdf4b', 'AGENT_PROXYBYPASSLIST': '["localhost","127\\\\.0\\\\.0\\\\.1","\\\\.***\\\\.com","\\\\.***\\\\.***\\\\.com","\\\\.corp\\\\.gwpnet\\\\.com","\\\\.***apps\\\\.com","\\\\.***apps-np\\\\.com","\\\\.***apps-ee\\\\.com"]', 'AGENT_BUILDDIRECTORY': '/workspace/1', 'AGENT_ID': '269139', 'AGENT_HOMEDIRECTORY': '/vsts/agent', 'AGENT_READONLYVARIABLES': 'true', 'BUILD_SOURCEVERSIONAUTHOR': '***', 'BUILD_REPOSITORY_ID': '69207665-f290-4fef-a6b8-31fd53ebaadd', 'MSDEPLOY_HTTP_USER_AGENT': 'VSTS_e99f0146-bd0f-443b-9865-7bc4fe807b64_build_33899_0', 'COMMON_TESTRESULTSDIRECTORY': '/workspace/1/TestResults', 'SYSTEM_JOBNAME': '__default', 'GIT_TERMINAL_PROMPT': '0', 'SYSTEM_POSTLINESSPEED': '500', 'BUILD_ARTIFACTSTAGINGDIRECTORY': '/workspace/1/a', 'BUILD_REQUESTEDFOREMAIL': '***@***.com', 'SYSTEM_WORKFOLDER': '/workspace', 'SYSTEM_TOTALJOBSINPHASE': '1', 'BUILD_REQUESTEDFORID': 'a84c2c70-e6c9-63f2-b20a-4a376eec0443', 'SYSTEM_DEFINITIONNAME': 'config-admin-daily-master-data-sync', 'AZURE_HTTP_USER_AGENT': 'VSTS_e99f0146-bd0f-443b-9865-7bc4fe807b64_build_33899_0', 'BUILD_STAGINGDIRECTORY': '/workspace/1/a', 'VSTSAGENT_DOCKER_ACTION_RETRIES': 'true', 'AGENT_ACCEPTTEEEULA': 'False', 'SYSTEM_PHASEATTEMPT': '1', 'AGENT_RETAINDEFAULTENCODING': 'false', 'SYSTEM_CULTURE': 'en-US', 'BUILD_REPOSITORY_GIT_SUBMODULECHECKOUT': 'False', 'BUILD_REPOSITORY_URI': 'https://***.visualstudio.com/RESAPP-RESPlatform/_git/res-platform-conf-qa', 'SYSTEM': 'build', 'SYSTEM_JOBTIMEOUT': '60', 'BUILD_DEFINITIONVERSION': '6', 'AGENT_ROOTDIRECTORY': '/workspace', 'AGENT_JOBNAME': 'Job', 'BUILD_REPOSITORY_NAME': 'res-platform-conf-qa', 'AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN': 'true', 'SYSTEM_ISAZUREVM': '0', 'SYSTEM_TEAMPROJECTID': '7ad18223-7781-48d8-a436-85e20dbde2c9', 'AGENT_TEMPDIRECTORY': '/workspace/_temp', 'SYSTEM_HOSTTYPE': 'build', 'SYSTEM_JOBPARALLELISMTAG': 'Private', 'BUILD_REQUESTEDFOR': '***', 'PIPELINE_WORKSPACE': '/workspace/1', 'AZP_AGENT_IGNORE_VSTSTASKLIB': 'true', 'SYSTEM_STAGENAME': '__default', 'RESOURCES_TRIGGERINGALIAS': '', 'SYSTEM_PIPELINESTARTTIME': '2023-11-02 07:52:50+00:00', 'BUILD_DEFINITIONFOLDERPATH': '\\', 'SYSTEM_COLLECTIONURI': 'https://***.visualstudio.com/', 'SYSTEM_STAGEDISPLAYNAME': '__default', 'SYSTEM_ISSCHEDULED': 'False', 'AGENT_ENABLEADDITIONALMASKINGREGEXES': 'On', 'SYSTEM_STAGEATTEMPT': '1', 'AGENT_RUNMODE': 'once', 'SYSTEM_SERVERTYPE': 'Hosted', 'SYSTEM_ARTIFACTSDIRECTORY': '/workspace/1/a', 'agent.jobstatus': 'Succeeded', 'AGENT_JOBSTATUS': 'Succeeded', 'BUILD_REPOSITORY_PROVIDER': 'TfsGit', 'SYSTEM_JOBATTEMPT': '1', 'AGENT_WORKFOLDER': '/workspace', 'SYSTEM_DEFINITIONID': '33899', 'SYSTEM_TEAMFOUNDATIONSERVERURI': 'https://***.visualstudio.com/', 'BUILD_SOURCEVERSIONMESSAGE': 'Reverting insertion of bash script.', 'BUILD_DEFINITIONNAME': 'config-admin-daily-master-data-sync', 'BUILD_REPOSITORY_LOCALPATH': '/workspace/1/s', 'USEPYTHONVERSION_PYTHONLOCATION': '/workspace/_tool/Python/3.10.12/x64', 'BUILD_REPOSITORY_CLEAN': 'False', 'AGENT_USEWORKSPACEID': 'true', 'AGENT_LOGTOBLOBSTORAGESERVICE': 'true', 'AGENT_TASKRESTRICTIONSENFORCEMENTMODE': 'Enabled', 'SYSTEM_ISDOCKERCONTAINER': '1', 'SYSTEM_STAGEID': '96ac2280-8cb4-5df5-99de-dd2da759617d', 'AGENT_DIAGNOSTIC': 'True', 'AGENT_MACHINENAME': '3b16618b01c9', 'TASK_SKIPTRANSLATORFORCHECKOUT': 'False', 'BUILD_BUILDURI': 'vstfs:///Build/Build/7442040', 'AGENT_ISSELFHOSTED': '1', 'BUILD_SOURCEBRANCH': 'refs/heads/master', 'SYSTEM_TASKDISPLAYNAME': 'PythonScript', 'BUILD_CONTAINERID': '25356812', 'BUILD_QUEUEDBY': '***', 'BUILD_BINARIESDIRECTORY': '/workspace/1/b', 'SYSTEM_PLANID': 'be313c2b-3473-436f-b401-122ade3dbc23', 'SYSTEM_ENABLEACCESSTOKEN': 'SecretVariable', 'BUILD_BUILDNUMBER': '20231102.4', 'BUILD_SOURCEVERSION': '4e0a383732ee03b9c52315519d92bb7c9b2b38dc', 'AZP_75787_ENABLE_COLLECT': 'true', 'TASK_DISPLAYNAME': 'PythonScript', 'SYSTEM_DEBUG': 'True', 'SYSTEM_TASKDEFINITIONSURI': 'https://***.visualstudio.com/', 'AGENT_TOOLSDIRECTORY': '/workspace/_tool', 'SYSTEM_PHASENAME': 'Job', 'AGENT_NAME': '***', 'SYSTEM_PULLREQUEST_ISFORK': 'False', 'AGENT_OSARCHITECTURE': 'X64', 'AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN': 'true', 'SYSTEM_DEFAULTWORKINGDIRECTORY': '/workspace/1/s', 'LC_CTYPE': 'C.UTF-8'})
2023-11-02T08:52:52.5051205Z ##[debug]Re-evaluate condition on job cancellation for step: 'PythonScript'.
2023-11-02T08:52:53.0154269Z ##[error]The operation was canceled.
2023-11-02T08:52:53.0205946Z ##[debug]System.OperationCanceledException: The operation was canceled.
   at System.Threading.CancellationToken.ThrowOperationCanceledException()
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, Boolean continueAfterCancelProcessTreeKillAttempt, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/Agent.Sdk/ProcessInvoker.cs:line 376
   at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, Boolean continueAfterCancelProcessTreeKillAttempt, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/Microsoft.VisualStudio.Services.Agent/ProcessInvoker.cs:line 308
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, Boolean continueAfterCancelProcessTreeKillAttempt, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/Agent.Worker/Handlers/StepHost.cs:line 76
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.NodeHandler.RunAsync() in /home/vsts/work/1/s/src/Agent.Worker/Handlers/NodeHandler.cs:line 214
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync() in /home/vsts/work/1/s/src/Agent.Worker/TaskRunner.cs:line 407
   at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken) in /home/vsts/work/1/s/src/Agent.Worker/StepsRunner.cs:line 238
2023-11-02T08:52:53.0232404Z ##[section]Finishing: PythonScript

Relevant log output

/workspace/_tool/Python/3.10.12/x64/bin/python /workspace/1/s/res-py-configurator/core/py/UtilsAdoDailyMasterDataSync.py
environ({'dotnet_SDK': '/usr/bin/dotnet', 'az': '/usr/bin/az', 'REQUESTS_CA_BUNDLE': '/etc/ssl/certs/ca-certificates.crt', 'ant': '/usr/bin/ant', '_': './bin/Agent.Listener', 'kubectl': '/usr/bin/kubectl', 'HTTP_PROXY': 'http://***.***.com:8080', 'dotnet_SDK_6.0': '/usr/bin/dotnet', 'Cloud_Foundry_CLI': '/usr/bin/cf', 'dotnet_SDK_3.1': '/usr/bin/dotnet', 'IBM_JAVA_8_SDK': '/opt/sr-ibm-java-sdk-8.0.6.30_01', 'maven': '/usr/share/maven/bin/mvn', 'JAVA_HOME_8_X64': '/usr/lib/jvm/java-8-openjdk-amd64', 'NO_PROXY': 'localhost,127.0.0.1,.***.com,.***.***.com,.corp.gwpnet.com,.***apps.com,.***apps-np.com,.***apps-ee.com', 'VSTS_AGENT': '***', 'DOCKER_CERT_PATH': '/certs/client', 'PowerShell_v7': '/usr/bin/pwsh', 'java': '/usr/bin/java', 'MAVEN_HOME': '/usr/share/maven', 'VSO_AGENT_IGNORE': '_,MAIL,OLDPWD,PATH,PWD,VSTS_AGENT,VSTS_ACCOUNT,VSTS_TOKEN_FILE,VSTS_TOKEN,VSTS_POOL,VSTS_WORK,VSO_AGENT_IGNORE', 'JAVA_HOME_17_X64': '/usr/lib/jvm/java-17-openjdk-amd64', 'dotnet': '/usr/bin/dotnet', 'https_proxy': 'http://***.***.com:8080', 'PowerShellCore': '/usr/bin/pwsh', 'VSTS_WORK': '/workspace', 'M2_HOME': '/usr/share/maven', 'PWD': '/vsts/agent', 'DOCKER_HOST_IP': '192.168.0.1', 'NODE_EXTRA_CA_CERTS': '/usr/local/share/ca-certificates/npm-chain.pem', 'DOCKER_HOST': 'tcp://localhost:2376', 'ANT_HOME': '/opt/apache-ant-1.10.14', 'NODEJS': '/usr/bin/node', 'node': '/usr/bin/node', 'DEBIAN_FRONTEND': 'noninteractive', 'docker-compose': '/usr/local/bin/docker-compose', 'TF_BUILD': 'True', 'Agent.Type': 'OneTimeAgent', 'powershell': '/usr/bin/pwsh', 'VSTS_POOL': 'Default-Linux', 'PATH': '/workspace/_tool/Python/3.10.12/x64/bin:/workspace/_tool/Python/3.10.12/x64:/opt/oracle/instantclient_19_14:/opt/mssql-tools/bin:/opt/go1.17.6/bin/:/opt/mage/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HTTPS_PROXY': 'http://***.***.com:8080', 'SHLVL': '0', 'npm': '/usr/bin/npm', 'HOME': '/vsts', 'HOSTNAME': '3b16618b01c9', 'JAVA_HOME': '/usr/lib/jvm/java-8-openjdk-amd64', 'JAVA_HOME_11_X64': '/usr/lib/jvm/java-11-openjdk-amd64', 'VSTS_ACCOUNT': '***', 'http_proxy': 'http://***.***.com:8080', 'no_proxy': 'localhost,127.0.0.1,.***.com,.***.***.com,.corp.gwpnet.com,.***apps.com,.***apps-np.com,.***apps-ee.com', 'DOCKER_TLS_VERIFY': '1', 'SV_SRUID': '***', 'SV_PWD': '***', 'SV_NSSO_UID': '***', 'SV_NSSO_PWD': '***', 'INPUT_SCRIPT': '', 'INPUT_ARGUMENTS': '', 'INPUT_PYTHONINTERPRETER': '', 'ENDPOINT_URL_SYSTEMVSSCONNECTION': 'https://***.visualstudio.com/', 'AGENT_ENABLE_PIPELINEARTIFACT_LARGE_CHUNK_SIZE': 'true', 'SYSTEM_TEAMFOUNDATIONCOLLECTIONURI': 'https://***.visualstudio.com/', 'USE_MSAL': 'true', 'AGENT_VERSION': '3.227.2', 'BUILD_SOURCEBRANCHNAME': 'master', 'VSTS_PROCESS_LOOKUP_ID': 'vsts_2853cad9-1720-4bb4-80da-73465ddbe18c', 'SYSTEM_JOBPOSITIONINPHASE': '1', 'BUILD_REASON': 'Manual', 'AGENT_CLOUDID': '', 'SYSTEM_TEAMPROJECT': 'RESAPP-RESPlatform', 'AGENT_PROXYURL': 'http://***.***.com:8080', 'SYSTEM_JOBIDENTIFIER': 'Job.__default', 'SYSTEM_TIMELINEID': 'be313c2b-3473-436f-b401-122ade3dbc23', 'SYSTEM_TASKINSTANCENAME': 'PythonScript', 'BUILD_BUILDID': '7442040', 'SYSTEM_PHASEDISPLAYNAME': 'Job', 'BUILD_QUEUEDBYID': 'a84c2c70-e6c9-63f2-b20a-4a376eec0443', 'SYSTEM_PHASEID': '3a3a2a60-14c7-570b-14a4-fa42ad92f52a', 'AGENTCAPABILITYSKIPYAMLDECORATOR': '', 'VSTSAGENT_CONTINUE_AFTER_CANCEL_PROCESSTREEKILL_ATTEMPT': 'true', 'SYSTEM_COLLECTIONID': 'e99f0146-bd0f-443b-9865-7bc4fe807b64', 'SYSTEM_JOBID': '12f1170f-54f2-53f3-20dd-22fc7dff55f9', 'SYSTEM_JOBDISPLAYNAME': 'Job', 'RESOURCES_TRIGGERINGCATEGORY': '', 'BUILD_SOURCESDIRECTORY': '/workspace/1/s', 'AGENT_OS': 'Linux', 'SYSTEM_TASKINSTANCEID': '840007cc-f097-5b1c-e29a-3ed6decfdf4b', 'AGENT_PROXYBYPASSLIST': '["localhost","127\\\\.0\\\\.0\\\\.1","\\\\.***\\\\.com","\\\\.***\\\\.***\\\\.com","\\\\.corp\\\\.gwpnet\\\\.com","\\\\.***apps\\\\.com","\\\\.***apps-np\\\\.com","\\\\.***apps-ee\\\\.com"]', 'AGENT_BUILDDIRECTORY': '/workspace/1', 'AGENT_ID': '269139', 'AGENT_HOMEDIRECTORY': '/vsts/agent', 'AGENT_READONLYVARIABLES': 'true', 'BUILD_SOURCEVERSIONAUTHOR': '***', 'BUILD_REPOSITORY_ID': '69207665-f290-4fef-a6b8-31fd53ebaadd', 'MSDEPLOY_HTTP_USER_AGENT': 'VSTS_e99f0146-bd0f-443b-9865-7bc4fe807b64_build_33899_0', 'COMMON_TESTRESULTSDIRECTORY': '/workspace/1/TestResults', 'SYSTEM_JOBNAME': '__default', 'GIT_TERMINAL_PROMPT': '0', 'SYSTEM_POSTLINESSPEED': '500', 'BUILD_ARTIFACTSTAGINGDIRECTORY': '/workspace/1/a', 'BUILD_REQUESTEDFOREMAIL': '***@***.com', 'SYSTEM_WORKFOLDER': '/workspace', 'SYSTEM_TOTALJOBSINPHASE': '1', 'BUILD_REQUESTEDFORID': 'a84c2c70-e6c9-63f2-b20a-4a376eec0443', 'SYSTEM_DEFINITIONNAME': 'config-admin-daily-master-data-sync', 'AZURE_HTTP_USER_AGENT': 'VSTS_e99f0146-bd0f-443b-9865-7bc4fe807b64_build_33899_0', 'BUILD_STAGINGDIRECTORY': '/workspace/1/a', 'VSTSAGENT_DOCKER_ACTION_RETRIES': 'true', 'AGENT_ACCEPTTEEEULA': 'False', 'SYSTEM_PHASEATTEMPT': '1', 'AGENT_RETAINDEFAULTENCODING': 'false', 'SYSTEM_CULTURE': 'en-US', 'BUILD_REPOSITORY_GIT_SUBMODULECHECKOUT': 'False', 'BUILD_REPOSITORY_URI': 'https://***.visualstudio.com/RESAPP-RESPlatform/_git/res-platform-conf-qa', 'SYSTEM': 'build', 'SYSTEM_JOBTIMEOUT': '60', 'BUILD_DEFINITIONVERSION': '6', 'AGENT_ROOTDIRECTORY': '/workspace', 'AGENT_JOBNAME': 'Job', 'BUILD_REPOSITORY_NAME': 'res-platform-conf-qa', 'AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN': 'true', 'SYSTEM_ISAZUREVM': '0', 'SYSTEM_TEAMPROJECTID': '7ad18223-7781-48d8-a436-85e20dbde2c9', 'AGENT_TEMPDIRECTORY': '/workspace/_temp', 'SYSTEM_HOSTTYPE': 'build', 'SYSTEM_JOBPARALLELISMTAG': 'Private', 'BUILD_REQUESTEDFOR': '***', 'PIPELINE_WORKSPACE': '/workspace/1', 'AZP_AGENT_IGNORE_VSTSTASKLIB': 'true', 'SYSTEM_STAGENAME': '__default', 'RESOURCES_TRIGGERINGALIAS': '', 'SYSTEM_PIPELINESTARTTIME': '2023-11-02 07:52:50+00:00', 'BUILD_DEFINITIONFOLDERPATH': '\\', 'SYSTEM_COLLECTIONURI': 'https://***.visualstudio.com/', 'SYSTEM_STAGEDISPLAYNAME': '__default', 'SYSTEM_ISSCHEDULED': 'False', 'AGENT_ENABLEADDITIONALMASKINGREGEXES': 'On', 'SYSTEM_STAGEATTEMPT': '1', 'AGENT_RUNMODE': 'once', 'SYSTEM_SERVERTYPE': 'Hosted', 'SYSTEM_ARTIFACTSDIRECTORY': '/workspace/1/a', 'agent.jobstatus': 'Succeeded', 'AGENT_JOBSTATUS': 'Succeeded', 'BUILD_REPOSITORY_PROVIDER': 'TfsGit', 'SYSTEM_JOBATTEMPT': '1', 'AGENT_WORKFOLDER': '/workspace', 'SYSTEM_DEFINITIONID': '33899', 'SYSTEM_TEAMFOUNDATIONSERVERURI': 'https://***.visualstudio.com/', 'BUILD_SOURCEVERSIONMESSAGE': 'Reverting insertion of bash script.', 'BUILD_DEFINITIONNAME': 'config-admin-daily-master-data-sync', 'BUILD_REPOSITORY_LOCALPATH': '/workspace/1/s', 'USEPYTHONVERSION_PYTHONLOCATION': '/workspace/_tool/Python/3.10.12/x64', 'BUILD_REPOSITORY_CLEAN': 'False', 'AGENT_USEWORKSPACEID': 'true', 'AGENT_LOGTOBLOBSTORAGESERVICE': 'true', 'AGENT_TASKRESTRICTIONSENFORCEMENTMODE': 'Enabled', 'SYSTEM_ISDOCKERCONTAINER': '1', 'SYSTEM_STAGEID': '96ac2280-8cb4-5df5-99de-dd2da759617d', 'AGENT_DIAGNOSTIC': 'True', 'AGENT_MACHINENAME': '3b16618b01c9', 'TASK_SKIPTRANSLATORFORCHECKOUT': 'False', 'BUILD_BUILDURI': 'vstfs:///Build/Build/7442040', 'AGENT_ISSELFHOSTED': '1', 'BUILD_SOURCEBRANCH': 'refs/heads/master', 'SYSTEM_TASKDISPLAYNAME': 'PythonScript', 'BUILD_CONTAINERID': '25356812', 'BUILD_QUEUEDBY': '***', 'BUILD_BINARIESDIRECTORY': '/workspace/1/b', 'SYSTEM_PLANID': 'be313c2b-3473-436f-b401-122ade3dbc23', 'SYSTEM_ENABLEACCESSTOKEN': 'SecretVariable', 'BUILD_BUILDNUMBER': '20231102.4', 'BUILD_SOURCEVERSION': '4e0a383732ee03b9c52315519d92bb7c9b2b38dc', 'AZP_75787_ENABLE_COLLECT': 'true', 'TASK_DISPLAYNAME': 'PythonScript', 'SYSTEM_DEBUG': 'True', 'SYSTEM_TASKDEFINITIONSURI': 'https://***.visualstudio.com/', 'AGENT_TOOLSDIRECTORY': '/workspace/_tool', 'SYSTEM_PHASENAME': 'Job', 'AGENT_NAME': '***', 'SYSTEM_PULLREQUEST_ISFORK': 'False', 'AGENT_OSARCHITECTURE': 'X64', 'AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN': 'true', 'SYSTEM_DEFAULTWORKINGDIRECTORY': '/workspace/1/s', 'LC_CTYPE': 'C.UTF-8'})
##[debug]Re-evaluate condition on job cancellation for step: 'PythonScript'.
##[error]The operation was canceled.

Aditional info

Since yesterday, PythonScript@0 is getting stuck in every run and the pipeline eventually gets canceled after an hour. Using the same script in bash task is working as expected. Could you please check, if there's any recent issues with this task?
Here's the snippet of the Python script, which is getting stuck.
print(os.environ)
if 'TF_BUILD' in os.environ and os.environ['TF_BUILD'] == 'True':
    print('Detected ADO execution environment.')
print(f'isodayofweek(): {datetime.datetime.now().isoweekday()}')
vmapetr commented 1 year ago

Hi @KristofKuli, thank you for reporting this issue. From what I saw in the logs the issue seems related to the machine environment or to the ADO agent, Could you please provide some additional information, that can help with the investigation? 1) Could you please specify what agent version is deployed on a self-hosted machine? 2) What PythonScript task version was used on successful runs?

KristofKuli commented 1 year ago

Hi Maksim, Most probably the issue is generated by the env vars, which are defined in the task. As I have mentioned earlier, in bash task everything worked fined, however when the envs were added, we experienced the same issue. We have also tried variable groups, but no luck.

image

The agent version of the self-hosted pool is 3.227.2 and the task version is the same on the previous successful runs, 0.229.3.

KristofKuli commented 1 year ago

Hi Maksim,

Today the pipeline finished successfully, without any change from our side. Was there any update released to the PythonSrcipt task?

vmapetr commented 1 year ago

@KristofKuli there were no updates for the PythonScript task - you can validate it by comparing the task version mentioned in the logs.

vmapetr commented 1 year ago

Those environment variables that you mentioned earlier - could you please provide more context about them? Are they somehow used in Python/Bash scripts, or are they used in some background service? Since they affect the pipeline execution, perhaps the issue here is related to the agent itself, and not to the tasks.

vmapetr commented 12 months ago

@KristofKuli could you please confirm that the issue still occurs?

KristofKuli commented 12 months ago

I think we can close this issue for now.

vmapetr commented 12 months ago

Thank you, we will close it then - please feel free to reopen it in case of issue reoccurs.