#279: Test GH CRABClient using preprod REST instance and CMSSW_12_0_1 CMSSW release

cmsbuild commented 2 years ago

Tests started for following configuration:

Configuration:

CRABClient_version: GH
- REST_Instance: preprod
- CMSSW_release: CMSSW_12_0_1
- SCRAM_ARCH: slc7_amd64_gcc900
- WMCore_tag: 1.3.6.crab6
- CRABServer_tag: HEAD
- DBS_tag: 3.14.0

Tests started:

[x] Client_Validation
[x] Client_Configuration_Validation
[ ] Task_Submission_Status_Tracking

Started at: 2022-07-01 17:26:08 Link to the job

cmsbuild commented 2 years ago

Task submission for Task_Submission_Status_Tracking successfully ended.

220701_152929:cmsbot_crab_20220701_172926
220701_152932:cmsbot_crab_20220701_172930
220701_152935:cmsbot_crab_20220701_172933
220701_152938:cmsbot_crab_20220701_172936
220701_152941:cmsbot_crab_20220701_172939
220701_152946:cmsbot_crab_20220701_172942

Task submission for Client_Configuration_Validation successfully ended.

220701_152746:cmsbot_crab_transferOutputs
220701_152749:cmsbot_crab_transferLogs
220701_152752:cmsbot_crab_activity
220701_152756:cmsbot_crab_inputFiles
220701_152759:cmsbot_crab_disableAutomaticOutputCollection
220701_152802:cmsbot_crab_outputFiles
220701_152805:cmsbot_crab_allowUndistributedCMSSW
220701_152808:cmsbot_crab_maxMemoryMB
220701_152812:cmsbot_crab_maxJobRuntimeMin
220701_152815:cmsbot_crab_numCores
220701_152818:cmsbot_crab_scriptExe
220701_152821:cmsbot_crab_scriptArgs
220701_152824:cmsbot_crab_sendPythonFolder
220701_152831:cmsbot_crab_sendExternalFolder
220701_152835:cmsbot_crab_inputDBS
220701_152838:cmsbot_crab_useParent
220701_152841:cmsbot_crab_secondaryInputDataset
220701_152844:cmsbot_crab_lumiMaskFile
220701_152847:cmsbot_crab_lumiMaskUrl
220701_152851:cmsbot_crab_outLFNDirBase
220701_152854:cmsbot_crab_runRange
220701_152858:cmsbot_crab_ignoreLocality
220701_152901:cmsbot_crab_userInputFiles
220701_152904:cmsbot_crab_whitelist
220701_152907:cmsbot_crab_blacklist
220701_152910:cmsbot_crab_ignoreGlobalBlacklist
220701_152913:cmsbot_crab_voRole
220701_152916:cmsbot_crab_voGroup
220701_152919:cmsbot_crab_scheddName
220701_152922:cmsbot_crab_collector
220701_152925:cmsbot_crab_extraJDL

Task submission for Client_Validation_Suite successfully ended.

220701_152743:cmsbot_crab_20220701_172739

Finished at: 2022-07-01 17:29:47 Find submission log here

cmsbuild commented 2 years ago

Test: Client validation Result: SUCCEEDED Finished at: 2022-07-01 17:53:31 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_ClientValidation/270/console All executed commands: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_ClientValidation/270//artifact/client-validation.log/*view*/ Message: All commands were executed successfully.

cmsbuild commented 2 years ago

Test: Client configuration validation Result: FULL-STATUS-UNKNOWN Attempt: 0 out of 4. Will run again. Finished at: 2022-07-01 17:55:37 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_ClientConfigurationValidation/397/console

RETRY TESTS:
transferLogs-check.sh 220701_152749:cmsbot_crab_transferLogs - 2
activity-check.sh 220701_152752:cmsbot_crab_activity - 2
numCores-check.sh 220701_152815:cmsbot_crab_numCores - 2
lumiMaskUrl-check.sh 220701_152847:cmsbot_crab_lumiMaskUrl - 2
outLFNDirBase-check.sh 220701_152851:cmsbot_crab_outLFNDirBase - 2
runRange-check.sh 220701_152854:cmsbot_crab_runRange - 2
whitelist-check.sh 220701_152904:cmsbot_crab_whitelist - 2
ignoreGlobalBlacklist-check.sh 220701_152910:cmsbot_crab_ignoreGlobalBlacklist - 2

SUCCESSFUL TESTS:
transferOutputs-check.sh 220701_152746:cmsbot_crab_transferOutputs - 0
inputFiles-check.sh 220701_152756:cmsbot_crab_inputFiles - 0
disableAutomaticOutputCollection-check.sh 220701_152759:cmsbot_crab_disableAutomaticOutputCollection - 0
outputFiles-check.sh 220701_152802:cmsbot_crab_outputFiles - 0
allowUndistributedCMSSW-check.sh 220701_152805:cmsbot_crab_allowUndistributedCMSSW - 0
maxMemoryMB-check.sh 220701_152808:cmsbot_crab_maxMemoryMB - 0
maxJobRuntimeMin-check.sh 220701_152812:cmsbot_crab_maxJobRuntimeMin - 0
scriptExe-check.sh 220701_152818:cmsbot_crab_scriptExe - 0
scriptArgs-check.sh 220701_152821:cmsbot_crab_scriptArgs - 0
sendPythonFolder-check.sh 220701_152824:cmsbot_crab_sendPythonFolder - 0
sendExternalFolder-check.sh 220701_152831:cmsbot_crab_sendExternalFolder - 0
inputDBS-check.sh 220701_152835:cmsbot_crab_inputDBS - 0
useParent-check.sh 220701_152838:cmsbot_crab_useParent - 0
secondaryInputDataset-check.sh 220701_152841:cmsbot_crab_secondaryInputDataset - 0
lumiMaskFile-check.sh 220701_152844:cmsbot_crab_lumiMaskFile - 0
ignoreLocality-check.sh 220701_152858:cmsbot_crab_ignoreLocality - 0
userInputFiles-check.sh 220701_152901:cmsbot_crab_userInputFiles - 0
blacklist-check.sh 220701_152907:cmsbot_crab_blacklist - 0
voRole-check.sh 220701_152913:cmsbot_crab_voRole - 0
voGroup-check.sh 220701_152916:cmsbot_crab_voGroup - 0
scheddName-check.sh 220701_152919:cmsbot_crab_scheddName - 0
collector-check.sh 220701_152922:cmsbot_crab_collector - 0
extraJDL-check.sh 220701_152925:cmsbot_crab_extraJDL - 0

FAILED TESTS:
 none

cmsbuild commented 2 years ago

Test: Client configuration validation Result: SUCCEEDED Attempt: 1 out of 4. Test is done. Finished at: 2022-07-01 18:21:37 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_ClientConfigurationValidation/398/console

RETRY TESTS:
 none

SUCCESSFUL TESTS:
transferOutputs-check.sh 220701_152746:cmsbot_crab_transferOutputs - 0
transferLogs-check.sh 220701_152749:cmsbot_crab_transferLogs - 0
activity-check.sh 220701_152752:cmsbot_crab_activity - 0
inputFiles-check.sh 220701_152756:cmsbot_crab_inputFiles - 0
disableAutomaticOutputCollection-check.sh 220701_152759:cmsbot_crab_disableAutomaticOutputCollection - 0
outputFiles-check.sh 220701_152802:cmsbot_crab_outputFiles - 0
allowUndistributedCMSSW-check.sh 220701_152805:cmsbot_crab_allowUndistributedCMSSW - 0
maxMemoryMB-check.sh 220701_152808:cmsbot_crab_maxMemoryMB - 0
maxJobRuntimeMin-check.sh 220701_152812:cmsbot_crab_maxJobRuntimeMin - 0
numCores-check.sh 220701_152815:cmsbot_crab_numCores - 0
scriptExe-check.sh 220701_152818:cmsbot_crab_scriptExe - 0
scriptArgs-check.sh 220701_152821:cmsbot_crab_scriptArgs - 0
sendPythonFolder-check.sh 220701_152824:cmsbot_crab_sendPythonFolder - 0
sendExternalFolder-check.sh 220701_152831:cmsbot_crab_sendExternalFolder - 0
inputDBS-check.sh 220701_152835:cmsbot_crab_inputDBS - 0
useParent-check.sh 220701_152838:cmsbot_crab_useParent - 0
secondaryInputDataset-check.sh 220701_152841:cmsbot_crab_secondaryInputDataset - 0
lumiMaskFile-check.sh 220701_152844:cmsbot_crab_lumiMaskFile - 0
lumiMaskUrl-check.sh 220701_152847:cmsbot_crab_lumiMaskUrl - 0
outLFNDirBase-check.sh 220701_152851:cmsbot_crab_outLFNDirBase - 0
runRange-check.sh 220701_152854:cmsbot_crab_runRange - 0
ignoreLocality-check.sh 220701_152858:cmsbot_crab_ignoreLocality - 0
userInputFiles-check.sh 220701_152901:cmsbot_crab_userInputFiles - 0
whitelist-check.sh 220701_152904:cmsbot_crab_whitelist - 0
blacklist-check.sh 220701_152907:cmsbot_crab_blacklist - 0
ignoreGlobalBlacklist-check.sh 220701_152910:cmsbot_crab_ignoreGlobalBlacklist - 0
voRole-check.sh 220701_152913:cmsbot_crab_voRole - 0
voGroup-check.sh 220701_152916:cmsbot_crab_voGroup - 0
scheddName-check.sh 220701_152919:cmsbot_crab_scheddName - 0
collector-check.sh 220701_152922:cmsbot_crab_collector - 0
extraJDL-check.sh 220701_152925:cmsbot_crab_extraJDL - 0

FAILED TESTS:
 none

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 0 out of 4. Will run again. Finished at: 2022-07-01 18:32:26 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_CheckTestResults/276/console

{'TN': '220701_152929:cmsbot_crab_20220701_172926', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 988, 'transferring': 9, 'running': 3}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152932:cmsbot_crab_20220701_172930', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 5, 'running': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152935:cmsbot_crab_20220701_172933', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 1}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152938:cmsbot_crab_20220701_172936', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152941:cmsbot_crab_20220701_172939', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152946:cmsbot_crab_20220701_172942', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 8, 'transferring': 2}, 'combinedStatus': 'SUBMITTED'}

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 1 out of 4. Will run again. Finished at: 2022-07-01 19:37:23 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_CheckTestResults/277/console

{'TN': '220701_152929:cmsbot_crab_20220701_172926', 'dbStatus': u'SUBMITTED', 'testResult': 'TestFailed', 'jobsPerStatus': {'failed': 1, 'finished': 998, 'transferring': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152932:cmsbot_crab_20220701_172930', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 6}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152935:cmsbot_crab_20220701_172933', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 1}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152938:cmsbot_crab_20220701_172936', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152941:cmsbot_crab_20220701_172939', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152946:cmsbot_crab_20220701_172942', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 8, 'transferring': 2}, 'combinedStatus': 'SUBMITTED'}

mapellidario commented 2 years ago

The purpose of this test was to validate that the machine works, even after removing DBS_tag and WMCore_tag from the script test/container/testingScripts/setupCRABClient.sh. I will close this issue even if the last check on statusTracking has not been run yet.

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 2 out of 4. Will run again. Finished at: 2022-07-01 20:39:38 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_CheckTestResults/278/console

{'TN': '220701_152929:cmsbot_crab_20220701_172926', 'dbStatus': u'SUBMITTED', 'testResult': 'TestFailed', 'jobsPerStatus': {'failed': 1, 'finished': 998, 'transferring': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152932:cmsbot_crab_20220701_172930', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 6}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152935:cmsbot_crab_20220701_172933', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 1}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152938:cmsbot_crab_20220701_172936', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152941:cmsbot_crab_20220701_172939', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152946:cmsbot_crab_20220701_172942', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 8, 'transferring': 2}, 'combinedStatus': 'SUBMITTED'}

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 3 out of 4. Will run again. Finished at: 2022-07-01 21:41:53 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_CheckTestResults/280/console

{'TN': '220701_152929:cmsbot_crab_20220701_172926', 'dbStatus': u'SUBMITTED', 'testResult': 'TestFailed', 'jobsPerStatus': {'failed': 1, 'finished': 998, 'transferring': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152932:cmsbot_crab_20220701_172930', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 6}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152935:cmsbot_crab_20220701_172933', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 1}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152938:cmsbot_crab_20220701_172936', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152941:cmsbot_crab_20220701_172939', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152946:cmsbot_crab_20220701_172942', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 8, 'transferring': 2}, 'combinedStatus': 'SUBMITTED'}

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 4 out of 4. Exceeded configured retries. If needed restart manually. Finished at: 2022-07-01 22:44:03 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/CRABServer_CheckTestResults/282/console

{'TN': '220701_152929:cmsbot_crab_20220701_172926', 'dbStatus': u'SUBMITTED', 'testResult': 'TestFailed', 'jobsPerStatus': {'failed': 1, 'finished': 998, 'transferring': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '220701_152932:cmsbot_crab_20220701_172930', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 6}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152935:cmsbot_crab_20220701_172933', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 1}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152938:cmsbot_crab_20220701_172936', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152941:cmsbot_crab_20220701_172939', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '220701_152946:cmsbot_crab_20220701_172942', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 8, 'transferring': 2}, 'combinedStatus': 'SUBMITTED'}

mapellidario / CRABServer

#279: Test GH CRABClient using preprod REST instance and CMSSW_12_0_1 CMSSW release #21