#144: Test prod CRABClient using preprod REST instance and CMSSW_7_6_7 CMSSW release

cmsbuild commented 2 years ago

Tests started for following configuration:

Configuration:

CRABClient_version: prod
- REST_Instance: preprod
- CMSSW_release: CMSSW_7_6_7
- SCRAM_ARCH: slc6_amd64_gcc493

Tests started:

[ ] Client_Validation
[ ] Client_Configuration_Validation
[ ] Task_Submission_Status_Tracking

Started at: 2021-12-19 20:40:47 Link to the job

cmsbuild commented 2 years ago

Task submission for Task_Submission_Status_Tracking successfully ended.

211219_194202:cmsbot_crab_20211219_204201
211219_194205:cmsbot_crab_20211219_204203
211219_194207:cmsbot_crab_20211219_204205
211219_194209:cmsbot_crab_20211219_204207
211219_194211:cmsbot_crab_20211219_204210
211219_194233:cmsbot_crab_20211219_204212

Task submission for Client_Configuration_Validation successfully ended.

211219_194123:cmsbot_crab_transferOutputs
211219_194126:cmsbot_crab_transferLogs
211219_194128:cmsbot_crab_activity
211219_194129:cmsbot_crab_inputFiles
211219_194131:cmsbot_crab_disableAutomaticOutputCollection
211219_194132:cmsbot_crab_outputFiles
211219_194133:cmsbot_crab_allowUndistributedCMSSW
211219_194134:cmsbot_crab_maxMemoryMB
211219_194135:cmsbot_crab_maxJobRuntimeMin
211219_194136:cmsbot_crab_numCores
211219_194137:cmsbot_crab_scriptExe
211219_194139:cmsbot_crab_scriptArgs
211219_194141:cmsbot_crab_sendPythonFolder
211219_194145:cmsbot_crab_sendExternalFolder
211219_194146:cmsbot_crab_inputDBS
211219_194147:cmsbot_crab_useParent
211219_194148:cmsbot_crab_secondaryInputDataset
211219_194149:cmsbot_crab_lumiMaskFile
211219_194150:cmsbot_crab_runRange
211219_194151:cmsbot_crab_ignoreLocality
211219_194153:cmsbot_crab_userInputFiles
211219_194154:cmsbot_crab_whitelist
211219_194155:cmsbot_crab_blacklist
211219_194156:cmsbot_crab_ignoreGlobalBlacklist
211219_194157:cmsbot_crab_scheddName
211219_194158:cmsbot_crab_collector
211219_194159:cmsbot_crab_extraJDL

Task submission for Client_Validation_Suite successfully ended.

211219_194121:cmsbot_crab_20211219_204117

Finished at: 2021-12-19 20:42:33 Find submission log here

cmsbuild commented 2 years ago

Test: Client validation Result: SUCCEEDED Finished at: 2021-12-19 21:03:44 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_ClientValidation/174/console All executed commands: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_ClientValidation/174//artifact/client-validation.log/*view*/ Message: All commands were executed successfully.

cmsbuild commented 2 years ago

Test: Client configuration validation Result: FULL-STATUS-UNKNOWN Attempt: 0 out of 4. Will run again. Finished at: 2021-12-19 21:04:29 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_ClientConfigurationValidation/105/console

RETRY TESTS:
numCores-check.sh 211219_194136:cmsbot_crab_numCores - 2
scriptExe-check.sh 211219_194137:cmsbot_crab_scriptExe - 2
useParent-check.sh 211219_194147:cmsbot_crab_useParent - 2
ignoreGlobalBlacklist-check.sh 211219_194156:cmsbot_crab_ignoreGlobalBlacklist - 2

SUCCESSFUL TESTS:
transferOutputs-check.sh 211219_194123:cmsbot_crab_transferOutputs - 0
transferLogs-check.sh 211219_194126:cmsbot_crab_transferLogs - 0
activity-check.sh 211219_194128:cmsbot_crab_activity - 0
inputFiles-check.sh 211219_194129:cmsbot_crab_inputFiles - 0
disableAutomaticOutputCollection-check.sh 211219_194131:cmsbot_crab_disableAutomaticOutputCollection - 0
allowUndistributedCMSSW-check.sh 211219_194133:cmsbot_crab_allowUndistributedCMSSW - 0
maxMemoryMB-check.sh 211219_194134:cmsbot_crab_maxMemoryMB - 0
maxJobRuntimeMin-check.sh 211219_194135:cmsbot_crab_maxJobRuntimeMin - 0
scriptArgs-check.sh 211219_194139:cmsbot_crab_scriptArgs - 0
sendPythonFolder-check.sh 211219_194141:cmsbot_crab_sendPythonFolder - 0
sendExternalFolder-check.sh 211219_194145:cmsbot_crab_sendExternalFolder - 0
inputDBS-check.sh 211219_194146:cmsbot_crab_inputDBS - 0
blacklist-check.sh 211219_194155:cmsbot_crab_blacklist - 0
scheddName-check.sh 211219_194157:cmsbot_crab_scheddName - 0
extraJDL-check.sh 211219_194159:cmsbot_crab_extraJDL - 0

FAILED TESTS:
outputFiles-check.sh 211219_194132:cmsbot_crab_outputFiles - 1
secondaryInputDataset-check.sh 211219_194148:cmsbot_crab_secondaryInputDataset - 1
lumiMaskFile-check.sh 211219_194149:cmsbot_crab_lumiMaskFile - 1
runRange-check.sh 211219_194150:cmsbot_crab_runRange - 1
ignoreLocality-check.sh 211219_194151:cmsbot_crab_ignoreLocality - 1
userInputFiles-check.sh 211219_194153:cmsbot_crab_userInputFiles - 1
whitelist-check.sh 211219_194154:cmsbot_crab_whitelist - 1
collector-check.sh 211219_194158:cmsbot_crab_collector - 1

cmsbuild commented 2 years ago

Test: Client configuration validation Result: FULL-STATUS-UNKNOWN Attempt: 1 out of 4. Will run again. Finished at: 2021-12-19 21:25:54 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_ClientConfigurationValidation/106/console

RETRY TESTS:
numCores-check.sh 211219_194136:cmsbot_crab_numCores - 2
scriptExe-check.sh 211219_194137:cmsbot_crab_scriptExe - 2
useParent-check.sh 211219_194147:cmsbot_crab_useParent - 2

SUCCESSFUL TESTS:
transferOutputs-check.sh 211219_194123:cmsbot_crab_transferOutputs - 0
transferLogs-check.sh 211219_194126:cmsbot_crab_transferLogs - 0
activity-check.sh 211219_194128:cmsbot_crab_activity - 0
inputFiles-check.sh 211219_194129:cmsbot_crab_inputFiles - 0
disableAutomaticOutputCollection-check.sh 211219_194131:cmsbot_crab_disableAutomaticOutputCollection - 0
allowUndistributedCMSSW-check.sh 211219_194133:cmsbot_crab_allowUndistributedCMSSW - 0
maxMemoryMB-check.sh 211219_194134:cmsbot_crab_maxMemoryMB - 0
maxJobRuntimeMin-check.sh 211219_194135:cmsbot_crab_maxJobRuntimeMin - 0
scriptArgs-check.sh 211219_194139:cmsbot_crab_scriptArgs - 0
sendPythonFolder-check.sh 211219_194141:cmsbot_crab_sendPythonFolder - 0
sendExternalFolder-check.sh 211219_194145:cmsbot_crab_sendExternalFolder - 0
inputDBS-check.sh 211219_194146:cmsbot_crab_inputDBS - 0
blacklist-check.sh 211219_194155:cmsbot_crab_blacklist - 0
scheddName-check.sh 211219_194157:cmsbot_crab_scheddName - 0
extraJDL-check.sh 211219_194159:cmsbot_crab_extraJDL - 0

FAILED TESTS:
outputFiles-check.sh 211219_194132:cmsbot_crab_outputFiles - 1
secondaryInputDataset-check.sh 211219_194148:cmsbot_crab_secondaryInputDataset - 1
lumiMaskFile-check.sh 211219_194149:cmsbot_crab_lumiMaskFile - 1
runRange-check.sh 211219_194150:cmsbot_crab_runRange - 1
ignoreLocality-check.sh 211219_194151:cmsbot_crab_ignoreLocality - 1
userInputFiles-check.sh 211219_194153:cmsbot_crab_userInputFiles - 1
whitelist-check.sh 211219_194154:cmsbot_crab_whitelist - 1
ignoreGlobalBlacklist-check.sh 211219_194156:cmsbot_crab_ignoreGlobalBlacklist - 1
collector-check.sh 211219_194158:cmsbot_crab_collector - 1

cmsbuild commented 2 years ago

Test: Task Submission Status Tracking Result: FULL-STATUS-UNKNOWN Attempt: 0 out of 4. Will run again. Finished at: 2021-12-19 21:45:00 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_CheckTestResults/81/console

{'TN': '211219_194202:cmsbot_crab_20211219_204201', 'dbStatus': u'SUBMITTED', 'testResult': 'TestRunning', 'jobsPerStatus': {'finished': 962, 'transferring': 5, 'running': 33}, 'combinedStatus': 'SUBMITTED'}
{'TN': '211219_194205:cmsbot_crab_20211219_204203', 'dbStatus': u'SUBMITTED', 'testResult': 'TestFailed', 'jobsPerStatus': {'failed': 4, 'finished': 1, 'running': 1}, 'combinedStatus': 'SUBMITTED'}
{'TN': '211219_194207:cmsbot_crab_20211219_204205', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 2}, 'combinedStatus': 'COMPLETED'}
{'TN': '211219_194209:cmsbot_crab_20211219_204207', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}
{'TN': '211219_194211:cmsbot_crab_20211219_204210', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 100}, 'combinedStatus': 'COMPLETED'}
{'TN': '211219_194233:cmsbot_crab_20211219_204212', 'dbStatus': u'SUBMITTED', 'testResult': 'TestPassed', 'jobsPerStatus': {'finished': 10}, 'combinedStatus': 'COMPLETED'}

cmsbuild commented 2 years ago

Test: Client configuration validation Result: FAILED Attempt: 2 out of 4. Test failed. Investigate manually Finished at: 2021-12-19 21:47:09 Test log: https://cmssdt.cern.ch/dmwm-jenkins/job/COPYCAT_CRABServer_ClientConfigurationValidation/107/console

RETRY TESTS:
 none

SUCCESSFUL TESTS:
transferOutputs-check.sh 211219_194123:cmsbot_crab_transferOutputs - 0
transferLogs-check.sh 211219_194126:cmsbot_crab_transferLogs - 0
activity-check.sh 211219_194128:cmsbot_crab_activity - 0
inputFiles-check.sh 211219_194129:cmsbot_crab_inputFiles - 0
disableAutomaticOutputCollection-check.sh 211219_194131:cmsbot_crab_disableAutomaticOutputCollection - 0
allowUndistributedCMSSW-check.sh 211219_194133:cmsbot_crab_allowUndistributedCMSSW - 0
maxMemoryMB-check.sh 211219_194134:cmsbot_crab_maxMemoryMB - 0
maxJobRuntimeMin-check.sh 211219_194135:cmsbot_crab_maxJobRuntimeMin - 0
numCores-check.sh 211219_194136:cmsbot_crab_numCores - 0
scriptArgs-check.sh 211219_194139:cmsbot_crab_scriptArgs - 0
sendPythonFolder-check.sh 211219_194141:cmsbot_crab_sendPythonFolder - 0
sendExternalFolder-check.sh 211219_194145:cmsbot_crab_sendExternalFolder - 0
inputDBS-check.sh 211219_194146:cmsbot_crab_inputDBS - 0
blacklist-check.sh 211219_194155:cmsbot_crab_blacklist - 0
scheddName-check.sh 211219_194157:cmsbot_crab_scheddName - 0
extraJDL-check.sh 211219_194159:cmsbot_crab_extraJDL - 0

FAILED TESTS:
outputFiles-check.sh 211219_194132:cmsbot_crab_outputFiles - 1
scriptExe-check.sh 211219_194137:cmsbot_crab_scriptExe - 1
useParent-check.sh 211219_194147:cmsbot_crab_useParent - 1
secondaryInputDataset-check.sh 211219_194148:cmsbot_crab_secondaryInputDataset - 1
lumiMaskFile-check.sh 211219_194149:cmsbot_crab_lumiMaskFile - 1
runRange-check.sh 211219_194150:cmsbot_crab_runRange - 1
ignoreLocality-check.sh 211219_194151:cmsbot_crab_ignoreLocality - 1
userInputFiles-check.sh 211219_194153:cmsbot_crab_userInputFiles - 1
whitelist-check.sh 211219_194154:cmsbot_crab_whitelist - 1
ignoreGlobalBlacklist-check.sh 211219_194156:cmsbot_crab_ignoreGlobalBlacklist - 1
collector-check.sh 211219_194158:cmsbot_crab_collector - 1

ddaina / variousStuff

#144: Test prod CRABClient using preprod REST instance and CMSSW_7_6_7 CMSSW release #99