search

dmwm / CRABServer

16 stars 38 forks source link

add checkS3Object and verify S3 upload with it #8740

Closed belforte closed 1 month ago

cmsdmwmbot commented 1 month ago

Jenkins results:

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-CRABServer-PR-test/2198/artifact/artifacts/PullRequestReport.html

cmsdmwmbot commented 1 month ago

Jenkins results:

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-CRABServer-PR-test/2199/artifact/artifacts/PullRequestReport.html

novicecpp commented 1 month ago

:rocket: :rocket: :rocket:

belforte commented 1 month ago

I had only tested from client. But uploading of twlog now fails with 

 'x-error-info': 'taskname is missing'
2024-10-17 11:46:45,900:DEBUG:ServerUtilities:upload to S3 of twlog ./logs/tasks/crabint1/241017_084637:crabint1_crab_20241017_104637.log completed
2024-10-17 11:46:45,900:DEBUG:ServerUtilities:make sure that object is ready for retrieve
2024-10-17 11:46:46,943:ERROR:RESTInteractions:Rejected request when connecting to https://cmsweb-test2.cern.ch:8443/crabserver/dev/cache using subresource=download&objecttype=twlog&clientmethod=head_object. Error details: {'content-type': 'text/html;charset=utf-8', 'date': 'Thu, 17 Oct 2024 09:46:46 GMT', 'response-proto': 'HTTP/1.1', 'response-status': '400 Bad Request', 'response-status-code': '400', 'response-time': '15.413685ms', 'response-time-seconds': '0.015414353', 'server': 'CherryPy/18.8.0', 'x-content-type-options': 'bla', 'x-error-detail': 'Missing required parameter', 'x-error-http': '400', 'x-error-id': '4e6d3d37d4501074f907d5e1956b71e1', 'x-error-info': 'taskname is missing', 'x-rest-status': '301', 'x-rest-time': '9303.570 us', 'content-length': '736'}
novicecpp commented 1 month ago

Ah yes. It is used for S3 path construction.

belforte commented 1 month ago

thanks Wa. No worry. It was my code, I will figure out. I have doing more than one thing at same time atm. gimme a bit to get to this

belforte commented 1 month ago

fixed

cmsdmwmbot commented 1 month ago

Jenkins results:

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-CRABServer-PR-test/2210/artifact/artifacts/PullRequestReport.html

belforte commented 1 month ago