search

dmwm / dbs2go

DBS server written in Go
MIT License
5 stars 4 forks source link

Wrong error message for error HTTP code status 409 #14

Closed amaltaro closed 2 years ago

amaltaro commented 2 years ago

While running standard integration tests with a pre-production WMAgent node - which points to DBS Int (cmsweb-testbed), I found this exception in the DBS3Upload component:

2022-03-18 15:27:07,523:140450372957952:INFO:DBSUploadPoller:About to insert block /HLTPhysicsIsolatedBunch/Run2016H-ReReco_RunBlockWhite_Mar2022_Val_Alanv6-v11/DQMIO#4a1cbb7d-53c3-4173-a6f1-7ba4547b1c10
2022-03-18 15:27:07,525:140450372957952:INFO:DBSUploadPoller:About to insert block /HLTPhysicsIsolatedBunch/Run2016H-ReReco_RunBlockWhite_Mar2022_Val_Alanv6-v11/MINIAOD#57e43b4f-8e7c-4470-99b9-b8668e16a8a1
2022-03-18 15:27:07,526:140450372957952:INFO:DBSUploadPoller:About to insert block /HLTPhysicsIsolatedBunch/Run2016H-ReReco_RunBlockWhite_Mar2022_Val_Alanv6-v11/AOD#038ab35d-01ee-4156-9e65-d9f3f505d619
2022-03-18 15:27:08,110:140450372957952:ERROR:DBSUploadPoller:Error trying to process block /HLTPhysicsIsolatedBunch/Run2016H-ReReco_RunBlockWhite_Mar2022_Val_Alanv6-v11/DQMIO#4a1cbb7d-53c3-4173-a6f1-7ba4547b1c10 through DBS. Error: {"exc
eption": 409, "message": "Dataset/[processed DS]/[dataset access type] not yet inserted by concurrent insert. ", "type": "HTTPError"}
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 489, in __parseForException
    raise HTTPError(http_error.url, data['exception'], data['message'], http_error.header, http_error.body)
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 409: Dataset/[processed DS]/[dataset access type] not yet inserted by concurrent insert. 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMComponent/DBS3Buffer/DBSUploadPoller.py", li
ne 85, in uploadWorker
    dbsApi.insertBulkBlock(blockDump=block)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 606, in insertBulkBlock
    result =  self.__callServer("bulkblocks", data=blockDump, callmethod='POST' )
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 448, in __callServer
    self.__parseForException(http_error)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 491, in __parseForException
    raise http_error
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 445, in __callServer
    self.http_response = method_func(self.url, method, params, data, request_headers)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-pycurl/3.17.7-comp/lib/python3.8/site-packages/RestClient/RestApi.py", line 42, in post
    return http_request(self._curl)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/py3-dbs3-pycurl/3.17.7-comp/lib/python3.8/site-packages/RestClient/RequestHandling/HTTPRequest.py", line 62, in __call__
    raise HTTPError(effective_url, http_code, http_response.msg, http_response.raw_header, http_response.body)
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 409: Conflict

The error itself looks Okay to me, since we tried to insert first a DQMIO data/datatier, when we should have injected first the AOD one.

The actual problem reported in this issue is:

HTTP Error 409: Dataset/[processed DS]/[dataset access type] not yet inserted by concurrent insert.

Perhaps the best would be to report back to the client something like:

HTTP Error 409: Dataset: BLAH not yet inserted by concurrent insert.

where BLAH is the parent name. Or perhaps something like: "Parent for dataset: BLAH has not been inserted yet.". Or any message that is more clear on what the actual error is.

vkuznet commented 2 years ago

Alan, this has nothing to do with dbs2go codebase since you're still using Python DBSWriter on testbed, see

# type of DBSWriter
scurl https://cmsweb-testbed.cern.ch/dbs/int/global/DBSWriter/serverinfo
{"dbs_instance": "int/global", "dbs_version": "3.16.0-comp5"}

# type of DBSReader
scurl https://cmsweb-testbed.cern.ch/dbs/int/global/DBSReader/serverinfo
[{"dbs_version":"v00.04.57","server":"dbs2go git=v00.04.57 go=go1.17.7 date=2022-16-03"}]

So, the DBSWriter is still python code, while only DBSReader is new server. I will close this issue and you need to open corresponding ticket (if any) on DBS python codebase. I think the issue should go to DBSClient rather the server.

amaltaro commented 2 years ago

Oh, you are correct! Apologies for the noise Valentin. I won't even bother creating such issue for python-based DBS. Thanks