search

dmwm / DBSClient

Apache License 2.0
3 stars 8 forks source link

exception is too much verbose #68

Open belforte opened 2 years ago

belforte commented 2 years ago

I am using py3-dbs3-client/4.0.7 on python 3.8 I find that exception message from DBS error is way too much verbose to be usable/useful in logs. Can something be done ?

Here's a quick example where I try to access a non-existing API [1]. Applications using DBS Client do not care to know the internal traceback (note in particular the During handling of the above exception, another exception occurred: In this case these two lines would have been plenty.

RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 404: 
URL=https://cmsweb.cern.ch:8443/dbs/prod/global/BAD/serverinfo

[1]

>>> import dbs.apis.dbsClient
>>> dbs.apis.dbsClient.__file__
'/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py'
>>> import sys
>>> sys.version
'3.8.2 (default, Jan 22 2021, 17:57:37) \n[GCC 6.3.0]'
>>> badApi = DbsApi(url='https://cmsweb.cern.ch/dbs/prod/global/BAD')
>>> badApi.serverinfo()
Traceback (most recent call last):
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 445, in __callServer
    self.http_response = method_func(self.url, method, params, data, request_headers)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-pycurl/3.17.7-comp/lib/python3.8/site-packages/RestClient/RestApi.py", line 36, in get
    return http_request(self._curl)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-pycurl/3.17.7-comp/lib/python3.8/site-packages/RestClient/RequestHandling/HTTPRequest.py", line 62, in __call__
    raise HTTPError(effective_url, http_code, http_response.msg, http_response.raw_header, http_response.body)
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 1838, in serverinfo
    return self.__callServer("serverinfo")
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 457, in __callServer
    self.__parseForException(data)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 483, in __parseForException
    raise http_error
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 404: 
URL=https://cmsweb.cern.ch:8443/dbs/prod/global/BAD/serverinfo
Code=404
Message=Not Found
Header=HTTP/1.1 404 Not Found
Date: Fri, 18 Mar 2022 13:57:40 GMT
Server: Apache
Content-Type: text/html
Content-Length: 146
CMS-Server-Time: D=11556 t=1647611860253268

Body=<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
belforte commented 2 years ago

a related (although somehow independent) problem is that the above manages to utterly confuse WMCore's DBSReader wrapper producing a really awful ouput. @amaltaro

>>> from WMCore.Services.DBS.DBSReader import DBSReader
>>> myDBS=DBSReader('https://cmsweb.cern.ch/dbs/prod/global/BAD')
ERROR:root:DBSReaderError
Message: Can't contact DBS at https://cmsweb.cern.ch/dbs/prod/global/BAD, got errors Instantiating DBS3Reader failed with HTTP Error 503: 
URL=https://cmsweb-prod.cern.ch:8443/dbs/prod/global/BAD/serverinfo
Code=503
Message=Service Unavailable
Header=HTTP/1.1 503 Service Unavailable
Date: Fri, 18 Mar 2022 14:15:50 GMT
Server: Apache
Set-Cookie: cms-node=6234941600972e824cc69a3dcc1058a852e72918879184b240f209d2e673ea58c935cd1735d6b86ed48f22fe;path=/;secure;httponly
CMS-Server-Time: D=95612 t=1647612950611354
Connection: close
Content-Type: text/html

Body=<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
 <head>
 <title>CMSWEB Error: Service unavailable</title>
 <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
 <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
 <link rel="stylesheet" type="text/css" media="screen" href="/css/cmsweb.css" />
</head>
<body>
 <div id="main">
  <div id="top">
   <div class="boxTitle"><img src="/img/title.gif" alt="CMSWEB (title)" /></div>
  </div>
  <div id="middle">
   <div class="boxLinkContainer">
    <h2>Service unavailable</h2>
    <p>The site you requested is not available.</p><p>CMS <a href='https://twiki.cern.ch/twiki/bin/view/CMS/ScheduledInterventions'>scheduled interventions page</a> lists planned CMS and CERN service downtimes and known site-wide computing system incidents. Planned updates are notified on the <a href='https://hypernews.cern.ch/HyperNews/CMS/get/cernCompAnnounce.html'>CERN computing announcements</a> forum.</p><p>Would you please be so kind and <a href='https://ggus.eu/?mode=ticket_cms'>create a ticket</a> to"CMS Web Tools" support unit</a> if this interruption was unexpected?</p>
   </div>
  </div>
  <div id="bottom">
   <div class="boxFooter">&nbsp;</div>
  </div>
 </div>
</body>
</html>

    ClassName : None
    ModuleName : WMCore.Services.DBS.DBSErrors
    MethodName : __init__
    ClassInstance : None
    FileName : /data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/crabtaskworker/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/lib/python3.8/site-packages/WMCore/Services/DBS/DBSErrors.py
    LineNumber : 38
    ErrorNr : 1002

Traceback (most recent call last):
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 485, in __parseForException
    data = json.loads(data)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/crabtaskworker/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/lib/python3.8/site-packages/WMCore/Services/DBS/DBSReader.py", line 19, in DBSReader
    dbs.dbs.serverinfo()
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 1838, in serverinfo
    return self.__callServer("serverinfo")
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 457, in __callServer
    self.__parseForException(data)
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/py3-dbs3-client/4.0.7/lib/python3.8/site-packages/dbs/apis/dbsClient.py", line 491, in __parseForException
    raise http_error
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 503: 
URL=https://cmsweb-prod.cern.ch:8443/dbs/prod/global/BAD/serverinfo
Code=503
Message=Service Unavailable
Header=HTTP/1.1 503 Service Unavailable
Date: Fri, 18 Mar 2022 14:15:50 GMT
Server: Apache
Set-Cookie: cms-node=6234941600972e824cc69a3dcc1058a852e72918879184b240f209d2e673ea58c935cd1735d6b86ed48f22fe;path=/;secure;httponly
CMS-Server-Time: D=95612 t=1647612950611354
Connection: close
Content-Type: text/html

Body=<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
 <head>
 <title>CMSWEB Error: Service unavailable</title>
 <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
 <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
 <link rel="stylesheet" type="text/css" media="screen" href="/css/cmsweb.css" />
</head>
<body>
 <div id="main">
  <div id="top">
   <div class="boxTitle"><img src="/img/title.gif" alt="CMSWEB (title)" /></div>
  </div>
  <div id="middle">
   <div class="boxLinkContainer">
    <h2>Service unavailable</h2>
    <p>The site you requested is not available.</p><p>CMS <a href='https://twiki.cern.ch/twiki/bin/view/CMS/ScheduledInterventions'>scheduled interventions page</a> lists planned CMS and CERN service downtimes and known site-wide computing system incidents. Planned updates are notified on the <a href='https://hypernews.cern.ch/HyperNews/CMS/get/cernCompAnnounce.html'>CERN computing announcements</a> forum.</p><p>Would you please be so kind and <a href='https://ggus.eu/?mode=ticket_cms'>create a ticket</a> to"CMS Web Tools" support unit</a> if this interruption was unexpected?</p>
   </div>
  </div>
  <div id="bottom">
   <div class="boxFooter">&nbsp;</div>
  </div>
 </div>
</body>
</html>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/crabtaskworker/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/lib/python3.8/site-packages/WMCore/Services/DBS/DBSReader.py", line 23, in DBSReader
    raise DBSReaderError("Can't contact DBS at %s, got errors %s" % (endpoint, msg))
WMCore.Services.DBS.DBSErrors.DBSReaderError: DBSReaderError
Message: Can't contact DBS at https://cmsweb.cern.ch/dbs/prod/global/BAD, got errors Instantiating DBS3Reader failed with HTTP Error 503: 
URL=https://cmsweb-prod.cern.ch:8443/dbs/prod/global/BAD/serverinfo
Code=503
Message=Service Unavailable
Header=HTTP/1.1 503 Service Unavailable
Date: Fri, 18 Mar 2022 14:15:50 GMT
Server: Apache
Set-Cookie: cms-node=6234941600972e824cc69a3dcc1058a852e72918879184b240f209d2e673ea58c935cd1735d6b86ed48f22fe;path=/;secure;httponly
CMS-Server-Time: D=95612 t=1647612950611354
Connection: close
Content-Type: text/html

Body=<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
 <head>
 <title>CMSWEB Error: Service unavailable</title>
 <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
 <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
 <link rel="stylesheet" type="text/css" media="screen" href="/css/cmsweb.css" />
</head>
<body>
 <div id="main">
  <div id="top">
   <div class="boxTitle"><img src="/img/title.gif" alt="CMSWEB (title)" /></div>
  </div>
  <div id="middle">
   <div class="boxLinkContainer">
    <h2>Service unavailable</h2>
    <p>The site you requested is not available.</p><p>CMS <a href='https://twiki.cern.ch/twiki/bin/view/CMS/ScheduledInterventions'>scheduled interventions page</a> lists planned CMS and CERN service downtimes and known site-wide computing system incidents. Planned updates are notified on the <a href='https://hypernews.cern.ch/HyperNews/CMS/get/cernCompAnnounce.html'>CERN computing announcements</a> forum.</p><p>Would you please be so kind and <a href='https://ggus.eu/?mode=ticket_cms'>create a ticket</a> to"CMS Web Tools" support unit</a> if this interruption was unexpected?</p>
   </div>
  </div>
  <div id="bottom">
   <div class="boxFooter">&nbsp;</div>
  </div>
 </div>
</body>
</html>

    ClassName : None
    ModuleName : WMCore.Services.DBS.DBSErrors
    MethodName : __init__
    ClassInstance : None
    FileName : /data/srv/TaskManager/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/slc7_amd64_gcc630/cms/crabtaskworker/v3.220314patch1-3391c4ccadfae468fd63c78475580c2d/lib/python3.8/site-packages/WMCore/Services/DBS/DBSErrors.py
    LineNumber : 38
    ErrorNr : 1002

>>>
vkuznet commented 2 years ago

@belforte , here you hit several issues which bundle into single output:

vkuznet commented 2 years ago

One specific suggestion may be useful (but may not be practical) is to avoid using WMCore DBS code and vice versa (using WMCore code within DBSClient) where it is not appropriate. At the end when to query DBS we only need to make HTTP calls, that's it, but it does not need WMCore internals (including exception handling, logging, etc.) And, as you can see there are so many wrappers like WMCore.Services.DBS.DBSErrors where they are may not be required.

belforte commented 2 years ago

thanks @vkuznet . I agree that when something goes wrong details are needed. I have always been tempted to skip WMCore/DBSReader layers and talk directly to DBS Client, maybe it is time to do it. Original developers of course had a point about code sharing across projects, but there's also something to say about useless wrappers :-(

I will figure out something

vkuznet commented 2 years ago

And, since we talk about wrappers. DBSClient is nothing else as yet another wrapper around DBS REST APIs. If you want to skip entire pipeline you may consider talk directly to DBS end-points. For instance DBS API call listDatasets() is nothing else as pure HTTP call to /dbs/.../datasets, etc. I understand the legacy but at the end clients should be aware of all of these intermediate steps and make conscious decision when it is useful or not. In my view we don't gain much going through client -> WMCore -> DBSClient -> DBS REST API, when we can replace entire pipeline with client -> DBS REST API, and our python codebase overhead is quite large in my view.

belforte commented 2 years ago

well. I would not make direct calls to REST anyhow but would need some common code to wrap them to deal with common exceptions, config, and some peristent object in memory for connection reclycling etc.. so I am happy to use DBSClient which already takes care of some stuff I not comfortable with and returns well documented JSON ! It is not urgent, but sooner or later I will get rid of the wmcore wrapping