Requests_t.py:TestRequests.testSecureWithProxy_with_pycurl fails in py3

[mapellidario]

Impact of the bug

src/python/WMCore/Services/Requests.py, and possibly some of the code that depends on it

Describe the bug

result = self.decode(result) transforms results from a bytes string containg for example {"hello": 1, "world"2} to a unicode string containg b'{"hello": 1, "world"2}', which is no longer parsable by json.loads()

How to reproduce it

py3 env,

nosetests test/python/WMCore_t/Services_t/Requests_t.py:TestRequests.testSecureWithProxy_with_pycurl

Expected behavior the unittest should not fail

Additional context and error message

adding the following log in src/python/WMCore/Services/Requests.py

    def decodeResult(self, result, decoder):
        """
        Decode the http/pycurl request result
        """
        logging.info("DM decodeResult: %s, %s", type(result), result)

when we run the unittest we have

[dmwm@028e0d827607 WMCore]$ nosetests test/python/WMCore_t/Services_t/Requests_t.py:TestRequests.testSecureWithProxy_with_pycurl
E
======================================================================
ERROR: https with proxy with pycurl
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dmwm/wmcore_unittest/WMCore/test/python/WMCore_t/Services_t/Requests_t.py", line 296, in testSecureWithProxy_with_pycurl
    if not isinstance(json.loads(out[0]), dict):
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre6/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre6/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre6/sw/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
-------------------- >> begin captured logging << --------------------
root: INFO: DM decodeResult: <class 'bytes'>, b'{"phedex":{"request_timestamp":1623409407.91401,"instance":"prod","request_url":"http://cmsweb.cern.ch:7001/phedex/datasvc/json/prod/groups","request_version":"2.5.0pre3-comp2","group":[{"name":"AnalysisOps","id":"42"},{"name":"B2G","id":"163"},{"name":"DataOps","id":"18"},{"name":"FacOps","id":"19"},{"name":"IB RelVal","id":"106"},{"name":"RelVal","id":"107"},{"name":"SMP","id":"162"},{"name":"b-physics","id":"8"},{"name":"b-tagging","id":"5"},{"name":"caf-alca","id":"102"},{"name":"caf-comm","id":"103"},{"name":"caf-lumi","id":"164"},{"name":"caf-phys","id":"104"},{"name":"deprecated-ewk","id":"2"},{"name":"deprecated-qcd","id":"16"},{"name":"deprecated-undefined","id":"142"},{"name":"dqm","id":"122"},{"name":"e-gamma_ecal","id":"4"},{"name":"exotica","id":"15"},{"name":"express","id":"108"},{"name":"forward","id":"17"},{"name":"heavy-ions","id":"10"},{"name":"higgs","id":"6"},{"name":"jets-met_hcal","id":"9"},{"name":"local","id":"22"},{"name":"muon","id":"3"},{"name":"susy","id":"12"},{"name":"tau-pflow","id":"13"},{"name":"top","id":"7"},{"name":"tracker-dpg","id":"14"},{"name":"tracker-pog","id":"82"},{"name":"trigger","id":"11"},{"name":"upgrade","id":"105"}],"request_call":"groups","call_time":0.00318,"request_date":"2021-06-11 11:03:27 UTC"}}'

[amaltaro]

@mapellidario please do not work on this one yet. I have faced similar problems while fixing those two WorkQueue_t unit tests and I have been trying a couple of things in https://github.com/dmwm/WMCore/pull/10588

We can touch base on slack as well.

[amaltaro]

10588 should be fixing it.