fossdriver setup not working with latest fossology container

[dl9pf]

I tried agl-fossdriver feature (default setup) with these values in conf/local.conf:

INHERIT += "fossdriver-host"

Fossdriver is installed on the local machine:

/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg /usr/lib/python2.7/site-packages/fossdriver-0.0.2-py2.7.egg

(installed both - not sure witch version is used by bitbake)

my ~/.fossdriverrc

{ "serverUrl": "http://xx.yy.zz.aa:8081/", "username": "fossy", "password": "fossy" }

The server is running the docker container . I tried "latest" and "3.3.0" . (e.g. docker run -d -p 8081:80 fossology/fossology:3.3.0 or
docker run -d -p 8081:80 fossology/fossology:latest )

Now when I run bitbake agl-image-minimal, I get this error (w 3.3.0):

ERROR: glibc-mtrace-2.28-r0 do_spdx: Error executing a python function in exec_python_func() autogenerated:The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: 0002:do_spdx(d) 0003: File: '/home/scratch/AGL/repobuild-fossology/external/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 112, function: do_spdx 0108: if not os.path.isfile( tar_name ): 0109: bb.warn(info['pn'] + "has no source, do nothing") 0110: return 0111: 0112: invoke_fossdriver(tar_name,sstatefile) 0113: if get_cached_spdx( sstatefile ) != None: 0114: write_cached_spdx( info,sstatefile,cur_ver_code ) 0115: ## CREATE MANIFEST(write to outfile ) 0116: create_manifest(info,sstatefile) File: '/home/scratch/AGL/repobuild-fossology/external/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 206, function: invoke_fossdriver 0202: 0203: server = FossServer(config) 0204: server.Login() 0205: bb.note("invoke_fossdriver : tar_file = %s " % tar_file) 0206: Upload(server, tar_file, "Software Repository").run() 0207: Scanners(server, tar_file, "Software Repository").run() 0208: SPDXTV(server, tar_file, "Software Repository", spdx_file).run() 0209: 0210:def create_manifest(info,sstatefile): File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/tasks.py', lineno: 73, function: run File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/tasks.py", line 73, in run self.server.WaitUntilAgentIsDone(newUploadNum, "ununpack", pollSeconds=5)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 410, function: WaitUntilAgentIsDone File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 410, in WaitUntilAgentIsDone jobNum = self._getMostRecentAgentJobNum(uploadNum, agent)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 250, function: _getMostRecentAgentJobNum File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 250, in _getMostRecentAgentJobNum jobs = self._getJobsForUpload(uploadNum)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 237, function: _getJobsForUpload File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 237, in _getJobsForUpload decodedContent = fossdriver.parser.decodeAjaxShowJobsData(results.content)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/parser.py', lineno: 142, function: decodeAjaxShowJobsData File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/parser.py", line 142, in decodeAjaxShowJobsData rj = json.loads(content)File: '/usr/lib64/python3.4/json/init.py', lineno: 312, function: loads 0308: 0309: """ 0310: if not isinstance(s, str): 0311: raise TypeError('the JSON object must be str, not {!r}'.format( 0312: s.class.name)) 0313: if s.startswith(u'\ufeff'): 0314: raise ValueError("Unexpected UTF-8 BOM (decode using utf-8-sig)") 0315: if (cls is None and object_hook is None and 0316: parse_int is None and parse_float is None and Exception: TypeError: the JSON object must be str, not 'bytes'ERROR: glibc-mtrace-2.28-r0 do_spdx: Function failed: do_spdx ERROR: Logfile of failure stored in: /home/scratch/AGL/repobuild-fossology/build-q64-fossdriver/tmp/work/corei7-64-agl-linux/glibc-mtrace/2.28-r0/temp/log.do_spdx.15199 NOTE: recipe glibc-mtrace-2.28-r0: task do_spdx: Failed ERROR: Task (/home/scratch/AGL/repobuild-fossology/external/poky/meta/recipes-core/glibc/glibc-mtrace_2.28.bb:do_spdx) failed with exit code '1' ERROR: quilt-native-0.65-r0 do_spdx: Error executing a python function in exec_python_func() autogenerated:The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: 0002:do_spdx(d) 0003: File: '/home/scratch/AGL/repobuild-fossology/external/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 112, function: do_spdx 0108: if not os.path.isfile( tar_name ): 0109: bb.warn(info['pn'] + "has no source, do nothing") 0110: return 0111: 0112: invoke_fossdriver(tar_name,sstatefile) 0113: if get_cached_spdx( sstatefile ) != None: 0114: write_cached_spdx( info,sstatefile,cur_ver_code ) 0115: ## CREATE MANIFEST(write to outfile ) 0116: create_manifest(info,sstatefile) File: '/home/scratch/AGL/repobuild-fossology/external/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 206, function: invoke_fossdriver 0202: 0203: server = FossServer(config) 0204: server.Login() 0205: bb.note("invoke_fossdriver : tar_file = %s " % tar_file) 0206: Upload(server, tar_file, "Software Repository").run() 0207: Scanners(server, tar_file, "Software Repository").run() 0208: SPDXTV(server, tar_file, "Software Repository", spdx_file).run() 0209: 0210:def create_manifest(info,sstatefile): File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/tasks.py', lineno: 73, function: run File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/tasks.py", line 73, in run self.server.WaitUntilAgentIsDone(newUploadNum, "ununpack", pollSeconds=5)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 410, function: WaitUntilAgentIsDone File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 410, in WaitUntilAgentIsDone jobNum = self._getMostRecentAgentJobNum(uploadNum, agent)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 250, function: _getMostRecentAgentJobNum File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 250, in _getMostRecentAgentJobNum jobs = self._getJobsForUpload(uploadNum)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py', lineno: 237, function: _getJobsForUpload File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/server.py", line 237, in _getJobsForUpload decodedContent = fossdriver.parser.decodeAjaxShowJobsData(results.content)File: '/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/parser.py', lineno: 142, function: decodeAjaxShowJobsData File "/usr/lib/python3.4/site-packages/fossdriver-0.0.2-py3.4.egg/fossdriver/parser.py", line 142, in decodeAjaxShowJobsData rj = json.loads(content)File: '/usr/lib64/python3.4/json/init.py', lineno: 312, function: loads 0308: 0309: """ 0310: if not isinstance(s, str): 0311: raise TypeError('the JSON object must be str, not {!r}'.format( 0312: s.class.name)) 0313: if s.startswith(u'\ufeff'): 0314: raise ValueError("Unexpected UTF-8 BOM (decode using utf-8-sig)") 0315: if (cls is None and object_hook is None and 0316: parse_int is None and parse_float is None and Exception: TypeError: the JSON object must be str, not 'bytes'ERROR: quilt-native-0.65-r0 do_spdx: Function failed: do_spdx ERROR: Logfile of failure stored in: /home/scratch/AGL/repobuild-fossology/build-q64-fossdriver/tmp/work/x86_64-linux/quilt-native/0.65-r0/temp/log.do_spdx.15198 NOTE: recipe quilt-native-0.65-r0: task do_spdx: Failed ERROR: Task (/home/scratch/AGL/repobuild-fossology/external/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_spdx) failed with exit code '1'

[dl9pf]

Cross-reference: https://jira.automotivelinux.org/browse/SPEC-2618

[leimaohui]

Fossdriver has some issues with python3.5. I have pushed patch into repository of fossdriver. Please reference to https://github.com/fossology/fossdriver/pull/16 .

[swinslow]

Apologies for delay in merging the fix from @leimaohui in https://github.com/fossology/fossdriver/pull/16 -- it is now merged, thank you!