search

priv-kweihmann / meta-sca

Layer for static code analysis and security hardening
Other
90 stars 36 forks source link

No module named 'magic' #191

Closed RobertBerger closed 5 years ago

RobertBerger commented 5 years ago 
$ bitbake python3-mastermind                                               [104/1064]
NOTE: Started PRServer with DBfile: /home/student/projects/resy-playground/build/container-x86-64/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 39481, PID: 22416 
Loading cache: 100% |###########################################################################################################################| Time: 0:00:0$
Loaded 3204 entries from dependency cache.                                                                                                                     
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-nmapwrapper_git.bb: SCA: clang will be disabled bec$
use meta-clang-layer cannot be found                                                                                                                           
Please clone it from https://github.com/kraj/meta-clang                                                                                                        
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-messageqclient_git.bb: SCA: clang will be disabled $
ecause meta-clang-layer cannot be found                                                                                                                        
Please clone it from https://github.com/kraj/meta-clang                                                                                                        
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-nmapwrapper_git.bb: Using SCA Module(s) ansiblelint$
bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pylint,p$
symcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd                                                     
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb: SCA: clang will be disabled beca$
se meta-clang-layer cannot be found                                                                                                                            
Please clone it from https://github.com/kraj/meta-clang                                                                                                        
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-messageqclient_git.bb: Using SCA Module(s) ansiblel$
nt,bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pylin$
,pysymcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd                                                  
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-localdatacollector_git.bb: SCA: clang will be disab$
ed because meta-clang-layer cannot be found                                                                                                                    
Please clone it from https://github.com/kraj/meta-clang
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-tcpclientserver_git.bb: SCA: clang will be disabled
because meta-clang-layer cannot be found
Please clone it from https://github.com/kraj/meta-clang
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mqttbrokerclient_git.bb: SCA: clang will be disabled
 because meta-clang-layer cannot be found
Please clone it from https://github.com/kraj/meta-clang
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mqttbrokerclient_git.bb: Using SCA Module(s) ansible
lint,bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pyli
nt,pysymcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb: Using SCA Module(s) ansiblelint,b
andit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pylint,pys
ymcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-localdatacollector_git.bb: Using SCA Module(s) ansib
lelint,bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,py
lint,pysymcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-tcpclientserver_git.bb: Using SCA Module(s) ansiblel
int,bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pyli$
NOTE: /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-tcpclientserver_git.bb: Using SCA Module(s) ansiblel
int,bandit,bitbake,cppcheck,cpplint,cspell,cve-check,dennis,detectsecrets,eslint,flint,gcc,htmlhint,jsonlint,kconfighard,oelint,proselint,pyfindinjection,pylin
t,pysymcheck,pytype,rats,ropgadget,score,shellcheck,standard,stylelint,textlint,tscancode,xmllint,yamllint,zrd
Parsing recipes: 100% |#########################################################################################################################| Time: 0:00:04
Parsing of 2168 .bb files complete (2162 cached, 6 parsed). 3210 targets, 312 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.42.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "x86_64-resy-linux-musl"
MACHINE              = "container-x86-64"
DISTRO               = "resy-musl-container"
DISTRO_VERSION       = "2.7"
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
meta
meta-poky
meta-yocto-bsp       = "2019-05-13-warrior-2.7+:9001e77e428d86b1d10e0a9ceb9113a76d221678"
meta-resy            = "master:6806a1735c17ecb8cec564ac3076bb24cc2db562"
meta-desire          = "master:3eabd42b6c83e6abbd80f38f1ee49536fdfd0785"
meta-oe
meta-python          = "warrior:6fa72d587aab8e9d56d67a2552eb9cfbe25c86d2"
meta-sca             = "HEAD:86e3aaaf0c9ced9cc298318120e750f4d581d7e0"

Initialising tasks: 100% |######################################################################################################################| Time: 0:00:02
Sstate summary: Wanted 36 Found 30 Missed 6 Current 269 (83% match, 98% complete)
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: python3-mastermind-1.0.0+gitAUTOINC+e252148a83-r0 do_sca_pysymcheck: Error executing a python function in exec_python_func() autogenerated:             

The stack trace of python calls that resulted in this exception/failure was:                                                                                   
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_sca_pysymcheck(d)
     0003:
File: '/home/student/projects/resy-playground/sources/poky/../meta-sca/classes/sca-pysymcheck.bbclass', lineno: 31, function: do_sca_pysymcheck
     0027:    if not os.path.exists(os.path.join(d.getVar("STAGING_BINDIR_NATIVE"), "pysymbolcheck", d.getVar("SCA_PYSYMCHECK_RULE_FILE"))):
     0028:        bb.warn("Rule-File {} does not exists - Empty results will be expected".format(d.getVar("SCA_PYSYMCHECK_RULE_FILE")))
     0029:
     0030:    _files = get_files_by_mimetype(d, d.getVar("B"), ["application/x-executable", 'application/x-sharedlib'],\
 *** 0031:                                   sca_filter_files(d, d.getVar("SCA_SOURCES_DIR"), clean_split(d, "SCA_FILE_FILTER_EXTRA")))
     0032:    ## Run
     0033:    cmd_output = ""
     0034:    tmp_result = os.path.join(d.getVar("T", True), "sca_raw_pysymcheck.txt")
     0035:    d.setVar("SCA_RAW_RESULT_FILE", tmp_result)
File: '/home/student/projects/resy-playground/sources/poky/../meta-sca/classes/sca-helper.bbclass', lineno: 89, function: get_files_by_mimetype
     0085:    return [x for x in res if os.path.isfile(x)]
     0086:
     0087:def get_files_by_mimetype(d, path, mime, excludes=[]):
     0088:    import os
 *** 0089:    import magic
     0090:    res = []
     0091:    for root, dirs, files in os.walk(path, topdown=True):
     0092:        for item in files:
     0093:            _filename = os.path.join(root, item)
Exception: ModuleNotFoundError: No module named 'magic'

ERROR: python3-mastermind-1.0.0+gitAUTOINC+e252148a83-r0 do_sca_pysymcheck: No module named 'magic'
ERROR: python3-mastermind-1.0.0+gitAUTOINC+e252148a83-r0 do_sca_pysymcheck: Function failed: do_sca_pysymcheck
ERROR: Logfile of failure stored in: /home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+$
itAUTOINC+e252148a83-r0/temp/log.do_sca_pysymcheck.24786
ERROR: Task (/home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb:do_sca_pysymcheck) failed $
ith exit code '1'
WARNING: python3-mastermind-1.0.0+gitAUTOINC+e252148a83-r0 do_sca_proselint_core: Traceback (most recent call last):                                   [2/1064]
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/bin/proselint", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3191, in <module>
    @_call_aside
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3175, in _call_aside
    f(*args, **kwargs)
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3204, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+e252148a83-r0/recip
e-sysroot-native/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'proselint==0.10.2' distribution was not found and is required by the application

WARNING: python3-mastermind-1.0.0+gitAUTOINC+e252148a83-r0 do_sca_proselint_core: Expecting value: line 1 column 1 (char 0)
NOTE: Tasks Summary: Attempted 1818 tasks of which 1790 didn't need to be rerun and 1 failed.
NOTE: The errors for this build are stored in /home/student/projects/resy-playground/build/container-x86-64/tmp/log/error-report/error_report_20190702201520.tx
t
You can send the errors to a reports server by running:
  send-error-report /home/student/projects/resy-playground/build/container-x86-64/tmp/log/error-report/error_report_20190702201520.txt [-s server]
NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or pro
prietary information when prompted before sending.

Summary: 1 task failed:
  /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb:do_sca_pysymcheck
Summary: There were 2 WARNING messages shown.
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.

host dependencies?

priv-kweihmann commented 5 years ago

That has been already fix with d552501047bb7a22c37b774fca6694bb4ec38aa5