splunk / eventgen

Splunk Event Generator: Eventgen
Apache License 2.0
376 stars 180 forks source link

[BUG] eventgen does not work with python 3.8 #386

Closed kenshinxu1 closed 4 years ago

kenshinxu1 commented 4 years ago

Describe the bug Follow the eventgen install document and try to install it with a new created python 3.8 venv.

a lot of errors comes up with command as below, after the switch to a newly created python3.7.5 venv, it works fine:

pip install splunk_eventgen-7.1.0.tar.gz .... Installing collected packages: lxml, pytest-mock, docutils, jmespath, python-dateutil, botocore, s3transfer, boto3, ujson, pyyaml, httplib2, MarkupSafe, jinja2, pyflakes, pycodestyle, entrypoints, mccabe, flake8, yapf, isort, click, Werkzeug, itsdangerous, Flask, redis, requests-futures, uuid, splunk-eventgen Running setup.py install for lxml ... error ERROR: Command errored out with exit status 1: command: /root/eventgen/dist/eventgen-venv/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m3thgm4h/lxml/setup.py'"'"'; file='"'"'/tmp/pip-install-m3thgm4h/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-nppa3bp8/install-record.txt --single-version-externally-managed --compile --install-headers /root/eventgen/dist/eventgen-venv/include/site/python3.8/lxml cwd: /tmp/pip-install-m3thgm4h/lxml/ Complete output (2506 lines): Building lxml version 4.3.4. Building without Cython. Using build configuration of libxslt 1.1.28 Building against libxml2/libxslt in the following directory: /usr/lib64 running install running build running build_py creating build creating build/lib.linux-x86_64-3.8 creating build/lib.linux-x86_64-3.8/lxml copying src/lxml/sax.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/_elementpath.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/builder.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/init.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/cssselect.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/doctestcompare.py -> build/lib.linux-x86_64-3.8/lxml creating build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/init.py -> build/lib.linux-x86_64-3.8/lxml/includes creating build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/diff.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/builder.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/init.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/clean.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-3.8/lxml/html copying src/lxml/html/defs.py -> build/lib.linux-x86_64-3.8/lxml/html creating build/lib.linux-x86_64-3.8/lxml/isoschematron copying src/lxml/isoschematron/init.py -> build/lib.linux-x86_64-3.8/lxml/isoschematron copying src/lxml/etree.h -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/etree_api.h -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-3.8/lxml copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/init.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-3.8/lxml/includes copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-3.8/lxml/includes creating build/lib.linux-x86_64-3.8/lxml/isoschematron/resources creating build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/rng copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/rng creating build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl creating build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1 running build_ext building 'lxml.etree' extension creating build/temp.linux-x86_64-3.8 creating build/temp.linux-x86_64-3.8/src creating build/temp.linux-x86_64-3.8/src/lxml gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/root/eventgen/dist/eventgen-venv/include -I/usr/local/include/python3.8 -c src/lxml/etree.c -o build/temp.linux-x86_64-3.8/src/lxml/etree.o -w src/lxml/etree.c: In function 'Pyx_InitCachedConstants': src/lxml/etree.c:228068:3: error: too many arguments to function 'PyCode_New' pyx_codeobj90 = (PyObject*)Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, pyx_empty_bytes, pyx_empty_tuple, pyx_empty_tuple, pyx_tuple89, pyx_empty_tuple, pyx_empty_tuple, pyx_kp_s_src_lxml_etree_pyx, pyx_n_s_getitem, 95, pyx_empty_bytes); if (unlikely(!pyx_codeobj90)) __PYX_ERR(0, 95, __pyx_L1_error) ^ In file included from /usr/local/include/python3.8/compile.h:5:0, from /usr/local/include/python3.8/Python.h:138, from src/lxml/etree.c:97: /usr/local/include/python3.8/code.h:122:28: note: declared here PyAPI_FUNC(PyCodeObject *) PyCode_New( ^ src/lxml/etree.c:228080:3: error: too many arguments to function 'PyCode_New'

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Actual behavior A clear and concise description of what happens after doing the reproduce steps.

Screenshots If applicable, add screenshots to help explain your problem.

Sample files and eventgen.conf file Please attach your sample files and eventgen conf file

Do you run eventgen with SA-eventgen? No(No means you run eventgen with pip module mode)

If you are using SA-Eventgen with Splunk (please complete the following information):

If you are using eventgen with pip module mode (please complete the following information):

Additional context Add any other context about the problem here.

jmeixensperger commented 4 years ago

@kenshinxu1 can you pull down the develop branch and see if the issue still exists?