Closed subbyte closed 1 year ago
Python 3.9.6 on macOS with pip list results:
Package Version
-------------------------------- -----------
adal 1.2.7
antlr4-python3-runtime 4.8
anyio 3.6.2
anytree 2.8.0
appdirs 1.4.4
appnope 0.1.3
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.2.1
attrs 22.2.0
Babel 2.12.1
backcall 0.2.0
beautifulsoup4 4.12.2
bleach 6.0.0
boto3 1.26.84
botocore 1.29.111
cattrs 22.2.0
certifi 2022.12.7
cffi 1.15.1
charset-normalizer 3.1.0
click 8.1.3
colorama 0.4.6
colorlog 6.7.0
comm 0.1.3
cpe 1.2.1
cryptography 39.0.2
debugpy 1.6.7
decorator 5.1.1
deepmerge 1.1.0
defusedxml 0.7.1
docker 6.0.1
exceptiongroup 1.1.1
executing 1.2.0
fastjsonschema 2.16.3
firepit 2.3.15
Flask 2.2.3
flatten-json 0.1.13
idna 3.4
ijson 3.2.0.post0
importlib-metadata 6.3.0
ipykernel 6.22.0
ipython 8.12.0
ipython-genutils 0.2.0
itsdangerous 2.1.2
jedi 0.18.2
Jinja2 3.1.2
jmespath 1.0.1
json-fix 0.5.1
json5 0.9.11
jsonmerge 1.9.0
jsonschema 4.17.3
jupyter_client 8.1.0
jupyter_core 5.3.0
jupyter-server 1.23.6
jupyterlab 3.5.3
jupyterlab-pygments 0.2.2
jupyterlab_server 2.15.2
kestrel-jupyter 1.0.7
kestrel-lang 1.5.12
lark 1.1.5
lark-parser 0.12.0
lxml 4.9.2
MarkupSafe 2.1.2
matplotlib-inline 0.1.6
mistune 2.0.5
nbclassic 0.5.5
nbclient 0.7.3
nbconvert 7.3.1
nbformat 5.8.0
nest-asyncio 1.5.6
notebook 6.5.4
notebook_shim 0.2.2
numpy 1.24.2
packaging 23.1
pandas 2.0.0
pandocfilters 1.5.0
parso 0.8.3
pexpect 4.8.0
pickleshare 0.7.5
pip 23.0.1
platformdirs 3.2.0
prometheus-client 0.16.0
prompt-toolkit 3.0.38
psutil 5.9.4
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 11.0.0
pycparser 2.21
Pygments 2.15.0
PyJWT 2.6.0
pyOpenSSL 23.0.0
pyrsistent 0.19.3
python-dateutil 2.8.2
pytz 2023.3
PyYAML 6.0
pyzmq 25.0.2
requests 2.28.2
requests-cache 1.0.1
requests-toolbelt 0.10.1
s3transfer 0.6.0
Send2Trash 1.8.0
setuptools 67.6.1
simplejson 3.19.1
six 1.16.0
sniffio 1.3.0
soupsieve 2.4
stack-data 0.6.2
stix-shifter 4.6.2
stix-shifter-modules-elastic-ecs 4.6.2
stix-shifter-utils 4.6.2
stix2-matcher 3.0.0
stix2-patterns 1.3.2
stix2-validator 3.0.2
tabulate 0.9.0
terminado 0.17.1
tinycss2 1.2.1
tomli 2.0.1
tornado 6.2
traitlets 5.9.0
typeguard 3.0.2
typer 0.7.0
typing_extensions 4.5.0
tzdata 2023.3
ujson 5.7.0
url-normalize 1.4.3
urllib3 1.26.15
wcwidth 0.2.6
webencodings 0.5.1
websocket-client 1.5.1
Werkzeug 2.2.3
wheel 0.40.0
xmltodict 0.13.0
zipp 3.15.0
I think the problem involves stix-shifter mappings that map multiple native result fields to the same STIX property; I will need to handle that in the ingest()
function. Maybe combining the duplicate columns into 1 (taking the first non-null value).
I noticed the problem doesn't happen with stix-shifter 4.6.1 but it does with 4.6.2. I can't explain why though; the mappings in question don't seem to have changed between those versions.
Fixed by 464f7d0380f7fd798049bd580d36140da6435a39
Looks like I get an error with fast translation enabled (executed successfully with fast translation disabled).
Where
host101
points to ourrainbow
instance with indexeswinlogbeat-*
andbeats
dialects enabled (dialect is not an issue since it works with fast translation disabled).Error
Log