search

OTRF / Security-Datasets

Re-play Security Events
MIT License
1.59k stars 237 forks source link

Modor-Elastic shipper inputs issues #39

Closed ayman-m closed 3 years ago

ayman-m commented 3 years ago

The following exceptions while passing a JSON dataset, submitting a PR shortly :

Problem with the inputs argument: ayman@iMac mordor % scripts/data-shippers/Mordor-Elastic.py --url http://192.168.20.50:9200 inputs datasets/large/apt29/day1/apt29_evals_day1_manual_2020-05-01225525.json Initializing Elasticsearch connection and index... Calulating total file size... N/A% (0 of 2) | | Elapsed Time: 0:00:00 ETA: --:--:--Traceback (most recent call last): File "/Volumes/Data/Coding/mordor/scripts/data-shippers/Mordor-Elastic.py", line 69, in total_size = sum([ File "/Volumes/Data/Coding/mordor/scripts/data-shippers/Mordor-Elastic.py", line 72, in for member in tarfile.open(path).getmembers() if member.isfile() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tarfile.py", line 1611, in open return func(name, "r", fileobj, kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tarfile.py", line 1675, in gzopen fileobj = GzipFile(name, mode + "b", compresslevel, fileobj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/gzip.py", line 173, in init fileobj = self.myfileobj = builtins.open(filename, mode or 'rb') FileNotFoundError: [Errno 2] No such file or directory: 'inputs' I submitted PR #40 to fix this issue**

Shipping a JSON file: ayman@iMac mordor % scripts/data-shippers/Mordor-Elastic.py --url http://192.168.20.50:9200 inputs datasets/large/apt29/day1/apt29_evals_day1_manual_2020-05-01225525.json Initializing Elasticsearch connection and index... Calulating total file size... N/A% (0 of 1) | | Elapsed Time: 0:00:00 ETA: --:--:--Traceback (most recent call last): File "/Volumes/Data/Coding/mordor/scripts/data-shippers/Mordor-Elastic.py", line 69, in total_size = sum([ File "/Volumes/Data/Coding/mordor/scripts/data-shippers/Mordor-Elastic.py", line 72, in for member in tarfile.open(path).getmembers() if member.isfile() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tarfile.py", line 1616, in open raise ReadError("file could not be opened successfully") tarfile.ReadError: file could not be opened successfully _tar -zcvf the json file fixed my issue, may submit another PR to handle different input file formats when I have a chance

Cyb3rWard0g commented 3 years ago

Thank you @ayman-m !