search

CrowdStrike / MISP-tools

Import CrowdStrike Threat Intelligence into your instance of MISP
MIT License
39 stars 10 forks source link

Publish flag does not work for all Event Types - Malware Events are not Published #123

Closed packet-rat closed 1 year ago

packet-rat commented 1 year ago

Passing the command line -p argument does not result in Publishing of Malware Family Events.

screen -d -m python3 misp_import.py -f -nb -d -v -p --fullmonty

image

packet-rat commented 1 year ago

Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (https://github.com/CrowdStrike/MISP-tools/issues/113)

jshcodes commented 1 year ago

Adding this to the current sprint.

jshcodes commented 1 year ago

Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (#113)

If it's running that long, reduce the indicator time frame down. (Drop it down to a few hours.)

packet-rat commented 1 year ago

This is intended to be the first initial load following —obliterate.

It’s currently set to the maximum. Use case here is to perform the initial bulk load and then drop back.

Do you expect this will eventually complete or will it just run indefinitely?

From: Joshua Hiller @.> Sent: Tuesday, April 25, 2023 7:10:52 AM To: CrowdStrike/MISP-tools @.> Cc: MARONEY, PATRICK @.>; Author @.> Subject: Re: [CrowdStrike/MISP-tools] Publish flag does not work for all Event Types - Malware Events are not Published (Issue #123)

Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (#113https://urldefense.com/v3/__https://github.com/CrowdStrike/MISP-tools/issues/113__;!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGaNyr8HH0$)

If it's running that long, reduce the indicator time frame down. (Drop it down to a few hours.)

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/CrowdStrike/MISP-tools/issues/123*issuecomment-1521608499__;Iw!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGac7VR_tw$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AAYSFYLA7K6PLCSTET4LHILXC6WLZANCNFSM6AAAAAAXISJ4YE__;!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGa3DX4lDo$. You are receiving this because you authored the thread.Message ID: @.***>

jshcodes commented 1 year ago

The maximum is 15 days worth of indicators. If you are including hashes, that will be several million or more, which will far exceed what you can import in a day. (Each iteration will also add new indicators, adding to the overall data segment size.)

packet-rat commented 1 year ago

FYSA:

; Initial data segment size

; REPORTS - Up to 1 year can be imported

; INDICATORS - Up to 15 days (20220 minutes) can be imported

; ADVERSARIES (Actors) - Up to 730 days (2 years) can be imported

init_reports_days_before = 365

init_indicators_minutes_before = 20220

;init_indicators_minutes_before = 300

init_actors_days_before = 730

The ingestion appeared to eventually complete AFAICT

However, I’m going to significantly reduce everything:

init_reports_days_before = 7

init_indicators_minutes_before = 2880

init_actors_days_before = 7