Closed packet-rat closed 1 year ago
Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (https://github.com/CrowdStrike/MISP-tools/issues/113)
Adding this to the current sprint.
Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (#113)
If it's running that long, reduce the indicator time frame down. (Drop it down to a few hours.)
This is intended to be the first initial load following —obliterate.
It’s currently set to the maximum. Use case here is to perform the initial bulk load and then drop back.
Do you expect this will eventually complete or will it just run indefinitely?
From: Joshua Hiller @.> Sent: Tuesday, April 25, 2023 7:10:52 AM To: CrowdStrike/MISP-tools @.> Cc: MARONEY, PATRICK @.>; Author @.> Subject: Re: [CrowdStrike/MISP-tools] Publish flag does not work for all Event Types - Malware Events are not Published (Issue #123)
Note that the '--fullmonty' in command line above from yesterday is still running. This could explain the Indicator Events not being published as Indicator ingestions have been looping indefinitely (#113https://urldefense.com/v3/__https://github.com/CrowdStrike/MISP-tools/issues/113__;!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGaNyr8HH0$)
If it's running that long, reduce the indicator time frame down. (Drop it down to a few hours.)
— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/CrowdStrike/MISP-tools/issues/123*issuecomment-1521608499__;Iw!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGac7VR_tw$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AAYSFYLA7K6PLCSTET4LHILXC6WLZANCNFSM6AAAAAAXISJ4YE__;!!BhdT!jSFLPdBVNlmgRh9Y_St9-Q7MXCf-hfrNYUtNQgO_9DbToGFsEzSSpFWGfGEcwZDUCPYvTbq4rV_IUpf5AFGa3DX4lDo$. You are receiving this because you authored the thread.Message ID: @.***>
The maximum is 15 days worth of indicators. If you are including hashes, that will be several million or more, which will far exceed what you can import in a day. (Each iteration will also add new indicators, adding to the overall data segment size.)
FYSA:
; Initial data segment size
; REPORTS - Up to 1 year can be imported
; INDICATORS - Up to 15 days (20220 minutes) can be imported
; ADVERSARIES (Actors) - Up to 730 days (2 years) can be imported
init_reports_days_before = 365
init_indicators_minutes_before = 20220
;init_indicators_minutes_before = 300
init_actors_days_before = 730
The ingestion appeared to eventually complete AFAICT
However, I’m going to significantly reduce everything:
init_reports_days_before = 7
init_indicators_minutes_before = 2880
init_actors_days_before = 7
Passing the command line -p argument does not result in Publishing of Malware Family Events.
screen -d -m python3 misp_import.py -f -nb -d -v -p --fullmonty