search

CrowdStrike / MISP-tools

Import CrowdStrike Threat Intelligence into your instance of MISP
MIT License
42 stars 10 forks source link

Ideal configuration to prevent indefinite ingestion of indicators #158

Open StressedOutMouse opened 7 months ago

StressedOutMouse commented 7 months ago

Hello,

Apologies as this is somewhat of a duplicate of: https://github.com/CrowdStrike/MISP-tools/issues/113 I wanted to ask what the ideal configuration of the MISP initialization script is? I've encountered an issue where after executing the script, it runs for a prolong period of time as the longest I've clocked is 12hrs+. Is it a matter of system resources, since with only 16GBs of RAM and the large volume of indicators, is the pull size too small?

Command: python3.8 misp_import.py --indicators --debug

[2024-04-05 13:38:38,814] DEBUG    config  client_id                                   value redacted, check config file
[2024-04-05 13:38:38,814] DEBUG    config  client_secret                               value redacted, check config file
[2024-04-05 13:38:38,814] DEBUG    config  crowdstrike_url                             auto
[2024-04-05 13:38:38,814] DEBUG    config  api_request_max                             2500
[2024-04-05 13:38:38,814] DEBUG    config  api_enable_ssl                              True
[2024-04-05 13:38:38,815] DEBUG    config  reports_timestamp_filename                  lastReportsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG    config  indicators_timestamp_filename               lastIndicatorsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG    config  actors_timestamp_filename                   lastActorsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG    config  init_reports_days_before                    1
[2024-04-05 13:38:38,815] DEBUG    config  init_indicators_minutes_before              60
[2024-04-05 13:38:38,815] DEBUG    config  init_actors_days_before                     1
[2024-04-05 13:38:38,815] DEBUG    config  reports_tags                                value not specified
[2024-04-05 13:38:38,815] DEBUG    config  indicators_tags                             value not specified
[2024-04-05 13:38:38,816] DEBUG    config  actors_tags                                 value not specified
[2024-04-05 13:38:38,816] DEBUG    config  unknown_mapping                             CrowdStrike:indicator:galaxy: UNATTRIBUTED
[2024-04-05 13:38:38,816] DEBUG    config  unattributed_title                          Unattributed indicators:
[2024-04-05 13:38:38,816] DEBUG    config  indicator_type_title                        Indicator Type:
[2024-04-05 13:38:38,816] DEBUG    config  malware_family_title                        Malware Family:
[2024-04-05 13:38:38,816] DEBUG    config  misp_url                                    [REDACTED]
[2024-04-05 13:38:38,816] DEBUG    config  misp_auth_key                               value redacted, check config file
[2024-04-05 13:38:38,816] DEBUG    config  crowdstrike_org_uuid                        [REDACTED]
[2024-04-05 13:38:38,817] DEBUG    config  miss_track_file                             no_galaxy_mapping.log
[2024-04-05 13:38:38,817] DEBUG    config  galaxies_map_file                           galaxy.ini
[2024-04-05 13:38:38,817] DEBUG    config  misp_enable_ssl                             False
[2024-04-05 13:38:38,817] WARNING  config  misp_enable_ssl                             SSL is disabled for MISP API requests
[2024-04-05 13:38:38,817] DEBUG    config  misp_malware_family_range                   5d
[2024-04-05 13:38:38,817] DEBUG    config  ind_attribute_batch_size                    1000
[2024-04-05 13:38:38,817] DEBUG    config  event_save_memory_refresh_interval          180
[2024-04-05 13:38:38,818] DEBUG    config  max_threads                                 16
[2024-04-05 13:38:38,818] DEBUG    config  tag_unknown_galaxy_maps                     True
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_kill-chain                        True
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_information-security-data-source  True
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_type                              True
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_iep                               False
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_iep2                              True
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_iep2_version                      False
[2024-04-05 13:38:38,818] DEBUG    config  taxonomic_tlp                               True
[2024-04-05 13:38:38,819] DEBUG    config  taxonomic_workflow                          True