packet-rat
commented
1 year ago FYSA - Still running several days later...
Closed packet-rat closed 1 year ago
packet-rat
commented
1 year ago FYSA - Still running several days later...
jshcodes
commented
1 year ago Something's definitely not right here. Might have to kill that process and try to restart at a newer timestamp. (I'm working on recreating this.)
packet-rat
commented
1 year ago The “Indicator” ingestion has been behaving like this forever.
jshcodes
commented
1 year ago Caused by requested data segment size.
StressedOutMouse
commented
5 months ago May I asked what the ideal configuration is, that alleviate this issue? I'm coming across the same thing. I believe my pull request is minimal however, it is also a matter of having lower technical specifications. With only 16GBs of RAM, I am working with currently, large volumes of data being pulled will cause the connect to break or the job gets killed due to memory exhaustion
[2024-04-05 13:38:38,814] DEBUG config client_id value redacted, check config file
[2024-04-05 13:38:38,814] DEBUG config client_secret value redacted, check config file
[2024-04-05 13:38:38,814] DEBUG config crowdstrike_url auto
[2024-04-05 13:38:38,814] DEBUG config api_request_max 2500
[2024-04-05 13:38:38,814] DEBUG config api_enable_ssl True
[2024-04-05 13:38:38,815] DEBUG config reports_timestamp_filename lastReportsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG config indicators_timestamp_filename lastIndicatorsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG config actors_timestamp_filename lastActorsUpdate.dat
[2024-04-05 13:38:38,815] DEBUG config init_reports_days_before 1
[2024-04-05 13:38:38,815] DEBUG config init_indicators_minutes_before 60
[2024-04-05 13:38:38,815] DEBUG config init_actors_days_before 1
[2024-04-05 13:38:38,815] DEBUG config reports_tags value not specified
[2024-04-05 13:38:38,815] DEBUG config indicators_tags value not specified
[2024-04-05 13:38:38,816] DEBUG config actors_tags value not specified
[2024-04-05 13:38:38,816] DEBUG config unknown_mapping CrowdStrike:indicator:galaxy: UNATTRIBUTED
[2024-04-05 13:38:38,816] DEBUG config unattributed_title Unattributed indicators:
[2024-04-05 13:38:38,816] DEBUG config indicator_type_title Indicator Type:
[2024-04-05 13:38:38,816] DEBUG config malware_family_title Malware Family:
[2024-04-05 13:38:38,816] DEBUG config misp_url [REDACTED]
[2024-04-05 13:38:38,816] DEBUG config misp_auth_key value redacted, check config file
[2024-04-05 13:38:38,816] DEBUG config crowdstrike_org_uuid [REDACTED]
[2024-04-05 13:38:38,817] DEBUG config miss_track_file no_galaxy_mapping.log
[2024-04-05 13:38:38,817] DEBUG config galaxies_map_file galaxy.ini
[2024-04-05 13:38:38,817] DEBUG config misp_enable_ssl False
[2024-04-05 13:38:38,817] WARNING config misp_enable_ssl SSL is disabled for MISP API requests
[2024-04-05 13:38:38,817] DEBUG config misp_malware_family_range 7d
[2024-04-05 13:38:38,817] DEBUG config ind_attribute_batch_size 50
[2024-04-05 13:38:38,817] DEBUG config event_save_memory_refresh_interval 180
[2024-04-05 13:38:38,818] DEBUG config max_threads 10
[2024-04-05 13:38:38,818] DEBUG config tag_unknown_galaxy_maps True
[2024-04-05 13:38:38,818] DEBUG config taxonomic_kill-chain True
[2024-04-05 13:38:38,818] DEBUG config taxonomic_information-security-data-source True
[2024-04-05 13:38:38,818] DEBUG config taxonomic_type True
[2024-04-05 13:38:38,818] DEBUG config taxonomic_iep False
[2024-04-05 13:38:38,818] DEBUG config taxonomic_iep2 True
[2024-04-05 13:38:38,818] DEBUG config taxonomic_iep2_version False
[2024-04-05 13:38:38,818] DEBUG config taxonomic_tlp True
[2024-04-05 13:38:38,819] DEBUG config taxonomic_workflow True
Loops indefinitely (every hour) after ingesting Indicators
Command:
python3 misp_import.py -f -nb -d -v -p -iThe following Hash shows the hourly repeat cycle
Snipped Log