Obliterate and Clean <xxx> are no longer working

packet-rat commented 1 year ago

Crowdstrike Artifacts (i.e., Reports, Actors) are not deleted after performing Obliterate and Clean functions functions.

Use of -f has no impact on outcomes
Have tried bulk --obliterate and individual -ca, -ca, ci, -ct

[2023-03-31 19:01:30,364] INFO     processor/main       Start clean up of CrowdStrike Adversary (BAT) events from MISP.
[2023-03-31 19:01:30,518] INFO     processor/main       Completed deletion of CrowdStrike BAT adversaries within MISP in 0.15 seconds
[2023-03-31 19:01:30,518] INFO     processor/main       Start clean up of CrowdStrike Adversary (BEAR) events from MISP.
[2023-03-31 19:01:30,607] INFO     processor/main       Completed deletion of CrowdStrike BEAR adversaries within MISP in 0.08 seconds
[2023-03-31 19:01:30,607] INFO     processor/main       Start clean up of CrowdStrike Adversary (BUFFALO) events from MISP.
[2023-03-31 19:01:30,693] INFO     processor/main       Completed deletion of CrowdStrike BUFFALO adversaries within MISP in 0.08 seconds
[2023-03-31 19:01:30,693] INFO     processor/main       Start clean up of CrowdStrike Adversary (CHOLLIMA) events from MISP.
[2023-03-31 19:01:30,795] INFO     processor/main       Completed deletion of CrowdStrike CHOLLIMA adversaries within MISP in 0.10 seconds
[2023-03-31 19:01:30,795] INFO     processor/main       Start clean up of CrowdStrike Adversary (CRANE) events from MISP.
[2023-03-31 19:01:31,177] INFO     processor/main       Completed deletion of CrowdStrike CRANE adversaries within MISP in 0.38 seconds
[2023-03-31 19:01:31,177] INFO     processor/main       Start clean up of CrowdStrike Adversary (JACKAL) events from MISP.
[2023-03-31 19:01:31,243] INFO     processor/main       Completed deletion of CrowdStrike JACKAL adversaries within MISP in 0.06 seconds
[2023-03-31 19:01:31,243] INFO     processor/main       Start clean up of CrowdStrike Adversary (HAWK) events from MISP.
[2023-03-31 19:01:31,368] INFO     processor/main       Completed deletion of CrowdStrike HAWK adversaries within MISP in 0.12 seconds
[2023-03-31 19:01:31,368] INFO     processor/main       Start clean up of CrowdStrike Adversary (KITTEN) events from MISP.
[2023-03-31 19:01:31,487] INFO     processor/main       Completed deletion of CrowdStrike KITTEN adversaries within MISP in 0.11 seconds
[2023-03-31 19:01:31,487] INFO     processor/main       Start clean up of CrowdStrike Adversary (LEOPARD) events from MISP.
[2023-03-31 19:01:31,585] INFO     processor/main       Completed deletion of CrowdStrike LEOPARD adversaries within MISP in 0.09 seconds
[2023-03-31 19:01:31,585] INFO     processor/main       Start clean up of CrowdStrike Adversary (LYNX) events from MISP.
[2023-03-31 19:01:31,700] INFO     processor/main       Completed deletion of CrowdStrike LYNX adversaries within MISP in 0.11 seconds
[2023-03-31 19:01:31,700] INFO     processor/main       Start clean up of CrowdStrike Adversary (OCELOT) events from MISP.
[2023-03-31 19:01:31,829] INFO     processor/main       Completed deletion of CrowdStrike OCELOT adversaries within MISP in 0.12 seconds
[2023-03-31 19:01:31,830] INFO     processor/main       Start clean up of CrowdStrike Adversary (PANDA) events from MISP.
[2023-03-31 19:01:32,007] INFO     processor/main       Completed deletion of CrowdStrike PANDA adversaries within MISP in 0.17 seconds
[2023-03-31 19:01:32,007] INFO     processor/main       Start clean up of CrowdStrike Adversary (SPIDER) events from MISP.
[2023-03-31 19:01:32,222] INFO     processor/main       Completed deletion of CrowdStrike SPIDER adversaries within MISP in 0.21 seconds
[2023-03-31 19:01:32,222] INFO     processor/main       Start clean up of CrowdStrike Adversary (TIGER) events from MISP.
[2023-03-31 19:01:32,340] INFO     processor/main       Completed deletion of CrowdStrike TIGER adversaries within MISP in 0.11 seconds
[2023-03-31 19:01:32,340] INFO     processor/main       Start clean up of CrowdStrike Adversary (WOLF) events from MISP.
[2023-03-31 19:01:32,476] INFO     processor/main       Completed deletion of CrowdStrike WOLF adversaries within MISP in 0.13 seconds
[2023-03-31 19:01:32,476] INFO     processor/main       Finished cleaning up CrowdStrike related events from MISP, 0 events deleted.

packet-rat commented 1 year ago

python3 misp_import.py -f -nb -d --obliterate

😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 

@@@  @@@  @@@   @@@@@@   @@@@@@@   @@@  @@@  @@@  @@@  @@@   @@@@@@@@  @@@
@@@  @@@  @@@  @@@@@@@@  @@@@@@@@  @@@@ @@@  @@@  @@@@ @@@  @@@@@@@@@  @@@
@@!  @@!  @@!  @@!  @@@  @@!  @@@  @@!@!@@@  @@!  @@!@!@@@  !@@        @@!
!@!  !@!  !@!  !@!  @!@  !@!  @!@  !@!!@!@!  !@!  !@!!@!@!  !@!        !@
@!!  !!@  @!@  @!@!@!@!  @!@!!@!   @!@ !!@!  !!@  @!@ !!@!  !@! @!@!@  @!@
!@!  !!!  !@!  !!!@!!!!  !!@!@!    !@!  !!!  !!!  !@!  !!!  !!! !!@!!  !!!
!!:  !!:  !!:  !!:  !!!  !!: :!!   !!:  !!!  !!:  !!:  !!!  :!!   !!:
:!:  :!:  :!:  :!:  !:!  :!:  !:!  :!:  !:!  :!:  :!:  !:!  :!:   !::  :!:
 :::: :: :::   ::   :::  ::   :::   ::   ::   ::   ::   ::   ::: ::::   ::
  :: :  : :     :   : :   :   : :  ::    :   :    ::    :    :: :: :   :::

😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 😱 

Obliterate is a destructive operation that will remove all CrowdStrike data
from your MISP instance. There is no going back once this process completes.

Are you sure you want to do this?

[Enter 'yes' to continue] ==> yes

         _.-^^---....,,---;
     _--/                  `--_
    <                        >)
    |        KA-BOOM!         |
     \._                   _./
        ```--. . , ; .--'''
              | |   |
           .-=||  | |=-.
           `-=#$%&%$#=-'
              | ;  :|
     _____.,-#%&$@%#&#~,._____
         COMMAND  ACCEPTED

[2023-03-31 19:16:11,848] INFO     misp_tools    MISP Import for CrowdStrike Threat Intelligence v0.6.8
[2023-03-31 19:16:11,849] INFO     config  CHECK CONFIG
[2023-03-31 19:16:11,850] DEBUG    config  client_id                                   value redacted, check config file
[2023-03-31 19:16:11,850] DEBUG    config  client_secret                               value redacted, check config file
[2023-03-31 19:16:11,850] DEBUG    config  crowdstrike_url                             US1
[2023-03-31 19:16:11,850] DEBUG    config  api_request_max                             5000
[2023-03-31 19:16:11,850] DEBUG    config  api_enable_ssl                              True
[2023-03-31 19:16:11,850] DEBUG    config  reports_timestamp_filename                  lastReportsUpdate.dat
[2023-03-31 19:16:11,850] DEBUG    config  indicators_timestamp_filename               lastIndicatorsUpdate.dat
[2023-03-31 19:16:11,850] DEBUG    config  actors_timestamp_filename                   lastActorsUpdate.dat
[2023-03-31 19:16:11,850] DEBUG    config  init_reports_days_before                    365
[2023-03-31 19:16:11,850] DEBUG    config  init_indicators_minutes_before              300
[2023-03-31 19:16:11,850] DEBUG    config  init_actors_days_before                     365
[2023-03-31 19:16:11,850] DEBUG    config  reports_unique_tag                          CrowdStrike: REPORT
[2023-03-31 19:16:11,850] DEBUG    config  indicators_unique_tag                       CrowdStrike: INDICATOR
[2023-03-31 19:16:11,851] DEBUG    config  actors_unique_tag                           CrowdStrike: ADVERSARY
[2023-03-31 19:16:11,851] DEBUG    config  reports_tags                                att:source="Crowdstrike.Report"
[2023-03-31 19:16:11,851] DEBUG    config  indicators_tags                             att:source="Crowdstrike.Indicators"
[2023-03-31 19:16:11,851] DEBUG    config  actors_tags                                 att:source="Crowdstrike.Actors"
[2023-03-31 19:16:11,851] DEBUG    config  unknown_mapping                             CrowdStrike:indicator:galaxy: UNATTRIBUTED
[2023-03-31 19:16:11,851] DEBUG    config  unattributed_title                          Unattributed indicators:
[2023-03-31 19:16:11,851] DEBUG    config  indicator_type_title                        Indicator Type:
[2023-03-31 19:16:11,851] DEBUG    config  malware_family_title                        Malware Family:
[2023-03-31 19:16:11,851] DEBUG    config  log_duplicates_as_sightings                 True
[2023-03-31 19:16:11,851] DEBUG    config  misp_url                                    https://3samisp
[2023-03-31 19:16:11,851] DEBUG    config  misp_auth_key                               value redacted, check config file
[2023-03-31 19:16:11,851] DEBUG    config  crowdstrike_org_uuid                        ca4f4b5d-db04-4a5e-a6de-e60636dc01be
[2023-03-31 19:16:11,851] DEBUG    config  misp_enable_ssl                             False
[2023-03-31 19:16:11,851] WARNING  config  misp_enable_ssl                             SSL is disabled for MISP API requests
[2023-03-31 19:16:11,851] DEBUG    config  ind_attribute_batch_size                    2500
[2023-03-31 19:16:11,851] DEBUG    config  event_save_memory_refresh_interval          180
[2023-03-31 19:16:11,851] DEBUG    config  max_threads                                 16
[2023-03-31 19:16:11,851] DEBUG    config  miss_track_file                             no_galaxy_mapping.log
[2023-03-31 19:16:11,851] DEBUG    config  galaxies_map_file                           galaxy.ini
[2023-03-31 19:16:11,851] DEBUG    config  tag_unknown_galaxy_maps                     True
[2023-03-31 19:16:11,851] DEBUG    config  taxonomic_kill-chain                        True
[2023-03-31 19:16:11,851] DEBUG    config  taxonomic_information-security-data-source  True
[2023-03-31 19:16:11,851] DEBUG    config  taxonomic_type                              True
[2023-03-31 19:16:11,851] DEBUG    config  taxonomic_iep                               False
[2023-03-31 19:16:11,851] DEBUG    config  taxonomic_iep2                              True
[2023-03-31 19:16:11,852] DEBUG    config  taxonomic_iep2_version                      False
[2023-03-31 19:16:11,852] DEBUG    config  taxonomic_tlp                               True
[2023-03-31 19:16:11,852] DEBUG    config  taxonomic_workflow                          True
[2023-03-31 19:16:12,753] INFO     config  No configuration errors found (1 warning)
[2023-03-31 19:16:12,753] INFO     config  
[2023-03-31 19:16:12,753] INFO     config  ____ _  _ ____ ____ _  _ ____    ___  ____ ____ ____ ____ ___
[2023-03-31 19:16:12,753] INFO     config  |    |__| |___ |    |_/  [__     |__] |__| [__  [__  |___ |  \
[2023-03-31 19:16:12,753] INFO     config  |___ |  | |___ |___ | \_ ___]    |    |  | ___] ___] |___ |__/
[2023-03-31 19:16:12,753] INFO     config  
[2023-03-31 19:16:14,174] INFO     processor/main       BEGIN DELETE
[2023-03-31 19:16:14,174] INFO     processor/main       Start clean up of CrowdStrike Adversary (BAT) events from MISP.
[2023-03-31 19:16:14,322] INFO     processor/main       Completed deletion of CrowdStrike BAT adversaries within MISP in 0.14 seconds
[2023-03-31 19:16:14,322] INFO     processor/main       Start clean up of CrowdStrike Adversary (BEAR) events from MISP.
[2023-03-31 19:16:14,467] INFO     processor/main       Completed deletion of CrowdStrike BEAR adversaries within MISP in 0.14 seconds
[2023-03-31 19:16:14,467] INFO     processor/main       Start clean up of CrowdStrike Adversary (BUFFALO) events from MISP.
[2023-03-31 19:16:14,599] INFO     processor/main       Completed deletion of CrowdStrike BUFFALO adversaries within MISP in 0.13 seconds
[2023-03-31 19:16:14,599] INFO     processor/main       Start clean up of CrowdStrike Adversary (CHOLLIMA) events from MISP.
[2023-03-31 19:16:14,691] INFO     processor/main       Completed deletion of CrowdStrike CHOLLIMA adversaries within MISP in 0.09 seconds
[2023-03-31 19:16:14,691] INFO     processor/main       Start clean up of CrowdStrike Adversary (CRANE) events from MISP.
[2023-03-31 19:16:14,895] INFO     processor/main       Completed deletion of CrowdStrike CRANE adversaries within MISP in 0.20 seconds
[2023-03-31 19:16:14,895] INFO     processor/main       Start clean up of CrowdStrike Adversary (JACKAL) events from MISP.
[2023-03-31 19:16:14,997] INFO     processor/main       Completed deletion of CrowdStrike JACKAL adversaries within MISP in 0.10 seconds
[2023-03-31 19:16:14,997] INFO     processor/main       Start clean up of CrowdStrike Adversary (HAWK) events from MISP.
[2023-03-31 19:16:15,240] INFO     processor/main       Completed deletion of CrowdStrike HAWK adversaries within MISP in 0.24 seconds
[2023-03-31 19:16:15,240] INFO     processor/main       Start clean up of CrowdStrike Adversary (KITTEN) events from MISP.
[2023-03-31 19:16:15,328] INFO     processor/main       Completed deletion of CrowdStrike KITTEN adversaries within MISP in 0.08 seconds
[2023-03-31 19:16:15,328] INFO     processor/main       Start clean up of CrowdStrike Adversary (LEOPARD) events from MISP.
[2023-03-31 19:16:15,432] INFO     processor/main       Completed deletion of CrowdStrike LEOPARD adversaries within MISP in 0.10 seconds
[2023-03-31 19:16:15,432] INFO     processor/main       Start clean up of CrowdStrike Adversary (LYNX) events from MISP.
[2023-03-31 19:16:15,516] INFO     processor/main       Completed deletion of CrowdStrike LYNX adversaries within MISP in 0.08 seconds
[2023-03-31 19:16:15,516] INFO     processor/main       Start clean up of CrowdStrike Adversary (OCELOT) events from MISP.
[2023-03-31 19:16:15,592] INFO     processor/main       Completed deletion of CrowdStrike OCELOT adversaries within MISP in 0.07 seconds
[2023-03-31 19:16:15,592] INFO     processor/main       Start clean up of CrowdStrike Adversary (PANDA) events from MISP.
[2023-03-31 19:16:15,753] INFO     processor/main       Completed deletion of CrowdStrike PANDA adversaries within MISP in 0.16 seconds
[2023-03-31 19:16:15,753] INFO     processor/main       Start clean up of CrowdStrike Adversary (SPIDER) events from MISP.
[2023-03-31 19:16:15,872] INFO     processor/main       Completed deletion of CrowdStrike SPIDER adversaries within MISP in 0.11 seconds
[2023-03-31 19:16:15,873] INFO     processor/main       Start clean up of CrowdStrike Adversary (TIGER) events from MISP.
[2023-03-31 19:16:15,992] INFO     processor/main       Completed deletion of CrowdStrike TIGER adversaries within MISP in 0.11 seconds
[2023-03-31 19:16:15,992] INFO     processor/main       Start clean up of CrowdStrike Adversary (WOLF) events from MISP.
[2023-03-31 19:16:16,133] INFO     processor/main       Completed deletion of CrowdStrike WOLF adversaries within MISP in 0.14 seconds
[2023-03-31 19:16:16,133] INFO     processor/main       Start clean up of CrowdStrike CSA report events from MISP.
[2023-03-31 19:16:16,252] INFO     processor/main       Completed deletion of CrowdStrike CSA reports within MISP in 0.11 seconds
[2023-03-31 19:16:16,252] INFO     processor/main       Start clean up of CrowdStrike CSAR report events from MISP.
[2023-03-31 19:16:16,405] INFO     processor/main       Completed deletion of CrowdStrike CSAR reports within MISP in 0.15 seconds
[2023-03-31 19:16:16,405] INFO     processor/main       Start clean up of CrowdStrike CSIR report events from MISP.
[2023-03-31 19:16:16,538] INFO     processor/main       Completed deletion of CrowdStrike CSIR reports within MISP in 0.13 seconds
[2023-03-31 19:16:16,538] INFO     processor/main       Start clean up of CrowdStrike CSDR report events from MISP.
[2023-03-31 19:16:16,646] INFO     processor/main       Completed deletion of CrowdStrike CSDR reports within MISP in 0.10 seconds
[2023-03-31 19:16:16,646] INFO     processor/main       Start clean up of CrowdStrike CSIT report events from MISP.
[2023-03-31 19:16:16,736] INFO     processor/main       Completed deletion of CrowdStrike CSIT reports within MISP in 0.08 seconds
[2023-03-31 19:16:16,736] INFO     processor/main       Start clean up of CrowdStrike CSGT report events from MISP.
[2023-03-31 19:16:16,868] INFO     processor/main       Completed deletion of CrowdStrike CSGT reports within MISP in 0.13 seconds
[2023-03-31 19:16:16,868] INFO     processor/main       Start clean up of CrowdStrike CSIA report events from MISP.
[2023-03-31 19:16:16,944] INFO     processor/main       Completed deletion of CrowdStrike CSIA reports within MISP in 0.07 seconds
[2023-03-31 19:16:16,944] INFO     processor/main       Start clean up of CrowdStrike CSQR report events from MISP.
[2023-03-31 19:16:17,192] INFO     processor/main       Completed deletion of CrowdStrike CSQR reports within MISP in 0.24 seconds
[2023-03-31 19:16:17,192] INFO     processor/main       Start clean up of CrowdStrike CSMR report events from MISP.
[2023-03-31 19:16:17,452] INFO     processor/main       Completed deletion of CrowdStrike CSMR reports within MISP in 0.26 seconds
[2023-03-31 19:16:17,452] INFO     processor/main       Start clean up of CrowdStrike CSTA report events from MISP.
[2023-03-31 19:16:17,553] INFO     processor/main       Completed deletion of CrowdStrike CSTA reports within MISP in 0.10 seconds
[2023-03-31 19:16:17,553] INFO     processor/main       Start clean up of CrowdStrike CSWR report events from MISP.
[2023-03-31 19:16:17,638] INFO     processor/main       Completed deletion of CrowdStrike CSWR reports within MISP in 0.08 seconds
[2023-03-31 19:16:17,639] INFO     processor/main       Start clean up of CrowdStrike TEST report events from MISP.
[2023-03-31 19:16:17,713] INFO     processor/main       Completed deletion of CrowdStrike TEST reports within MISP in 0.07 seconds
[2023-03-31 19:16:17,713] INFO     processor/main       Start clean up of CrowdStrike HASH_MD5 indicator events from MISP.
[2023-03-31 19:16:17,811] INFO     processor/main       Start clean up of CrowdStrike HASH_SHA256 indicator events from MISP.
[2023-03-31 19:16:17,897] INFO     processor/main       Start clean up of CrowdStrike HASH_SHA1 indicator events from MISP.
[2023-03-31 19:16:17,982] INFO     processor/main       Start clean up of CrowdStrike HASH_IMPHASH indicator events from MISP.
[2023-03-31 19:16:18,268] INFO     processor/main       Start clean up of CrowdStrike FILE_NAME indicator events from MISP.
[2023-03-31 19:16:18,355] INFO     processor/main       Start clean up of CrowdStrike FILE_PATH indicator events from MISP.
[2023-03-31 19:16:18,426] INFO     processor/main       Start clean up of CrowdStrike URL indicator events from MISP.
[2023-03-31 19:16:18,536] INFO     processor/main       Start clean up of CrowdStrike MUTEX_NAME indicator events from MISP.
[2023-03-31 19:16:18,618] INFO     processor/main       Start clean up of CrowdStrike BITCOIN_ADDRESS indicator events from MISP.
[2023-03-31 19:16:18,718] INFO     processor/main       Start clean up of CrowdStrike COIN_ADDRESS indicator events from MISP.
[2023-03-31 19:16:18,795] INFO     processor/main       Start clean up of CrowdStrike EMAIL_ADDRESS indicator events from MISP.
[2023-03-31 19:16:18,871] INFO     processor/main       Start clean up of CrowdStrike EMAIL_SUBJECT indicator events from MISP.
[2023-03-31 19:16:18,949] INFO     processor/main       Start clean up of CrowdStrike REGISTRY indicator events from MISP.
[2023-03-31 19:16:19,156] INFO     processor/main       Start clean up of CrowdStrike DEVICE_NAME indicator events from MISP.
[2023-03-31 19:16:19,336] INFO     processor/main       Start clean up of CrowdStrike DOMAIN indicator events from MISP.
[2023-03-31 19:16:19,466] INFO     processor/main       Start clean up of CrowdStrike CAMPAIGN_ID indicator events from MISP.
[2023-03-31 19:16:19,640] INFO     processor/main       Start clean up of CrowdStrike IP_ADDRESS indicator events from MISP.
[2023-03-31 19:16:19,724] INFO     processor/main       Start clean up of CrowdStrike SERVICE_NAME indicator events from MISP.
[2023-03-31 19:16:19,831] INFO     processor/main       Start clean up of CrowdStrike USER_AGENT indicator events from MISP.
[2023-03-31 19:16:19,928] INFO     processor/main       Start clean up of CrowdStrike PORT indicator events from MISP.
[2023-03-31 19:16:19,990] INFO     processor/main       Start clean up of CrowdStrike PASSWORD indicator events from MISP.
[2023-03-31 19:16:20,247] INFO     processor/main       Start clean up of CrowdStrike USERNAME indicator events from MISP.
[2023-03-31 19:16:20,377] INFO     processor/main       Start clean up of CrowdStrike X509_SERIAL indicator events from MISP.
[2023-03-31 19:16:20,509] INFO     processor/main       Start clean up of CrowdStrike X509_SUBJECT indicator events from MISP.
[2023-03-31 19:16:20,692] INFO     processor/main       Start clean up of CrowdStrike MD5 hashes indicator type events from MISP.
[2023-03-31 19:16:20,757] INFO     processor/main       Start clean up of CrowdStrike SHA256 hashes indicator type events from MISP.
[2023-03-31 19:16:20,849] INFO     processor/main       Start clean up of CrowdStrike SHA1 hashes indicator type events from MISP.
[2023-03-31 19:16:20,939] INFO     processor/main       Start clean up of CrowdStrike IMP hashes indicator type events from MISP.
[2023-03-31 19:16:21,133] INFO     processor/main       Start clean up of CrowdStrike File names indicator type events from MISP.
[2023-03-31 19:16:21,235] INFO     processor/main       Start clean up of CrowdStrike File directory paths indicator type events from MISP.
[2023-03-31 19:16:21,374] INFO     processor/main       Start clean up of CrowdStrike Web addresses indicator type events from MISP.
[2023-03-31 19:16:21,498] INFO     processor/main       Start clean up of CrowdStrike Mutexes indicator type events from MISP.
[2023-03-31 19:16:21,598] INFO     processor/main       Start clean up of CrowdStrike BTC addresses indicator type events from MISP.
[2023-03-31 19:16:21,715] INFO     processor/main       Start clean up of CrowdStrike BIC addresses indicator type events from MISP.
[2023-03-31 19:16:21,805] INFO     processor/main       Start clean up of CrowdStrike Email addresses indicator type events from MISP.
[2023-03-31 19:16:21,943] INFO     processor/main       Start clean up of CrowdStrike Email subjects indicator type events from MISP.
[2023-03-31 19:16:22,116] INFO     processor/main       Start clean up of CrowdStrike Registry key locations indicator type events from MISP.
[2023-03-31 19:16:22,210] INFO     processor/main       Start clean up of CrowdStrike Device host names indicator type events from MISP.
[2023-03-31 19:16:22,292] INFO     processor/main       Start clean up of CrowdStrike Web domains indicator type events from MISP.
[2023-03-31 19:16:22,355] INFO     processor/main       Start clean up of CrowdStrike Campaign IDs indicator type events from MISP.
[2023-03-31 19:16:22,445] INFO     processor/main       Start clean up of CrowdStrike IP addresses indicator type events from MISP.
[2023-03-31 19:16:22,581] INFO     processor/main       Start clean up of CrowdStrike Service names indicator type events from MISP.
[2023-03-31 19:16:22,726] INFO     processor/main       Start clean up of CrowdStrike User-Agent strings indicator type events from MISP.
[2023-03-31 19:16:22,896] INFO     processor/main       Start clean up of CrowdStrike TCP ports indicator type events from MISP.
[2023-03-31 19:16:22,964] INFO     processor/main       Start clean up of CrowdStrike Password credentials indicator type events from MISP.
[2023-03-31 19:16:23,216] INFO     processor/main       Start clean up of CrowdStrike Credential user names indicator type events from MISP.
[2023-03-31 19:16:23,360] INFO     processor/main       Start clean up of CrowdStrike Certificate serial numbers indicator type events from MISP.
[2023-03-31 19:16:23,465] INFO     processor/main       Start clean up of CrowdStrike Certificate subjects indicator type events from MISP.
[2023-03-31 19:16:23,682] INFO     processor/main       Start clean up of CrowdStrike malware family indicator events from MISP.
[2023-03-31 19:16:23,806] INFO     processor/main       Completed deletion of CrowdStrike indicators within MISP in 6.09 seconds
[2023-03-31 19:16:23,806] INFO     processor/main       Finished cleaning up CrowdStrike related events from MISP, 0 events deleted.
[2023-03-31 19:16:23,806] INFO     processor/main       BEGIN DELETE
[2023-03-31 19:16:23,807] INFO     processor/main       Retrieving list of tags to remove from MISP instance (may take several minutes).
[2023-03-31 19:16:24,194] INFO     processor/main       Finished cleaning up CrowdStrike related tags from MISP, 0 tags deleted.
[2023-03-31 19:16:24,194] INFO     misp_tools    FINISHED

packet-rat commented 1 year ago

I believe I may have found the Root Cause for this issue:

I'd deleted the CrowdStrike Tags as part of another set of Tag related Issues.

I suspect you may be totally relying on Tags to identify the Obliterate target list. If so, a more resilient methodology would include criteria like Organization. We'd still need some parameter driven tag specification. In our case our AT&T specific Tags we apply would be used for the deletions that focus on a given category (i.e., Actors, Reports, Indicators).

Let's validate the assumption that bulk deletions are currently based solely on Tags and take it from there?

packet-rat commented 1 year ago

After doing a manual BulkDeletion, FullMonty, and then an Obliterate - the following artifacts remained:

jshcodes commented 1 year ago

Interesting. The two events above do not have the CrowdStrike tag used to identify records for deletion. I've seen this before, but only when I crashed / stopped import in a non-graceful manner.

jshcodes commented 1 year ago

If you cleared the tags, then I'm sure that's the cause. I've tried adding in org ID to the filter used to select deleted records, and that doesn't seem to work using pymisp. That was a couple versions back, so I can retry this in a follow up revision.

Closing this one for now, as I think we've found it. Feel free to reopen if you find another example.

CrowdStrike / MISP-tools

Obliterate and Clean <xxx> are no longer working #109