Clear Tags Operation Fails

[packet-rat]

Clear Tags fails with two errors:

• cannot convert dictionary update sequence element #0 to a sequence
• MISP Error: (405, {'name': 'You do not have permission to use this

python3 misp_import.py -ct

[2022-12-17 21:23:22,858] INFO     misp_tools    
[2022-12-17 21:23:22,858] INFO     misp_tools    '##::::'##:'####::'######::'########:::::'########::'#######:::'#######::'##::::::::'######::
[2022-12-17 21:23:22,858] INFO     misp_tools     ###::'###:. ##::'##... ##: ##.... ##::::... ##..::'##.... ##:'##.... ##: ##:::::::'##... ##:
[2022-12-17 21:23:22,858] INFO     misp_tools     ####'####:: ##:: ##:::..:: ##:::: ##::::::: ##:::: ##:::: ##: ##:::: ##: ##::::::: ##:::..::
[2022-12-17 21:23:22,858] INFO     misp_tools     ## ### ##:: ##::. ######:: ########:::::::: ##:::: ##:::: ##: ##:::: ##: ##:::::::. ######::
[2022-12-17 21:23:22,858] INFO     misp_tools     ##. #: ##:: ##:::..... ##: ##.....::::::::: ##:::: ##:::: ##: ##:::: ##: ##::::::::..... ##:
[2022-12-17 21:23:22,858] INFO     misp_tools     ##:.:: ##:: ##::'##::: ##: ##:::::::::::::: ##:::: ##:::: ##: ##:::: ##: ##:::::::'##::: ##:
[2022-12-17 21:23:22,858] INFO     misp_tools     ##:::: ##:'####:. ######:: ##:::::::::::::: ##::::. #######::. #######:: ########:. ######::
[2022-12-17 21:23:22,858] INFO     misp_tools    ..:::::..::....:::......:::..:::::::::::::::..::::::.......::::.......:::........:::......:::
[2022-12-17 21:23:22,858] INFO     misp_tools               _____
[2022-12-17 21:23:22,858] INFO     misp_tools                /  '
[2022-12-17 21:23:22,858] INFO     misp_tools             ,-/-,__ __
[2022-12-17 21:23:22,858] INFO     misp_tools            (_/  (_)/ (_
[2022-12-17 21:23:22,858] INFO     misp_tools                         _______                        __ _______ __        __ __
[2022-12-17 21:23:22,858] INFO     misp_tools                        |   _   .----.-----.--.--.--.--|  |   _   |  |_.----|__|  |--.-----.
[2022-12-17 21:23:22,858] INFO     misp_tools                        |.  1___|   _|  _  |  |  |  |  _  |   1___|   _|   _|  |    <|  -__|
[2022-12-17 21:23:22,858] INFO     misp_tools                        |.  |___|__| |_____|________|_____|____   |____|__| |__|__|__|_____|
[2022-12-17 21:23:22,858] INFO     misp_tools                        |:  1   |                         |:  1   |
[2022-12-17 21:23:22,858] INFO     misp_tools                        |::.. . |                         |::.. . |  Threat Intelligence v0.6.5
[2022-12-17 21:23:22,858] INFO     misp_tools                        `-------'                         `-------'
[2022-12-17 21:23:22,858] INFO     misp_tools    
[2022-12-17 21:23:22,858] INFO     config  
[2022-12-17 21:23:22,858] INFO     config  _______ _     _ _______ _______ _     _      _______  _____  __   _ _______ _____  ______
[2022-12-17 21:23:22,858] INFO     config  |       |_____| |______ |       |____/       |       |     | | \  | |______   |   |  ____
[2022-12-17 21:23:22,858] INFO     config  |_____  |     | |______ |_____  |    \_      |_____  |_____| |  \_| |       __|__ |_____|
[2022-12-17 21:23:22,858] INFO     config  
[2022-12-17 21:23:22,860] WARNING  config  misp_enable_ssl                             SSL is disabled for MISP API requests
[2022-12-17 21:23:23,759] INFO     config  No configuration errors found (1 warning)
[2022-12-17 21:23:23,759] INFO     config  
[2022-12-17 21:23:23,759] INFO     config  ____ _  _ ____ ____ _  _ ____    ___  ____ ____ ____ ____ ___
[2022-12-17 21:23:23,759] INFO     config  |    |__| |___ |    |_/  [__     |__] |__| [__  [__  |___ |  \
[2022-12-17 21:23:23,759] INFO     config  |___ |  | |___ |___ | \_ ___]    |    |  | ___] ___] |___ |__/
[2022-12-17 21:23:23,759] INFO     config  
[2022-12-17 21:23:24,362] INFO     processor/main       
[2022-12-17 21:23:24,362] INFO     processor/main       ______  _______        _______ _______ _______
[2022-12-17 21:23:24,362] INFO     processor/main       |     \ |______ |      |______    |    |______
[2022-12-17 21:23:24,362] INFO     processor/main       |_____/ |______ |_____ |______    |    |______
[2022-12-17 21:23:24,362] INFO     processor/main       
[2022-12-17 21:23:24,362] INFO     processor/main       Retrieving list of tags to remove from MISP instance
[2022-12-17 21:23:25,088] WARNING  processor/thread_1   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,089] WARNING  processor/thread_1   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,125] WARNING  processor/thread_2   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,126] WARNING  processor/thread_0   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,126] WARNING  processor/thread_2   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,133] WARNING  processor/thread_5   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,133] WARNING  processor/thread_8   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,134] WARNING  processor/thread_0   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,151] WARNING  processor/thread_10  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,152] WARNING  processor/thread_14  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,152] WARNING  processor/thread_3   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,153] WARNING  processor/thread_4   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,159] WARNING  processor/thread_6   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,166] WARNING  processor/thread_7   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,423] WARNING  processor/thread_7   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,179] WARNING  processor/thread_15  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,214] WARNING  processor/thread_12  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,223] WARNING  processor/thread_11  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,442] WARNING  processor/thread_11  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,236] WARNING  processor/thread_13  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,443] WARNING  processor/thread_13  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,285] WARNING  processor/thread_10  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,331] WARNING  processor/thread_14  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,347] WARNING  processor/thread_3   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,363] WARNING  processor/thread_4   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,418] WARNING  processor/thread_6   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,178] WARNING  processor/thread_9   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,443] WARNING  processor/thread_9   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,439] WARNING  processor/thread_15  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,442] WARNING  processor/thread_12  Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,229] WARNING  processor/thread_5   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,259] WARNING  processor/thread_8   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,442] WARNING  processor/thread_1   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,444] WARNING  processor/thread_1   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,506] WARNING  processor/thread_2   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,506] WARNING  processor/thread_2   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,603] WARNING  processor/thread_0   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,603] WARNING  processor/thread_0   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,754] WARNING  processor/thread_7   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,755] WARNING  processor/thread_7   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,771] WARNING  processor/thread_4   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,771] WARNING  processor/thread_4   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,799] WARNING  processor/thread_14  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,799] WARNING  processor/thread_14  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,801] WARNING  processor/thread_13  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,801] WARNING  processor/thread_13  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,818] WARNING  processor/thread_10  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,818] WARNING  processor/thread_8   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,819] WARNING  processor/thread_15  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,819] WARNING  processor/thread_9   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,819] WARNING  processor/thread_10  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,819] WARNING  processor/thread_8   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,819] WARNING  processor/thread_15  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,819] WARNING  processor/thread_9   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,822] WARNING  processor/thread_3   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,822] WARNING  processor/thread_12  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,823] WARNING  processor/thread_5   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,823] WARNING  processor/thread_3   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,823] WARNING  processor/thread_12  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,823] WARNING  processor/thread_5   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,827] WARNING  processor/thread_11  cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,828] WARNING  processor/thread_6   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:25,828] WARNING  processor/thread_11  Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:25,828] WARNING  processor/thread_6   Retrying request in 0.60 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:26,073] ERROR    processor/thread_1   Unresolvable error received from the MISP server.
[2022-12-17 21:23:26,073] ERROR    processor/thread_1   MISP Error: (405, {'name': 'You do not have permission to use this functionality.', 'message': 'You do not have permission to use this functionality.', 'url': '/tags/delete/93513'})
[2022-12-17 21:23:26,073] ERROR    processor/thread_1   Exceeded number of retries. (╯°□°）╯︵ ┻━┻
[2022-12-17 21:23:26,113] WARNING  processor/thread_1   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:26,113] WARNING  processor/thread_1   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:26,135] ERROR    processor/thread_2   Unresolvable error received from the MISP server.
[2022-12-17 21:23:26,135] ERROR    processor/thread_2   MISP Error: (405, {'name': 'You do not have permission to use this functionality.', 'message': 'You do not have permission to use this functionality.', 'url': '/tags/delete/93514'})
[2022-12-17 21:23:26,135] ERROR    processor/thread_2   Exceeded number of retries. (╯°□°）╯︵ ┻━┻
[2022-12-17 21:23:26,161] WARNING  processor/thread_2   cannot convert dictionary update sequence element #0 to a sequence
[2022-12-17 21:23:26,161] WARNING  processor/thread_2   Retrying request in 0.30 seconds. ¯\_(ツ)_/¯
[2022-12-17 21:23:26,231] ERROR    processor/thread_0   Unresolvable error received from the MISP server.
[2022-12-17 21:23:26,231] ERROR    processor/thread_0   MISP Error: (405, {'name': 'You do not have permission to use this functionality.', 'message': 'You do not have permission to use this functionality.', 'url': '/tags/delete/93512'})
[2022-12-17 21:23:26,231] ERROR    processor/thread_0   Exceeded number of retries. (╯°□°）╯︵ ┻━┻
<=SNIP=>

[jshcodes]

I'm having a hard time recreating this one still. (I haven't seen the "You do not have permission" message before.)

I have added the strict_tagname keyword to the tag lookup to reduce potential mismatches against synonyms, etc.

Is this a consistent failure regardless of INI settings?

[packet-rat]

Strict Tag Name should be fine. What specific settings should I validate? Without release notes I have to guess at new additions/changes to the ini file (I do a diff and try to reconcile)

[jshcodes]

The only real settings that should impact this would be tag names and potentially org_id (but not really, I'm just now testing this additional filter).

It crashes like this every single time?

[packet-rat]

Yes. Let me change the API User to Admin.

[packet-rat]

Changing the API Key user to a full Admin role cleared the issue. We use a “Sync User as Tagger” Role for all API Accounts:

[jshcodes]

This is helpful. Adding documentation detail for necessary MISP server permissions is now on the list.

Thank you! 😄

[packet-rat]

We’ve asked CIRCL to “fix” the Tag Editor Role. This Role doesn’t allow one to actually Edit Tags!

https://github.com/MISP/MISP/issues/8787