Issue with credit card normalisation

[himynamesdave]

 python3 txt2stix.py \
        --relationship_mode standard \
        --input_file tests/inputs/extraction_types/generic_bank_card_mastercard.txt \
        --name 'Test 3.1.34 pattern_bank_card_mastercard' \
        --tlp_level clear \
        --confidence 100 \
        --use_extractions pattern_bank_card_mastercard

Produces

bundle--ce871ba3-3850-4840-b24d-4abbdd03504a.json

You can see 2 dupe indicator are created for bank-card (5588601012060076)

The test file

generic_bank_card_mastercard:
  test_positive_examples:
    - '5588601012060076'
    - '5588 6010 1206 0076'

Has the same card number (one with spaces, one without).

I suspect the 2 indicators are being created because the creation is done before normalisation (remove spaces) of these two cards into one.

Expected result is 1 indicator, 1 bank card for each unique card number (once spaces removed).

[himynamesdave]

@fqrious it the same test now extracts 0 results

bundle--3539e8c5-e7fa-5a72-a0b4-c7a9717bb15c.json

[fqrious]

it's failing luhn test, I added luhn validation

[himynamesdave]

@fqrious the test

 python3 txt2stix.py \
        --relationship_mode standard \
        --input_file tests/inputs/extraction_types/generic_bank_card_mastercard.txt \
        --name 'Test 3.1.34 pattern_bank_card_mastercard' \
        --tlp_level clear \
        --confidence 100 \
        --use_extractions pattern_bank_card_mastercard

still fails with no extractions, expecting 1 extraction

[fqrious]

it's invalid