Missing tables with "fast translation"

When a stix-shifter connector's "to_stix_map" doesn't use an object name in a mapping, those objects could be silently dropped by async translate/ingest. This happens e.g. with qradar's software:name mapping. You can tell by inspecting the DB (in this case PostgreSQL but also happens with sqlite3):

  otype   | path | shortname | dtype 
----------+------+-----------+-------
 software | name | name      | str
(1 row)
# \d
              List of relations
 Schema  |      Name       | Type  |  Owner   
---------+-----------------+-------+----------
 flat-id | __columns       | table | postgres
 flat-id | __contains      | table | postgres
 flat-id | __metadata      | table | postgres
 flat-id | __queries       | table | postgres
 flat-id | __symtable      | table | postgres
 flat-id | artifact        | table | postgres
 flat-id | domain-name     | table | postgres
 flat-id | email-message   | table | postgres
 flat-id | file            | table | postgres
 flat-id | identity        | table | postgres
 flat-id | ipv4-addr       | table | postgres
 flat-id | ipv6-addr       | table | postgres
 flat-id | network-traffic | table | postgres
 flat-id | observed-data   | table | postgres
 flat-id | url             | table | postgres
 flat-id | x-ibm-finding   | table | postgres
 flat-id | x-oca-event     | table | postgres
 flat-id | x-qradar        | table | postgres
(18 rows)

From this example you can see that firepit recorded a software:name column (meaning it was in the native qradar data passed into the translate function) which means there should be a software table with id and name columns, but listing the tables in the database shows that software is missing.

opencybersecurityalliance / firepit

Missing tables with "fast translation" #109