Reorganise TypeDBSource/TypeDBSink 2: Create get_list() and get_objects()

[brettforbes]

1 . Object and Method Model should look like this

1. TypeDBSink:

• add(data) -> add function, where data is a list, dict or bundle

2. TypeDBSource:

• clear_db() -> clear all except for marking objects
• delete(stix-id) -> delete the object with the stix-id
• get_stix_ids() - get all of the ids in the db except for marking
• get(stix-id) - get a single stix object with the id
• get_list(stix_id_list) -> get a list of stix objects based on a list of stix-ids (parallelised)
• get_objects(obj, properties, embedded, sub_prop, import_type): - > get a set of objects of a type, based on constraints

3. TypeDBStore:

• all methods for the mixin class

2. Task for this Issue

1. Implement function https://github.com/os-threat/Stix-ORM/blob/incident/try_refactor.py#L1553, changing the auth object usage for mapping function
2. Implement function https://github.com/os-threat/Stix-ORM/blob/incident/try_refactor.py#L1531, but set it up so it automatically parallelises and runs as fast as possible

[brettforbes]

Note that here there is an existing test running system for the get_objects() method that works correctly https://github.com/os-threat/Stix-ORM/blob/incident/try_refactor.py#L1592C5-L1592C21

[brettforbes]

hey, @dfjosullivan thanks for the current, it is really good, especially the logging, which is sick,

However, I really neeed the above complex get function, and i notice that the links i put in above are really screwed up. Sorry about that, my fault completely

I really need to use this capability to get objects for Type Refinery.

ITs essential to do things like get lists of objects, or objects that fulfil a value criteria. Check out the signature below def get_objects(obj, properties, embedded=[], sub_prop=[], import_type=import_type):

    """Interface for getting one or more STIX objects from TypeDB.

    Can be based on object tpe, with property constraints, embedded and sub-object constraints

    Args:
        - obj_typeql (string) - a valid Stx-ORM object that exists in the database
        - properties ([dict]) - a list of dicts providing comparisons between properties and constants
                                  - dict:
                                          - property-name - a TypeDB property
                                          - comparator - a two letter comparison
                                          - constant - a constant against which the property value is compared
        - embedded([stix-id]) - a list of valid stix-ids that exist in the database
        - sub_prop([dict]) - a list of valid stix-ids that exist in the database
                                  - dict:
                                          - sub-object typeql name
                                          - property-name - a TypeDB property
                                          - comparator - a two letter comparison
                                          - constant - a constant against which the property value is compared

    """```

There is a test function directly below it, and below that a sub-function of get_objects()
Its at this line on main branch. [](https://github.com/os-threat/Stix-ORM/blob/main/try_refactor.py#L1699 )

And also please accept a few schema fixes on main, due to some errors i found in the mapping (e.g. variable of string instead of boolean).

We can go ahead and build the user interface if we get the `get_objects()` method above and this `get_list(ids)` method below, which is super simple

def get_list(stix_id_list): ''' TypeDBSource Method To be poarallelised and sped-up by Denis, main

Args:
    - stix_id_list ([stix-id]) - a list of valid stix-id's that exists in the database

Returns
    - list of stix objects or an error message
'''
obj_list = []
typedb = TypeDBSource(connection, import_type)
for stix_id in stix_id_list:
    obj = typedb.get(stix_id)
    obj_list.append(obj)

return obj_list```

[brettforbes]

Plus this one here, super simple to start with, could be improved

def get_list(stix_id_list):
    '''
    TypeDBSource Method
    To be poarallelised and sped-up by Denis, main

    Args:
        - stix_id_list ([stix-id]) - a list of valid stix-id's that exists in the database

    Returns
        - list of stix objects or an error message
    '''
    obj_list = []
    typedb = TypeDBSource(connection, import_type)
    for stix_id in stix_id_list:
        obj = typedb.get(stix_id)
        obj_list.append(obj)

    return obj_list```