COO Dictionary: Add contains_refs support

[StephenOTT]

Add contains_refs support. Requires other objects to exist like File and Directory Types

[anlklsim]

A solution needs to be used across all the CyberObservableObjects where _ref or _refs are attributes. If we don't want to use a Map for the object_refs and manually set the key/index when we create the object, we'll have to have a complicated serializer/deserializer to hydrate the objects to Java. Here's an example for email objects: Simple Email Message { "0": { "type": "email-addr", "value": "jdoe@example.com", "display_name": "John Doe", "belongs_to_ref":"3" }, "1": { "type": "email-addr", "value": "mary@example.com", "display_name": "Mary Smith", "belongs_to_ref":"4" }, "2": { "type": "email-message", "from_ref": "0", "to_refs": ["1"], "is_multipart": false, "date": "1997-11-21T15:55:06.000Z", "subject": "Saying Hello" }, "3": { "type":"user-account", "user_id":"jdoe" }, "4": { "type":"user-account", "user_id":"msmith" } }

[StephenOTT]

Ya. Not sure I want to do hydration again. It's a pain. Not happy about it in the SDO and relationships.

But this dictionary is even more complicated as it's using number values rather than UUID.... I am leaning towards have a extra method that does the lookup on the map. Let them define their own keys and the method will search for the key to create the relationship only when the user wants to requests the data.

[anlklsim]

I looked in the oasis python stix lib and they have objects defined as a map (although almost everything is a dictionary in python anyway). I've been trying to think of an elegant way to get around having a map and the user dealing with the keys. A Serializer/Des. would have to be at the Observed_Data level in order to keep track of the integer keys and assign them within the COO's that have refs (that's the hard part). How to pass the last int assigned through to the rest of the object tree? As in the example above, object 3 and 4 are user-accounts, referenced from objects 0 and 1, which are in turn referenced from object 2. The whole object tree would need to be deserialized together in order to increment the key integers. Although I’d like to have static type checking in the classes, I think there is enough reflection and hydration going on with the Immutables and Validation. We could add another annotation to say what the reference is supposed to point to if needed.

I would prefer just to keep references as String representation of an integer and have the objects be a Map<String,Object> and the user maintain the integer strings. If we do that, I’d say we can start using the library (and add more tests of course)

[StephenOTT]

I worked this out this morning: the issue is the ahead of deserialization with immutables. And I don't want us to maintain a large deserializer just for refs. So yes I did as you thought: refs are mapped as Strings. Everything else is ready to go. Will be committing shortly. 👍🏻

[StephenOTT]

I also did not use a integer as a key. I introduced a new field in the observable that captures the key value. And it's managed as a set of objects in the ObservedData SDO. By default it will use UUID as the key, but the key can be manually set per object. This is then converted into a map like format during serialization, and back into a set during deserialization

[StephenOTT]

Committed