Exposing a simple/generic Sequel API for adding / updating / removing shared objects

[PetrosKataras]

At the moment there is no api exposed for manipulating an arbitrary number of user defined objects through the objectFactory / objectMap interface.

This will be an attempt to resolve this situation ..

An initial attempt that I have tries to follow the same paradigm as the frameData object. More specifically objectMap holds a vector of co::Object* with all user added objects.

The user objects are set in the same logic as the frame/initData object is being set in masterConfig through setInit/FrameData where UUID's are being assigned..

Custom objects are then being registered through ObjectMap::register_ on MasterConfig::run ..

This seems to work for registering the objects properly but I am not sure if this would be the proper way. Also not sure what would be the best way to go about the mapping / syncing of the objects on slave instances.

At the moment I tried to explicitly map the user objects in a similar way as ObjectMap::getFrameData provides but I dont think this is the correct way to continue..

These are some initial thoughts.. More to follow..

[eile]

Both Application and Renderer should expose an API following the internal ObjectMap (register/deregister/map/unmapObject). The ObjectMap will then track synchronization.

[PetrosKataras]

Should both Application and Renderer use the default seq::detail::ObjectMap for registering/mapping etc. or would be better to keep a separate ObjectMap just for the user data ?

Also should the default behaviour be automatic creation based on type or explicit creation of the objects ?

[eile]

I would use the existing object maps, since they are already set up and synchronized correctly.

Objects should be created explicitly (lazy) when the application access them the first time on the clients.

[PetrosKataras]

I was wondering if its safe to freely call functions like register/map etc ( after the object maps have been created ) from the Application / Renderer side or if there is some order that has to be preserved ..

From what I understand now the object maps are being created and used internally in Slave/MasterConfig on the application side and in Pipe for the renderers ..

I guess what I am asking is if it would be safe to expose these specific objectMaps to the application and renderer instances and de/register , un/map * at will * from there ??

[PetrosKataras]

Something like Application::setObjectMap( ObjectMap _objectMap ) that can then be set from MasterConfig for example like this :

getAppImpl()->setObjectMap( _objects )

And then register from the application side with something like :

getAppImpl()->getObjectMap()->register_( object )

Hope it makes sense ..

[eile]

The register obviously has to happen before the map, and the unmap should happen before the deregister. The ordering between objects does not matter, unless the object serialization code makes any assumptions on it.

[eile]

I would simply set up forwarding functions in app and renderer, e.g., Application::registerObject()

[PetrosKataras]

I m a bit confused with the two versions of frameData that coexist in the seqPly example ..

The way I understand it so far is that the frameData object that is allocated on the stack is the object responsible for keeping in-sync frame specific data.

The frameData object that is created on the heap I thought that its there for illustrating one possible way of creating new shared objects..

What confuses me is that if I run the 2-node config the heap allocated master version of frameData has a different UUID from that of the slave version..

What am I missing here ??

[PetrosKataras]

Another thing that confuses me is the need for both application and renderer to expose the full objectMap api ..

By full, I mean de/register , un/map..

What I initially had in mind ( based also on the logic of the frameData object ) is that the application master needs to register / deregister through the objectMap created in MasterConfig and that the application slaves and renderers need to map / unmap through the object maps that are created in SlaveConfig and Pipe namely ..

Sorry if my questions sound trivial but I am trying to get my head around the internals of seq and unfortunately the documentation does not help me in this case..

[eile]

Re FrameData: This is a common pattern: You do have one instance in the application process' main thread, which is the master version being modified. Each render thread, including the ones in the application process, has a slave instance mapped to this master. These are needed for async execution, allowing the app to modify the master instance and the renderers sync to their version when appropriate. The UUID should be the same once the slave instance is mapped (it is constructed with a random one and mapObject() should set it).

[eile]

Re app/renderer exposure: This goes back to the previous comment. The app-mapped instances are once per process and should only be accessed from the renderer if you can be sure of thread-safety, and the renderer-mapped instances are one per thread and are therefore thread-safe within their renderer.

Renderer probably does not need register/deregister for now.

[PetrosKataras]

Much clearer now . Thanx for the info

[PetrosKataras]

Hey, coming back to this..

I have been trying a couple of stuff since yesterday and I managed to get something working in a "hackish" way so now I am reviewing in order to see how it could be done in a proper way.

Right now the idea is the following:

Application exposes a registerObject( co::Object* ) function which in turn forwards register and setUserData to the internal objectmap which I think does the job for registering properly on the app node.

The setUserData function works in a similar way like the setInit/FrameData function.

The objectmap holds a vector of all valid shared object UUID's that where created during setUserData on the app node and serializes those to the clients.

The clients then call a getUserData function similar to getFrameData that goes through the previously mentioned vector and maps the slave instances based on the de-serialized UUID vector which in turn causes the proper object creation through the object factory and createObject callback.

What I would like to achieve though based also on the previous discussion here is not having to use the the objectFactory approach .. This is where I start to get confused cause I am not sure how I can then properly map the slave instances since I dont have any info at that point on the client side about the relevant master instance.

[eile]

What is the difference between registerObject and setUserData? I would have expected to only see registerObject (used on master) and mapObject (used on slaves) in the public API. Plus their deregister/unmap friend, of course.

[eile]

If you don't have the ObjectFactory/Map, you need to recursively distribute your objects from init or frame data. What's your motivation of not using the objectFactory approach?

[PetrosKataras]

I forked and put something up to have something that we can discuss about. You can find it here : https://github.com/PetrosKataras/Equalizer/tree/sequel-user-object-registration

[PetrosKataras]

This just illustrates registration with OBJECTTYPE_CUSTOM for all user objects. I suppose registerUserObject should accept a second parameter with the user-defined type.

[PetrosKataras]

Deregister also added now. I am not against using the objectFactory per se. Probably it is me misunderstanding some concepts cause even through the objectFactory approach is still unclear to me how I could explicitly map an object on its master instance through a mapObject interface.

Right now I am just brutally going through all the UUID's that I am receiving on the slaves from here and I call map( UUID ) which then creates ( through the objectFactory ) the slave instance but this is completely incorrect and useless of cource.

[PetrosKataras]

So, not sure if I am going toward the right direction but now I can at least map user defined instances by explicitly passing an object ( and its type ) to the mapUserData function.

Deregistration would also have to be updated now.

[eile]

I suppose registerUserObject should accept a second parameter with the user-defined type

And then I would just call it registerObject() ;)

[eile]

still unclear to me how I could explicitly map an object on its master instance through a mapObject interface.

I don't understand 'on its master instance'. The mapObject would be used to map or create and map slave instances using the master object ID.

[eile]

Right now I am just brutally going through all the UUID's that I am receiving on the slaves from here and I call map( UUID ) which then creates ( through the objectFactory ) the slave instance but this is completely incorrect and useless of cource.

The map only happens when the application requests is upon accessing the object the first time.

[eile]

I think the only thing needed is to expose co::ObjectMap register_, deregister, map, unmap through the seq::Application and Renderer API.

[PetrosKataras]

I don't understand 'on its master instance'. The mapObject would be used to map or create and map slave instances using the master object ID. .... The map only happens when the application requests is upon accessing the object the first time.

Maybe a small example will illustrate what I am talking about :

Lets say that on the master you do something like this :

registerObject( object1, OBJECT_TYPE1 )
registerObject( object2, OBJECT_TYPE1 )
registerObject( object3, OBJECT_TYPE2 )

We have two objects of the same type and a third one of a different type.

Now, after syncing, the slave objectMap would receive those 3 master ID's which you can use on the clients side to map ( or create and map ) slave instances for those master instances right ?

Here is where it starts to become fuzzy for me..

Since right now on the slave objectMap the only info that I have are the 3 different ID's how I would go for creating and mapping a slave instance explicitly to object2 for example ?

mapObject( ? )

I am not sure how I could distinguish the object1 ID from object2 ID, for example ( on the client side ), in order to explicitly create and map a slave instance to the master object2.

Here, I would need to be able to say something like: mapObject( _object2ID ) but this mechanism of knowing which of the 3 available ID's on the client side is the one of object2 is what I am missing right now..

[eile]

Since right now on the slave objectMap the only info that I have are the 3 different ID's how I would go for creating and mapping a slave instance explicitly to object2 for example ?

mapObject( id ); will call mapObject( factory.createObject( type ));

The ObjectMap will track object types. The objects are distinguishable since each has a distinct UUID. The object id need to be unique, the type ids not.

[PetrosKataras]

Hi Stefan. I have uploaded here a version of seq that exposes on both application and renderer the full register / deregister, map / unmap api of the internal objectmap.

I have also added on the sequel example a minimal custom defined shared object illustrating registration.

If at any point you have some time to illustrate how you would go about mapping this object ( and thus any user defined distributed object ) it would be of great help to move forward with this.

[OlafRocket]

Hi Stefan

I have implemented a similar approach to that proposed by Petros and I have the same doubt. In my application I have a class which inherit from co::ObjectMap. When I add or remove items to the master instance of this class, I would like the slave instances to create and remove items during synchronization. In the documentation it is stated that "The objects on slave ObjectMap instances are mapped explicitly, either by providing an already constructed instance or using implicit object creation through the co::ObjectFactory.". When an item is added to the master instance of an ObjectMap, the new id is distributed to all the slave instances synchronizing the "ObjecMap::detail::ObjectMap IDVector added" list in the deserialize(), but only the id and the master version are saved and no instance or map(id) is called. Note that this behaviour is asymmetrical compared to the DIRTY_REMOVED case in deserialize() where _impl->_removeObject( it->second ) is called for each removed id. Where and when it is supposed the slave must use map(id) to create all the new instances? I have temporary resolved using the push pattern to distribute the new id + instance data to the slave instances, but I don't know if it could be the correct approach because it's not solving the "DIRTY_ALL" case.

[eile]

Where and when it is supposed the slave must use map(id) to create all the new instances?

Lazy, when the application calls map() on the ObjectMap.

Where and when it is supposed the slave must use map(id) to create all the new instances?

Whenever it needs to access the object.

It's asymmetric in the sense that we don't want to map all objects on all nodes by default. The remove will only delete the object if it was created by the ObjectMap in map() - so semantically the API is symmetric.

HTH - if not ask more. :)

[OlafRocket]

if I understand correctly, the list of Entries in ObjectMap slaves is used to group id and type, but no instances are attached to it or created during ObjectMap::deserialize. Only when ObjectMap::map(id) is called on the slaves, a real instance is created using the factory, or passed by the application. This is the lazy behaviour you are describing. This use case implies that the list of ids must be distributed through a different channel (a command, a shared file, etc.), otherwise the application using the ObjectMap slaves would not be able to call map(id) with an existing id.

My use case is slightly different: I need a map of co::Objects which is able to autonomously map and create new instances (possibly through the factory) when new objects are added to the master instance. It is very similar to the current ObjectMap but during sync the deserialize cases DIRTY_ALL and DIRTY_ADDED call ObjectMap::map(id) with the added id. I am trying to understand if I can extend co::ObjectMap with a derived class, but I cannot override deserialize because the map containing id, type and objects is hidden inside _impl .

Do I need to reimplement the whole class?

[eile]

This use case implies that the list of ids must be distributed through a different channel (a command, a shared file, etc.), otherwise the application using the ObjectMap slaves would not be able to call map(id) with an existing id.

Yes, although we could make the IDs available from the ObjectMap directly.

My use case is slightly different ...

Interesting, we did not have this one yet. We could add notify callbacks to add/remove from deserialize?

[OlafRocket]

Thanks for the fast reply.

About callbacks in deserialize, what do you think about adding two virtual functions: virtual void notifyObjectAdded(uint128_t id); -> deserialize calls it as the last function in the while loop for DIRTY_ALL and DIRTY_ADDED virtual void notifyObjectRemoved(uint128_t id); -> deserialize calls it before _removeObject in the DIRTY_REMOVED case.

My only concern is that during notifyObjectAdded I would call ObjectMap::map(id) and I don't know if it is the correct place or it can create any race condition.

About exposing the internal map. I think it would widen the possible use cases. A tentative interface: const IDVector getObjectIds() const; co::Object * getObject(uint128_t id); // the pointer can be null because there is no instance for the id

If you think all of the above make sense, I can implement it.

I really like working with Collage, it's a well conceived library and reading the documentation more than once it is possible to find a lot of information.

[eile]

The notify callbacks look good to me. Mapping from there is fine. I would probably use ObjectsAdded/Removed and pass the vector so one can efficiently used the async mapping.

[eile]

I would prefer not exposing the object IDs unless there is a concrete use case.

[OlafRocket]

I implemented the callbacks, but when I call map from the added callback the _impl->lock is already locked, so the mutex start spinning forever. I think I need to add a protected mapNB which doesn't lock, and probably also an unmap function, etc.. What do you think?

[eile]

I think the following is cleaner:

• Move added/removed vectors to the function scope
• Change lock scope to cover current code
• Call notify callbacks outside of the lock scope.

[OlafRocket]

ok, I have protected with the mutex only the access to the _impl->map container. To test it, I have subclassed the modified co::ObjectMap to create a container for my own (proxy) objects. I need to know when a new proxy is created or removed and the new callbacks are perfect for this use case. The callbacks receive a list of ids, but in the notifyObjectsToBeRemoved I need to access the mapped co::Object. There are 3 possible ways: 1) the parameter of the notifyObjectsToBeRemoved is a list of co::Object* instead of ids. 2) add co::Object * getObject(uint128_t id) to access a registered/mapped object by id. 3) let my subclass deals with it duplicating the map and add/remove actions. I think this use case could be enough to justify a partial exposure of the internal map, so I have implemented 2). Is it ok for you?

[eile]

Yes.