Assets created by one instance can't be seen by another instance on same MySQL DB

[dukeofurl]

Hey guys, we're setting up Collins for CMDB use and loving it so far, but hit a snag when trying to create an HA cluster. Throughout exhaustive testing, it seems that Collins instances can only see assets they have created themselves. More info:

• 2 instances created from the same official tumblr/collins docker image, connecting to the same MySQL DB, with the same ID (root being used for this test).
• Assets written by instance A can't be seen by instance B, and likewise the other way around. Directly querying the asset by name brings back results, but only in the API - the web UI never finds them.
• Updating an asset by directly calling it by name and changing a value on it makes it visible to the other instance, but this isn't really a feasible workaround if we have many assets haha
• All records are visible in the MySQL DB using a SELECT * query. No distinguishing "ownership" fields were found when walking through the schema
• Same problem occurred with Hazelcast, then cache was turned off completely and verified off via the admin/cache page.
• If containers A and B are deleted and a new container, C, is created from the same official tumblr/collins docker image (and connected to the same MySQL DB with data from previous instances), it can't see any assets created by A and B, only ones it has created.
• Same issue occurs with multicollins activated.
• application.secret matches across all instances
• crypto.key matches as well, but no encrypted keys are in use

Hopefully we're just missing some configuration option, but if you need any thing else to track down this bug(?), I'm here to help.

Thanks!

[dukeofurl]

Through deeper inspection of all open file handles in the containers running Collins, we found lucene indexes in use. Solr stores lucene index files locally, and in order to implement either mulicollins or clustered collins instances, an external Solr instance has to be set up, but this is not mentioned in the documentation. Problem has been resolved, but documentation of this requirement would have saved a few days work.

[defect]

Hi, sorry for the very slow response.

Yes, in order to use HA collins you need a shared Solr instance. There is also some hazelcast functionality that needs to be configured for cache coherency between collins instances.

In all honesty, the area of HA collins hasn't been very well used (we use a hot/standby configuration) and put through the tests of productionizing (as far as I know), which is probably why it isn't very well documented.