Closed stuarteberg closed 5 years ago
Here's the actual rule for general messages:
func (d *Data) ProduceKafkaMsg(b []byte) error {
// create topic (repo ID + data instance uuid)
// NOTE: Kafka server must be configured to allow topic creation from
// messages sent to a non-existent topic
rootuuid, _ := d.DAGRootUUID()
datauuid := d.DataUUID()
topic := "dvidrepo-" + string(rootuuid) + "-data-" + string(datauuid)
// send message if kafka initialized
return storage.KafkaProduceMsg(b, topic)
}
If you are using a kafka with topic creation ON you could potentially create an empty topic by querying it I think.
I have a question about the listed
RepoUUID
for a particular data instance.Have a look at the info for
segmentation_todo
:http://emdata3:8900/api/node/28841c8277e044a7b187dda03e18da13/segmentation_todo/info
TheRepoUUID
is listed as4ea3bd5698204089b82408c178fe1d55
But that is not the root node for the production repo. (The root node isa776af0b132f44c3a428fe7607ba0da0
) Is that expected? The weird thing is that the kafka server lists TWO different topics for this instance. Well, three if you count thehemiMirror
topic:dvidrepo-a776af0b132f44c3a428fe7607ba0da0-data-1e76b4afa159488b836a828afe772d86
dvidrepo-4ea3bd5698204089b82408c178fe1d55-data-1e76b4afa159488b836a828afe772d86
hemiMirror-dvidrepo-a776af0b132f44c3a428fe7607ba0da0-data-1e76b4afa159488b836a828afe772d86
However,
dvidrepo-4ea3bd5698204089b82408c178fe1d55-data-1e76b4afa159488b836a828afe772d86
contains no messages.Looking at other data instances, I guess the
RepoUUID
is defined as "the first node in which this data instance appeared"? But the KAFKA topic is defined using the ROOT node for the repo, regardless of the instance's declaredRepoUUID
? Is that the rule?