search

TeaEngineering / libchronicle

Shared-memory interprocess communication from C using OpenHFT's chronicle-queue protocol
Apache License 2.0
14 stars 10 forks source link

Chronicle Queue 5.x uses metadata.cq4t #11

Closed wsargent closed 2 years ago

wsargent commented 3 years ago

There are references to directory-listing and not metadata:

https://github.com/TeaEngineering/libchronicle/blob/master/native/shmipc.c#L140

https://github.com/TeaEngineering/libchronicle/blob/master/bindings/kdb/shm.q#L23

This file is not used in Chronicle Queue 5:

They are almost the same, except files directory-listing.cq4t was in earlier versions of Chronicle Queue, while metadata.cq4t is applicable for Chronicle Queue 5.0 onwards.

https://github.com/OpenHFT/Chronicle-Queue/blob/master/docs/FAQ.adoc#what-are-the-differences-between-the-files-directory-listing-cq4t-and-metadata-cq4t

shuckc commented 3 years ago

I've been through the changes in Chronicle Queue 4 -> 5, admittedly not in so much detail as when I wrote this library, and I didn't really like the direction the project has moved. The changes purport to fix a race when rolling queue files. It seemed to me that these could be fixed much more simply (within v4 spec) by holding the write lock in the active file whilst opening the next queue file and taking the next write lock there, then writing EOF signal to the original file and releasing it. So I'm torn between:

So at the moment I've stayed with v4 scheme, bug-for-bug as it were until the need arises. There's much work to be done to complete v4 support anyway (indexing, rolling) that it hasn't really arisen yet.

shuckc commented 3 years ago

Obviously if you are trying to tail a v5 writer with this library, then we might need some compatibility fixes. I'd certainly welcome code that does a stat to detect the new file, puts the queue handle into V5 mode and then reads from the new file. It seems that the presence of the new file is the indication the upgrade has occurred.

shuckc commented 1 year ago

v5 is now detected and handled. You can specify the desired version before you open the queue and this asserts the version matches. If you create the queue from scratch, your choice is used.