search

sirixdb / sirix

SirixDB is an an embeddable, bitemporal, append-only database system and event store, storing immutable lightweight snapshots. It keeps the full history of each resource. Every commit stores a space-efficient snapshot through structural sharing. It is log-structured and never overwrites data. SirixDB uses a novel page-level versioning approach.
https://sirix.io
BSD 3-Clause "New" or "Revised" License
1.09k stars 243 forks source link

test suite only works on linux #692

Closed christophsturm closed 6 months ago

christophsturm commented 7 months ago

it seem the test suite does not run on macos because one.jasyncfio is linux only. It would be great if this could be mentioned in the readme file. are there plans to support macos for running tests?

welcome[bot] commented 7 months ago

Thanks for opening your first issue 🦁 Your feedback, is the most valuable thing besides PRs 💚

JohannesLichtenberger commented 7 months ago

Can you check if it runs, when in ResourceConfiguration the storage type is set to file channel for instance? It's a bug and I think at least with this iouring implementation it's even slightly slower due to the event queue maybe. I'm currently on my phone

JohannesLichtenberger commented 7 months ago

I think I've changed it stupidly for testing lately. So, I've made a commit / push to fix the issue directly on the main branch.

JohannesLichtenberger commented 6 months ago

@christophsturm can you check if it works? As the documentation is rather messy, I'm working on a tutorial for a local shell setup (of course can be used as a library, too):

https://sirix.io/docs/jsoniq-tutorial.html

And I hope that

https://sirix.io/docs/cocepts.html

explains the architecture in some detail.

HTH Johannes

christophsturm commented 6 months ago

The test suite now works on my mac. I can't do more investigation currently because I have no time, but my usecase for sirix would be to use it as a cache and history for api requests that my service does to other hosts. so I want to add documents via a kotlin(java) api and also query versions of documents via an api. is that something that should be easy?

JohannesLichtenberger commented 6 months ago

I think that should be fine. However, I'm not sure if it'll create some overhead if your JSON data is super small for instance. On the other hand SirixDB is not production ready, but I'd love to make it more stable (and I think it's quite different and in some cases better suited maybe than other solutions, where simply indexes are stored in some kind of storage system, but there's no notion of of a coherent system). Though, help would be great, as I switched jobs recently from Java/Kotlin backend stuff to embedded systems with mostly C/C++...

As SirixDB keeps all secondary indexes up-to-date at all times and as well as a path summary (an unordered set of all paths in a resource stored in a tiny tree-structure), I think in some cases it's more flexible due to fine granular secondary indexes (for instance specific fields, but then based on the path summary also path indexes and path + value (content-and-structure / CAS indexes), based on ideas from the XML world (despite that almost everyone hates XML, I think the ideas developed in the context of XML databases are also applicable to JSON).

So, SirixDB is not really a document store, but it shreds the JSON file / string into nodes of the inherent tree-structure and thus the primary index indexes every node instead of documents / JSON objects. A consequence for instance is, that objects can be arbitrarily large as long as it's auto-committed during the import (or via a low-level Java API).

Well, to put it bluntly I'd love to see the first real usage of SirixDB (and I'm happy to help -- but maybe you'd have to help a bit with some stuff, as I'm mostly the only one working on the project more or less consistently (with some gaps) since 10 years now (and before at the University of Konstanz on the predecessor, Treetank since 2006)) with contributions from a lot of people, however.

very long answer now, but I HTH a bit Johannes