search

njh / redstore

RedStore is a lightweight RDF triplestore written in C using the Redland library.
https://www.aelius.com/njh/redstore/
GNU General Public License v3.0
62 stars 6 forks source link

Redstore hashes + BDB: persistent? #26

Closed ctenolophon closed 13 years ago

ctenolophon commented 13 years ago

I'm using the hashes and BDB module, and redstore runs fine via http. However, the file size of the .db files never increases, and when I kill the process, and restart using the same db (without -n), there are no data stored. It seems nothing is being written to the files, even though the data is stored fine in memory. Perhaps I have the wrong options? I'm using (e.g.):

  $ redstore -p 8000 -s hashes -t "hash-type='bdb'" -n mydb

which gives output:

  [INFO]   Tue Jun  7 15:25:01 2011  Storage name: mydb
  [INFO]   Tue Jun  7 15:25:01 2011  Storage type: hashes
  [INFO]   Tue Jun  7 15:25:01 2011  Storage options: write='yes', hash-type='bdb', new='yes', contexts='yes'
  [INFO]   Tue Jun  7 15:25:02 2011  Starting HTTP server on port 8000

Files mydb-contexts.db, mydb-po2s.db, mydb-so2p.db and mydb-sp2o.db are created, with contents:

  VERSION=3
  format=bytevalue
  type=btree
  duplicates=1
  db_pagesize=4096
  HEADER=END
  DATA=END

I'd like the data to live on after I kill the process. Any help much appreciated.

(Versions: raptor2-2.0.0, rasqal-0.9.24, redland-1.0.13, redstore-0.5.2)

njh commented 13 years ago

Fingers crossed, this was fixed in: https://github.com/njh/redstore/commit/f06283cdba0be29ea7853c76a93b3bb77253cc05

Sorry about that! I should add some tests for various storages.

njh commented 13 years ago

Although the commit above should improve things, I am not sure if it actually fixes anything - partly because I have been unable to re-produce the problem.

What OS are you using? Which version of BDB are you using?

njh commented 13 years ago

I have written a script to test the various storage modules, including BDB, which was committed here: 778cff22044fb8118ae05a305bec5f56094c8b09

Could you see if the tests pass for you?

make check

They are passing for me on Mac OS X and Linux (BDB version 4.8): http://ci.aelius.com/job/redstore/658/console

ctenolophon commented 13 years ago

Hi Nicholas,

# Starting RedStore on port 12870.
# /home/cam/Desktop/newred/bin/redstore -q -b localhost -p 12870 -s hashes -n -t hash-type='bdb',dir='.' redstore-test
librdf error - storage mysql already registered
ok 23 - Process 28318 running
ok 24 - hashes - getting homepage is successful
ok 25 - Loading foaf.nt into http://localhost:12870/data/foaf.nt is successful.
# Starting RedStore on port 11698.
# /home/cam/Desktop/newred/bin/redstore -q -b localhost -p 11698 -s hashes -t hash-type='bdb',dir='.' redstore-test
librdf error - storage mysql already registered
ok 26 - hashes - SPARQL SELECT query is successful
ok 27 - hashes - SPARQL SELECT query Content Type data is text/csv
ok 28 - hashes - SPARQL response contains fifteen lines
not ok 29 - hashes - First line of SPARQL response contains CSV header
#   Failed test 'hashes - First line of SPARQL response contains CSV header'
#   at redstore-storage_test.pl line 47.
#          got: 'Result,s,p,o'
#     expected: 's,p,o'
ok 30 - Deleting storage file redstore-test-contexts.db
ok 31 - Deleting storage file redstore-test-po2s.db
ok 32 - Deleting storage file redstore-test-so2p.db
ok 33 - Deleting storage file redstore-test-sp2o.db

Thanks for working on this.

Cam

njh commented 13 years ago

Great, glad it is working for you :)

rasqal-0.9.27 changed to using the new SPARQL 1.1 standard format, so that is why that test is failing.