unfriendly error message in strusInspect when getting unkown metadata

[andreasbaumann]

strusInspect -s 'path=storage/xxx' metadata THIS_META_FIELD_DOES_NOT_EXIST

results in:

EXCEPTION array bound read in function get

[patrickfrey]

Need more info info to reproduce. I get:

$ strusCreate -s path=storage storage successfully created. $ strusInspect -s 'path=storage' metadata THIS_META_FIELD_DOES_NOT_EXIST ERROR unhandled error in inspect storage: database transaction with error: error meta data get element handle: meta data element with name 'THIS_META_FIELD_DOES_NOT_EXIST' is not defined

[andreasbaumann]

gdb --args strusInspect -s 'path=storage/xxx' metadata doclen2
GNU gdb (GDB) 7.12
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from strusInspect...done.
(gdb) catch throw
rCatchpoint 1 (throw)
(gdb) run
Starting program: /usr/bin/strusInspect -s path=storage/rrb metadata doclen2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Catchpoint 1 (exception thrown), __cxxabiv1::__cxa_throw (
    obj=obj@entry=0x6371b0, 
    tinfo=0x7ffff66b3ab0 <typeinfo for std::runtime_error>, 
    dest=0x7ffff63dde30 <std::runtime_error::~runtime_error()>)
    at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:62
62      /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory.
(gdb) bt
#0  __cxxabiv1::__cxa_throw (obj=obj@entry=0x6371b0, 
    tinfo=0x7ffff66b3ab0 <typeinfo for std::runtime_error>, 
    dest=0x7ffff63dde30 <std::runtime_error::~runtime_error()>)
    at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:62
#1  0x00007ffff562f555 in strus::MetaDataDescription::getHandle (
    this=<optimized out>, name_="doclen2")
    at /data/work/strus/strus/src/storage/metaDataDescription.cpp:122
#2  0x00007ffff56365f0 in strus::MetaDataReader::elementHandle (
    this=this@entry=0x6366f0, name="doclen2")
    at /data/work/strus/strus/src/storage/metaDataReader.cpp:26
#3  0x0000000000405933 in inspectDocMetaData (storage=..., 
    key=key@entry=0x7fffffffea08, size=size@entry=1)
    at /data/work/strus/strusUtilities/src/strusInspect/strusInspect.cpp:370
#4  0x0000000000408856 in main (argc=<optimized out>, argv=<optimized out>)
    at /data/work/strus/strusUtilities/src/strusInspect/strusInspect.cpp:913
(gdb) 

If I retry I get:

 !strus
strusInspect -s 'path=storage/xxx' metadata doclen2
ERROR failed to create storage client: error creating database client: failed to open key value store database: IO error: storage/rrb/LOCK: No such file or directory

The only difference I can see is that the index has been created with strusWebService, maybe this is a prolem of the index and some non-closed transactions.

Closing for now, Seems to be a follow-up bug.

[andreasbaumann]

There seems to be an index corruption issue when used in the webservice:

https://github.com/Eurospider/strusWebService/issues/63

[andreasbaumann]

Suspecting that deletions cause the metadata iterator to stumble.. This is another difference to the manual test, the webservice has insertions, updates and deletions..

[andreasbaumann]

Inserted 24 documents via strusWebService, strusInspect on attribute docid works. strusInspect on metadata doclen works. strusInspect on indexterms works. Deleting one document.

strusInspect crashes in various ways:

for instance indexterms:

#0  0x00007fd356683769 in unpackInt32_ (
    end=0x7fd34fce59b8 <error: Cannot access memory at address 0x7fd34fce59b8>, itr=@0x7ffdfd6f89e8: 0x0)
    at /data/work/strus/strus/src/storage/indexPacker.cpp:24
24              int charlen = utf8charlen( *itr);
(gdb) bt
#0  0x00007fd356683769 in unpackInt32_ (
    end=0x7fd34fce59b8 <error: Cannot access memory at address 0x7fd34fce59b8>, itr=@0x7ffdfd6f89e8: 0x0)
    at /data/work/strus/strus/src/storage/indexPacker.cpp:24
#1  strus::unpackIndex (itr=@0x7ffdfd6f89e8: 0x0, 
    end=0x7fd34fce59b8 <error: Cannot access memory at address 0x7fd34fce59b8>)
    at /data/work/strus/strus/src/storage/indexPacker.cpp:81
#2  0x00007fd35668b1f4 in strus::InvTermBlock::element_at (this=this@entry=0x118df10, itr=<optimized out>)
    at /data/work/strus/strus/src/storage/invTermBlock.cpp:18
#3  0x00007fd356682e3f in strus::DocumentTermIterator::nextTerm (this=0x118de80, value=...)
    at /data/work/strus/strus/src/storage/documentTermIterator.cpp:71
#4  0x000000000040ad9e in inspectDocumentIndexTerms (storage=..., key=key@entry=0x7ffdfd6f9588, 
    size=size@entry=1) at /data/work/strus/strusUtilities/src/strusInspect/strusInspect.cpp:174
#5  0x000000000040879b in main (argc=<optimized out>, argv=<optimized out>)
    at /data/work/strus/strusUtilities/src/strusInspect/strusInspect.cpp:901

But dumping the docid attribute works, metadata doclen shows 0 in positions where the document has been deleted.

[andreasbaumann]

Still not sure whether this is a bug in the webservice, a asynchronization problem or both.

[patrickfrey]

I fixed a segfault due to an array bound read in the document term iterator (documentTermIterator.cpp:71) that is most likely the root cause of this bug.