search

orientechnologies / orientdb

OrientDB is the most versatile DBMS supporting Graph, Document, Reactive, Full-Text and Geospatial models in one Multi-Model product. OrientDB can run distributed (Multi-Master), supports SQL, ACID Transactions, Full-Text indexing and Reactive Queries.
https://orientdb.dev
Apache License 2.0
4.75k stars 871 forks source link

Lucene Spatial Index is closed without any reason #7585

Closed EricSchreiner closed 7 years ago

EricSchreiner commented 7 years ago

OrientDB Version: 2.2.24 (Embedded API)

Java Version: 1.8

OS: Windows 10

Expected behavior

Lucene Spatial Index is closed without any reason. ( org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed) This does not happen in 2.2.22 or 2.2.23

Actual behavior

ERROR@ 15:18:08.644 Exception Error PhotoCrawler.DirectoryConsumer.handlePhotoUpdate:D:\Arbeit\Photos\1988\Portugal\Portugal-Spanien-DEV-PIC_0104.jpg: com.orientechnologies.orient.core.exception.ODatabaseException: Error during saving of record with rid #25:1755 DB name="db.2.2.22" EXCEP@ ============================================================ EXCEP@ Exception at: 2017-07-25 15:18:08 EXCEP@ Msg: EXCEP@ Error during saving of record with rid #25:1755 DB name="db.2.2.22" EXCEP@ ------------------------------------------------------------ EXCEP@ com.orientechnologies.orient.core.exception.ODatabaseException: Error during saving of record with rid #25:1755 EXCEP@ DB name="db.2.2.22" EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:203) EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2758) EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2638) EXCEP@ at de.contecon.picapport.db.Photo.saveToDocument(Photo.java:226) EXCEP@ at de.contecon.picapport.db.Photo.insert(Photo.java:169) EXCEP@ at de.contecon.picapport.db.DbWrapper.insertPhoto(DbWrapper.java:417) EXCEP@ at de.contecon.picapport.db.DbWrapper.updatePhoto(DbWrapper.java:352) EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.handlePhotoUpdate(PhotoCrawler.java:403) EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.consume(PhotoCrawler.java:278) EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.run(PhotoCrawler.java:190) EXCEP@ at java.lang.Thread.run(Unknown Source) EXCEP@ Caused by: com.orientechnologies.orient.core.exception.OStorageException: Cannot put key value entry in index EXCEP@ DB name="db.2.2.22" EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateIndexEntry(OAbstractPaginatedStorage.java:2256) EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.updateIndexEntry(OAbstractPaginatedStorage.java:2153) EXCEP@ at com.orientechnologies.lucene.index.OLuceneIndexNotUnique.put(OLuceneIndexNotUnique.java:317) EXCEP@ at com.orientechnologies.spatial.index.OLuceneSpatialIndex.put(OLuceneSpatialIndex.java:50) EXCEP@ at com.orientechnologies.spatial.index.OLuceneSpatialIndex.put(OLuceneSpatialIndex.java:35) EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.putInIndex(OClassIndexManager.java:659) EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.addIndexEntry(OClassIndexManager.java:554) EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.addIndexesEntries(OClassIndexManager.java:541) EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.onRecordAfterCreate(OClassIndexManager.java:419) EXCEP@ at com.orientechnologies.orient.core.hook.ODocumentHookAbstract.onTrigger(ODocumentHookAbstract.java:221) EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.callbackHooks(ODatabaseDocumentTx.java:1144) EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.callbackHookSuccess(ODatabaseDocumentTx.java:3273) EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.executeSaveRecord(ODatabaseDocumentTx.java:2237) EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveNew(OTransactionNoTx.java:241) EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:171) EXCEP@ ... 10 more EXCEP@ Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed EXCEP@ at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:719) EXCEP@ at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:733) EXCEP@ at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1471) EXCEP@ at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254) EXCEP@ at org.apache.lucene.index.TrackingIndexWriter.addDocument(TrackingIndexWriter.java:112) EXCEP@ at com.orientechnologies.lucene.engine.OLuceneIndexEngineAbstract.addDocument(OLuceneIndexEngineAbstract.java:155) EXCEP@ at com.orientechnologies.spatial.engine.OLuceneLegacySpatialIndexEngine.put(OLuceneLegacySpatialIndexEngine.java:167) EXCEP@ at com.orientechnologies.spatial.engine.OLuceneSpatialIndexEngineDelegate.put(OLuceneSpatialIndexEngineDelegate.java:144) EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateIndexEntry(OAbstractPaginatedStorage.java:2242) EXCEP@ ... 24 more

Steps to reproduce

robfrank commented 7 years ago

I'm fixing. BTW, the legacy spatial index is deprecated and will be removed in the future. My suggestion is to migrate to Spatial Module: http://orientdb.com/docs/last/Spatial-Index.html

EricSchreiner commented 7 years ago

??? We are using the Spatial Module (http://central.maven.org/maven2/com/orientechnologies/orientdb-spatial/2.2.23/orientdb-spatial-2.2.23-dist.jar)

robfrank commented 7 years ago

ok, but you are using the legacy part: http://orientdb.com/docs/last/Spatial-Index.html#legacy

not the one with functions

robfrank commented 7 years ago

just pushed the fix: https://github.com/orientechnologies/orientdb-spatial/commit/0fec0b9d7c4cfd1fa805cc2a3227137013280ed4 Can you test with the latest snapshot? https://oss.sonatype.org/content/repositories/snapshots/com/orientechnologies/orientdb-spatial/2.2.25-SNAPSHOT/orientdb-spatial-2.2.25-20170725.155011-12-dist.jar

just put this jar instead of the 2.2.24

EricSchreiner commented 7 years ago

..this is what we do:DbSchema.createSchema: Photo.longitude (type=DOUBLE) may be null (CREATE INDEX idxlocation ON Photo(latitude,longitude) SPATIAL ENGINE LUCENE METADATA {ignoreNullValues: true}) Will this be removed? What should we do instead?

lvca commented 7 years ago

@EricSchreiner did you try latest fix from @robfrank ?

robfrank commented 7 years ago

Starting from 2.2, the new spatial module allow indexing of points and other spatial structures, such as lines and polygons:

http://orientdb.com/docs/last/Spatial-Index.html#geometry-data

It needs that spatial data are stored in specilized classes:

http://orientdb.com/docs/last/Spatial-Index.html#geometry-data-example

and offers a lot of functions (ST_: Spatial type from The Open Geospatial Consortium) to handle spatial queries:

http://orientdb.com/docs/last/Spatial-Index.html#functions

The legacy api can index only points and offers only near and within operators.

So, don't worry, the legacy API will be there on 2.2.x. But if you can migrate to the new api, you will gain a lot of benefits.

EricSchreiner commented 7 years ago

Hi @robfrank, @lvca your Fix seems to work :-) when can we expect an official 2.2.25?

lvca commented 7 years ago

Cool, closing it.

EricSchreiner commented 7 years ago

@lvca any idea when the next hotfix will come?

robfrank commented 7 years ago

we have just released 2.2.24, I think at least 2 weeks.

EricSchreiner commented 7 years ago

... 2 weeks for a serious Bug seems to be a long time.....

lvca commented 7 years ago

Is this issue fixed in 2.2.x? Confirmed?

luigidellaquila commented 7 years ago

@lvca yes, it's fixed, maybe we can release sooner...?

Luigi

andrii0lomakin commented 7 years ago

Reproduced on latest snapshot

robfrank commented 7 years ago

@laa he uses a 2.2.23 with lucene jar from 2.2.26-SNAPSHOT. But this was fixed on 2.2.25.

andrii0lomakin commented 7 years ago

@robfrank sorry issue still persist in 2.2.26

andrii0lomakin commented 7 years ago

You see this from the latest log published in #7586

robfrank commented 7 years ago

Ok, there was still a sync problem on autoclose of indexes. Testing

robfrank commented 7 years ago

this problem should be fixed in 2.2.26. Can confirm?

EricSchreiner commented 7 years ago

@robfrank confirmed