H2 db files sometimes don't get removed after executing SHUTDOWN command.

GoogleCodeExporter commented 9 years ago

1. Create H2 db with and create a table in it.
2. Create lucene index for that table.
3. Store something like a thousand rows in the table.
4. Execute SHUTDOWN. Db files will not always be removed.

version: 1.3.156
urgent: yes
workaround: delete db folder manually after some delay has elapsed since 
SHUTDOWN command execution.

Original issue reported on code.google.com by renat.ak...@gmail.com on 30 Jun 2011 at 8:15

GoogleCodeExporter commented 9 years ago

'Shutdown' isn't supposed to delete database files...

Original comment by thomas.t...@gmail.com on 30 Jun 2011 at 8:23

GoogleCodeExporter commented 9 years ago

Ok. How to do this correctly?

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 8:25

GoogleCodeExporter commented 9 years ago

Run the SQL statement 'drop all objects delete files', and then close all 
connections.

Original comment by thomas.t...@gmail.com on 30 Jun 2011 at 8:26

GoogleCodeExporter commented 9 years ago

I'm sorry, I think I wasn't enough precise filing this issue. The problem is 
that we can't actually remove db files manually after SHUTDOWN command. 
Sometimes it is allowed, sometimes it's not (I believe because some h2 threads 
keep working with them).

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 8:27

GoogleCodeExporter commented 9 years ago

Is this an issue at all and if it's not then would you please give some tips 
how to deal with it?

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 8:29

GoogleCodeExporter commented 9 years ago

I tried it but it didn't help either.

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 8:30

GoogleCodeExporter commented 9 years ago

Hm, I didn't know the database file isn't closed... It should be.
Could you provide a reproducible test case please?

Original comment by thomas.t...@gmail.com on 30 Jun 2011 at 8:35

GoogleCodeExporter commented 9 years ago

Yes, I'll provide it.

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 9:22

GoogleCodeExporter commented 9 years ago

Attached is the JUnit test that reproduce the problem.

Some comments:
1) Using 'drop all objects delete files' and closing all connections obviously 
doesn't remove db files, although, if I understand API correctly, it should.
2) However, if I use additionally "shutdown" statement then db files are 
removed consistently. Unfortunately, in more complex environment even using 
these two statements doesn't remove db files so I will try to make the test 
more complex to reproduce it.

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 11:38

Attachments:

H2Test.java

GoogleCodeExporter commented 9 years ago

I created another version of this test where I added Lucene indexing and 
multithreaded execution of writes, reads and lucene searches. Now I'm 
consistently getting another error I didn't even meet before:

java.sql.SQLException: Error while indexing document
    at org.h2.fulltext.FullTextLucene.convertException(FullTextLucene.java:216)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:455)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:182)
    at H2MultithreadedTest$3.run(H2MultithreadedTest.java:273)
    at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is 
closed
    at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:263)
    at org.apache.lucene.index.DirectoryReader.terms(DirectoryReader.java:686)
    at org.apache.lucene.search.PrefixTermEnum.<init>(PrefixTermEnum.java:41)
    at org.apache.lucene.search.PrefixQuery.getEnum(PrefixQuery.java:45)
    at org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:34)
    at org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:87)
    at org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:215)
    at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:291)
    at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:592)
    at org.apache.lucene.search.Query.weight(Query.java:100)
    at org.apache.lucene.search.IndexSearcher.createW>>> INFO: Reads: 0
eight(IndexSearcher.java:662)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:301)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:289)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:417)
    ... 3 more
Exception in thread "Thread-5" java.lang.RuntimeException: 
java.sql.SQLException: Error while indexing document
    at H2MultithreadedTest$3.run(H2MultithreadedTest.java:284)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.sql.SQLException: Error while indexing document
    at org.h2.fulltext.FullTextLucene.convertException(FullTextLucene.java:216)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:455)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:182)
    at H2MultithreadedTest$3.run(H2MultithreadedTest.java:273)
    ... 1 more
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is 
closed
    at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:263)
    at org.apache.lucene.index.DirectoryReader.terms(DirectoryReader.java:686)
    at org.apache.lucene.search.PrefixTermEnum.<init>(PrefixTermEnum.java:41)
    at org.apache.lucene.search.PrefixQuery.getEnum(PrefixQuery.java:45)
    at org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:34)
    at org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:87)
    at org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:215)
    at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:291)
    at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:592)
    at org.apache.lucene.search.Query.weight(Query.java:100)
    at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:662)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:301)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:289)
    at org.h2.fulltext.FullTextLucene.search(FullTextLucene.java:417)

Please take a look at this. It would be great if you told what's wrong in my 
code.

Thanks.

Original comment by renat.ak...@gmail.com on 30 Jun 2011 at 12:53

Attachments:

H2MultithreadedTest.java

GoogleCodeExporter commented 9 years ago

You have used 'assert stmt.executeUpdate() > 0;' - this is really dangerous; 
assertions should never have such side effects. Use Assert.assertTrue(..) 
instead.

You are using multiple thread; in one thread you are adding to the Lucene index 
and in another you are reading - but not by calling the database search method, 
but directly invoking the method. This will not work as the database (in one 
thread) closes and re-opens the reader, while your application (in another 
thread) tries to use the reader.

I will document this is not supported. I understand the use case, but if you 
need this feature I suggest to implement synchronization yourself and submit a 
patch (with a better test case) - I am not interested in this feature 
currently, so I will not create a patch.

Anyway, the summary of the issue is incorrect as far as I understand. I will 
close the issue. I'm sorry but I don't have the time required to solve your 
problem; I suggest you try to understand the problem yourself, and if required, 
submit a patch or a new test case (if it is in fact a bug in the database).

Original comment by thomas.t...@gmail.com on 2 Jul 2011 at 4:29

Changed state: Invalid

GoogleCodeExporter commented 9 years ago

I will add the following the documentation:

The Lucene fulltext search implementation is not synchronized internally. If 
you update the database and query the fulltext search concurrently (directly 
using the Java API of H2 or Lucene itself), you need to ensure operations are 
properly synchronized. If this is not the case, you may get exceptions such as 
org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed.

Original comment by thomas.t...@gmail.com on 2 Jul 2011 at 4:35

GoogleCodeExporter commented 9 years ago

It's ok about Lucene, I'll keep in mind what you said. But what about "drop all 
objects delete files"(first version of my test case)? Should I file a separate 
ticket?

Original comment by renat.ak...@gmail.com on 2 Jul 2011 at 7:59

GoogleCodeExporter commented 9 years ago

Well, H2Test.java works for me. Does it not work for you? If not, could you 
append ;TRACE_LEVEL_SYSTEM_OUT=3 to the database URL and try again, and then 
post the system out here? What version of H2 do you use?

Original comment by thomas.t...@gmail.com on 2 Jul 2011 at 8:13

google-code-export / h2database

H2 db files sometimes don't get removed after executing SHUTDOWN command. #327