search

jankotek / JDBM3

Embedded Key Value Java Database
367 stars 94 forks source link

java.lang.OutOfMemoryError: Map failed #92

Closed reuschling closed 12 years ago

reuschling commented 12 years ago

I'm new to JDBM, and I tried to insert music track names as Strings into a TreeSet. Sadly I recieve an Exception at the ~9.680.000th add(..) invocation. I use 32bit jdk1.7.0_06 on an OpenSuse Linux with -Xmx2G. The JDBM version is the org.apache.jdbm 3.0-alpha4 from the central maven repository. Is this a bug?

The code:

jdbmDB = DBMaker.openFile("simRecMaps").disableTransactions().deleteFilesAfterClose().enableHardCache().disableLocking().make(); Set hsTrackNames = jdbmDB.createTreeSet("tracks"); hsTrackNames.add(strArtistName);

Exception in thread "main" java.io.IOError: java.io.IOException: Map failed at org.apache.jdbm.DBStore.commit(DBStore.java:450) at org.apache.jdbm.DBCache.commit(DBCache.java:101) at org.apache.jdbm.DBCacheRef.commit(DBCacheRef.java:36) at org.apache.jdbm.DBCacheRef.update(DBCacheRef.java:181) at org.apache.jdbm.BTreeNode.insert(BTreeNode.java:384) at org.apache.jdbm.BTreeNode.insert(BTreeNode.java:392) at org.apache.jdbm.BTreeNode.insert(BTreeNode.java:392) at org.apache.jdbm.BTreeNode.insert(BTreeNode.java:392) at org.apache.jdbm.BTreeNode.insert(BTreeNode.java:392) at org.apache.jdbm.BTree.insert(BTree.java:281) at org.apache.jdbm.BTreeMap.put(BTreeMap.java:285) at org.apache.jdbm.BTreeSet.add(BTreeSet.java:40) ... Caused by: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:849) at org.apache.jdbm.StorageDiskMapped.write(StorageDiskMapped.java:113) at org.apache.jdbm.PageFile.commit(PageFile.java:260) at org.apache.jdbm.PageManager.commit(PageManager.java:204) at org.apache.jdbm.DBStore.commit(DBStore.java:446) ... 14 more Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:846) ... 18 more

jankotek commented 12 years ago

Looks like generic OutOfMemory exception. Java uses memory mapped files, so OOEM can happen even with large heap. Also JVM 32bit so addressable memory space is small.

Need more information, please provide test case.

reuschling commented 12 years ago

Fortunately, during creating the test case, I was not able to reproduce the problem again. I saved all the music track names from the database inside a file, and with this test case, jdbm works like a charme :) Seems that the mysql-based PreparedStatement/Resultset becomes very big, and that the small overhead from jdbm was enough to force the OutOfMemory Exception. I was not aware of that database Resultsets becomes big in memory - my fault, sorry for the inconvenience. From my side, this issue can be closed - thank you very much for this very handy tool