jankotek / JDBM3

Embedded Key Value Java Database
367 stars 94 forks source link

JDBM and Scala #88

Closed ghost closed 12 years ago

ghost commented 12 years ago

Hi everybody,

I have some issues working with JDBM in Scala. I am not sure yet what the problem exactly is.

All I can say is that nested Maps seems to be the issue and the Exception thrown is a NullPointerException because when it tries to deserialize an array (the elems property of the Map object), the class type is null.

I see that there is a field in serialization called classId2class and in this case this map is null.

Thank you very much in advance. I will try to see what can be done to resolve this issue on my side but as I have no clue how the serialization / deserialization works I'm guessing you guys will be more efficient than me.

Here is a small test I wrote to reproduce the issue:

package com.yoocos.test

import org.junit.Test
import org.apache.jdbm.DBMaker
import collection.JavaConversions._
import java.util

/**
 * Yoocos Sàrl
 * User: David
 * Date: 02.07.12
 * Time: 10:18
 */

class JdbmIssue {

  @Test
  def test(){
    val jdbm = DBMaker.openFile("scalaError").make()
    val map = jdbm.createHashMap[String,Any]("test")
    val otherMap = Map(
      "test" -> "me",
      "test2" -> "encore",
      "mapTest" -> Map("test" -> "MapTest"),
      "lstTest" -> List("test", "my", "list")
    ).toMap
    val data:Map[String,Any] = Map(
      "test" -> "me",
      "test2" -> "encore",
      "java mix" -> (new util.ArrayList[String]()).toBuffer.toList,
      "mapTest" -> Map("test" -> "MapTest"),
      "lstTest" -> List("test", "my", "list"),
      "self" -> otherMap
    )
    val data2 = Map("mais oui" -> data,"ispublic" -> true)
    map.putAll(data2 ++ data)
    jdbm.commit()

    jdbm.close()
    val jdbm2 = DBMaker.openFile("scalaError").deleteFilesAfterClose().make()
    val map2 = jdbm2.getHashMap("test")
    map2.get("test")
    map2.clear()
    jdbm2.commit()
  }
}
ghost commented 12 years ago

I did other tests.

it seems that the error occures when you try to add an object of type Map with more than 5 objects in it.

It does the same error when I do that:

val data:Map[String,Any] = Map(
      "test1" -> "test1",
      "test2" -> "test2",
      "test3" -> "test3",
      "test4" -> "test3",
      "test5" -> "test3"
    )

    val data2 = Map("mais oui" -> data)
    map.putAll(data2)

jdbm.commit()

    jdbm.close()
    val jdbm2 = DBMaker.openFile("scalaError").deleteFilesAfterClose().make()
    val map2 = jdbm2.getHashMap("test")
    map2.clear()
    jdbm2.commit()
ghost commented 12 years ago

I am still very interested in knowing why I get this NPE but It seems I have found a workaround.

If you use scala.collection.mutable.Map instead of the immutable version, it seems to work.

ghost commented 12 years ago

the stacktrace

java.lang.Error: Could not instanciate class at org.apache.jdbm.SerialClassInfo.readObject(SerialClassInfo.java:493) at org.apache.jdbm.Serialization.readObject(Serialization.java:40) at org.apache.jdbm.Serialization.deserialize(Serialization.java:901) at org.apache.jdbm.Serialization.deserialize(Serialization.java:614) at org.apache.jdbm.DBStore.fetch2(DBStore.java:400) at org.apache.jdbm.DBStore.fetch(DBStore.java:366) at org.apache.jdbm.DBCacheMRU.fetch(DBCacheMRU.java:165) at org.apache.jdbm.BTreeLazyRecord.get(BTreeLazyRecord.java:28) at org.apache.jdbm.HTreeBucket.removeElement(HTreeBucket.java:179) at org.apache.jdbm.HTreeDirectory.remove(HTreeDirectory.java:294) at org.apache.jdbm.HTree.remove(HTree.java:219) at org.apache.jdbm.HTreeDirectory$HDIterator.remove(HTreeDirectory.java:611) at org.apache.jdbm.HTree.clear(HTree.java:258) at com.yoocos.test.JdbmIssue.test(JdbmIssue.scala:43) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) Caused by: java.lang.NullPointerException at java.lang.reflect.Array.newArray(Native Method) at java.lang.reflect.Array.newInstance(Array.java:52) at org.apache.jdbm.Serialization.deserializeArrayObject(Serialization.java:1094) at org.apache.jdbm.Serialization.deserialize(Serialization.java:910) at org.apache.jdbm.SerialClassInfo.readObject(SerialClassInfo.java:487) ... 34 more

ghost commented 12 years ago

Ok I found something that seems to work.

The issue is that the two properties classId2class and class2classId are not populated.

So instead of using them in deserializeArrayObject I use the property registered like that:

Class clazz = Class.forName(registered.get(classId).getName());

my test is passing now.

jankotek commented 12 years ago

Closing, reopen issue in JDBM4 if it is still a problem.