NullPointer on map creation on FreeBSD

[ghost]

Chronicle-Map 3.13.0 / Java8u121

Steps to reproduce:

1. Create a simple ChronicleMap (source with gradle: MapTest-src.zip, important part below)
2. Run it on FreeBSD 10.3 on a ZFS/UFS File system

Reproduce with VirtualBox:

1. Download the Virtualbox image (Will be deleted automatically in 14 days)
2. Log in with root / no password
3. run ./MapTest/bin/MapTest

Expected:

Map is created (program prints "Map created")

Actual:

Exception in thread "main" java.lang.NullPointerException at com.sun.jna.Native.toNative(Native.java:1736) at net.openhft.chronicle.hash.impl.util.jna.PosixMsync.msync(Native Method) at net.openhft.chronicle.hash.impl.util.jna.PosixMsync.msync(PosixMsync.java:32) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.msync(VanillaChronicleHash.java:878) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.msync(VanillaChronicleHash.java:864) at net.openhft.chronicle.map.ChronicleMapBuilder.commitChronicleMapReady(ChronicleMapBuilder.java:416) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithNewFile(ChronicleMapBuilder.java:1721) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithFile(ChronicleMapBuilder.java:1578) at net.openhft.chronicle.map.ChronicleMapBuilder.createPersistedTo(ChronicleMapBuilder.java:1484) at maptest.MapTest.main(MapTest.java:18)

Map source code:

 ChronicleMap<Integer, Integer> storage = ChronicleMapBuilder.of(Integer.class, Integer.class).
                name("test").
                constantKeySizeBySample(1).
                constantValueSizeBySample(1).
                entries(10000).
                createPersistedTo(new File("test.dat"));

System.out.println("Map created");

[ghost]

Update: This fails on UFS as well, so I am assuming its not filesystem-dependent

[leventov]

@Quurks thanks for this report. Seems like problem with a problem JNA, JNA is not able to capture msync system call on FreeBSD. I'm able to reproduce the problem, but I'm not able to test with different test build, because I'm not familiar with FreeBSD and seems like it doesn't support anything (I even cannot unzip or untar an archive). Could you rebuild Chronicle Map with JNA 4.4.0 and try to run this test?

[ghost]

@leventov I build ChronicleMap 3.13 with JNA 4.4.0. The Map creation succeeds now, but the JVM crashes (SIGSEGV) on put.

Since I seem to be the only FreeBSD user and the fix / debug process is probably nontrivial I am going to look for another embedded Map.

On the off chance you are going to fix this or it becomes relevant later:

• JNA 4.3+ does not contain the freebsd binaries, devel/jna needs to be installed via ports
• Maven needs to be downloaded manually, pkg install maven3 is way outdated
• (This was before installing the port, didnt try it after: ) mvn test crashes the JVM
• The gradle wrapper does not work, pkg install gradle is required (thus you need to use gradleinstead of ./gradlew)
• You can install packages, i.e. unzip via pkg install unzip

[leventov]

@Quurks SIGSEGV on put may be related to the feature of Linux and Windows on which Chronicle Map relies, is that file length is adjusted automatically when a region of a file (beyond the prev. file length) is mapped into memory, and then the first access to that memory occurs. FreeBSD may not support that. You may try something like

File file = new File("test.dat");
ChronicleMap<Integer, Integer> storage = ChronicleMapBuilder.of(Integer.class, Integer.class).
                name("test").
                constantKeySizeBySample(1).
                constantValueSizeBySample(1).
                entries(10000).
                createPersistedTo(file);
try (RandomAccessFile raf = new RandomAccessFile(file, "rw") {
  raf.setLength(storage.offHeapMemoryUsed());
}

If it works we would add an opt-in feature to support this inside the Chronicle Map library.

[leventov]

However I just checked and seems that we already set the correct file length before memory mapping, so the above message may be completely misleading

[ghost]

Yes, does not seem to make a difference: hs_err_new.txt hs_err_old.txt

[leventov]

Could you please try to add the following code in VanillaChronicleHash.zeroOutNewlyMappedTier():

        for (long i = tierOffset; i < tierOffset + tierSize - tierEntrySpaceOuterSize; i++) {
            bytesStore.writeByte(i, 0);
        }

[ghost]

hs_err_3.txt

I will be back in 10 hours.

[leventov]

Looks like a bug in Chronicle Map. Thanks for report and help with testing

[RobAustin]

Do you still get this issue with the latest version?

[minborg]

I am closing this issue now @Quurks . Let me know if you want the issue reopened (if the problem remains in a more recent version of Map)