Cannot remove files created by IndexedChronicle on Windows 7 (32-bit JVMs only)

[rwicke]

On my Windows 7 box I have run into an issue with Chronicle 1.6 where I cannot remove the data/index files created by an IndexedChronicle even after I have closed it. I only get this issue on 32-bit versions of Java 6 or Java 7. 64-bit versions are fine. I have also tested on Linux and do not get any problems there. I realise that it's probably more a JVM then a code issue but, oddly, everything works fine with an earlier version of Chronicle (1.3, I believe).

When I look at the JVM process in procexp on Windows I can see that the process no longer has file handles for the data/index files after calling close(). On 32-bit JVMs it does, however, have DLL handles to them. Weird.

The following code below reproduces the problem.

import java.io.File;

import com.higherfrequencytrading.chronicle.Excerpt;
import com.higherfrequencytrading.chronicle.impl.IndexedChronicle;

public class TestChronicleFileRemove {

    public static void main(String[] args) throws Exception {
        String basePath = args[0];
        boolean useUnsafe = args.length > 1 ? Boolean.parseBoolean(args[1]) : false;
        IndexedChronicle chronicle = new IndexedChronicle(args[0], 16);
        chronicle.useUnsafe(useUnsafe);
        Excerpt excerpt = chronicle.createExcerpt();
        excerpt.startExcerpt(32);
        excerpt.writeChars("Hello");
        excerpt.finish();
        excerpt.close();
        chronicle.close();

        removeFile(basePath + ".data");
        removeFile(basePath + ".index");

    }

    private static void removeFile(String path) {
        if (new File(path).delete()) {
            System.out.println("removed: " + path);
        } else {
            System.out.println("could not remove: " + path);
        }
    }

}

[peter-lawrey]

I don't have a 32-bit windows, but can you run a System.gc() before attempting to remove the files. AFAIK, it is the stop the world full collection which actually releases the memory mappings.

BTW There is a ChronicleTools.deleteOnExit(path)

[rwicke]

To clarify, I am testing on a 64-bit Windows box but with a 32-bit and 64-bit JVMs.

I added an explicit System.gc() call but that made no difference. ChronicleTools.deleteOnExit also does not remove the files.

Since we're running our Production code on Linux the behaviour on Windows is not really a big deal. It breaks some of my unit tests when I run them in my dev environment but it should be easy to work around that. Feel free to close this issue. It was more of an FYI.

[peter-lawrey]

While it doesn't appear to require urgent attention, I would prefer to keep the issue open until I have either found a fix or give up. ;)

[AMilassin]

I have the exact same problem (Win 7, 32-bit Java, both Java 6 and 7 fails). I traced back the source of the issue with a simple test to the git change of c93c02f for the ticket #18 . Before that it was OK.

[AMilassin]

Testing with "new IndexedChronicle("file", 24, ByteOrder.nativeOrder(), false);" fixes the issue. It seems the minimized buffers are preventing the files from deleting even after closing the chronicle resource.

[agaluzzi]

I am also having issues with being able to delete .data/.index files, due to file handles that remain open.

I believe one problem is that an IndexedChronicle is 'opened' in the constructor. If the constructor fails, there is no way to then close the chronicle and release the resources. Specifically, I have seen this in a scenario where the thread creating the chronicle is interrupted during construction, in which case a call to FileChannel.size() throws an exception while trying to find the last record. This is after the data/index file channels have been opened, and there seems to be no way to close them.

I would propose that either 1) the chronicle constructor should be more careful to close all resources in the case that an exception is raised, or 2) the chronicle should have a separate 'open()' method, in which case 'close()' could be called after a failed opening of the chronicle

[peter-lawrey]

That sounds like a good thing to raise as an issue so it won't be forgotten.

On 15 November 2013 12:24, agaluzzi notifications@github.com wrote:

I am also having issues with being able to delete .data/.index files, due to file handles that remain open.

I believe one problem is that an IndexedChronicle is 'opened' in the constructor. If the constructor fails, there is no way to then close the chronicle and release the resources. Specifically, I have seen this in a scenario where the thread creating the chronicle is interrupted during construction, in which case a call to FileChannel.size() throws an exception while trying to find the last record. This is after the data/index file channels have been opened, and there seems to be no way to close them.

I would propose that either 1) the chronicle constructor should be more careful to close all resources in the case that an exception is raised, or 2) the chronicle should have a separate 'open()' method, in which case 'close()' could be called after a failed opening of the chronicle

— Reply to this email directly or view it on GitHubhttps://github.com/peter-lawrey/Java-Chronicle/issues/19#issuecomment-28591484 .

[peter-lawrey]

Resource management has significantly improved in the x.20 releases and should resolve this issue.