multiple processes using the binding, use the same tmp folder -> exception

[sdemunck]

I have multiple services (java processes) using the sevenzipjbinding on the same server. Only the first launched service can load the binding sucessfully. The other ones get this error on first use:

Caused by: java.lang.RuntimeException: Error initializing SevenZipJBinding native library: can't copy native library out of a resource file to the temporary location: 'C:\Users\Bsaf\AppData\Local\Temp\SevenZipJBinding-0EsPGE1ZKaCb\lib7-Zip-JBinding.dll'
    at net.sf.sevenzipjbinding.SevenZip.copyLibraryToFS(SevenZip.java:844) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.copyOrSkipLibraries(SevenZip.java:566) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.initSevenZipFromPlatformJARIntern(SevenZip.java:455) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.initSevenZipFromPlatformJAR(SevenZip.java:340) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.ensureLibraryIsInitialized(SevenZip.java:801) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.openOutArchiveIntern(SevenZip.java:1039) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at net.sf.sevenzipjbinding.SevenZip.openOutArchive7z(SevenZip.java:979) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    at be.fed.minfin.bsaf.batch.tasklet.ArbutusJobCreationTasklet.compress(ArbutusJobCreationTasklet.java:395) ~[classes!/:na]
    ... 66 common frames omitted
Caused by: java.io.FileNotFoundException: C:\Users\Bsaf\AppData\Local\Temp\SevenZipJBinding-0EsPGE1ZKaCb\lib7-Zip-JBinding.dll (The process cannot access the file because it is being used by another process)
    at java.io.FileOutputStream.open0(Native Method) ~[na:1.8.0_191]
    at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[na:1.8.0_191]
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[na:1.8.0_191]
    at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[na:1.8.0_191]
    at net.sf.sevenzipjbinding.SevenZip.copyLibraryToFS(SevenZip.java:833) ~[sevenzipjbinding-16.02-2.01.jar:16.02-2.01]
    ... 73 common frames omitted

It seems that the binding is using the build.ref in the properties file to determine the tmp-folder name, which is the same for all three services and crashes because the file is locked.

Currently I worked around this issue by setting java.io.tmpdir for each service.

[borisbrodski]

Sorry, this is a known bug. With the current release you have to synchronize the initialization of the 7-Zip-JBinding across multiple processes by yourself.

Actually, you have to make sure, that the first initialization call ever for the specified user is not disturbed by concurrent initialization calls (as to my current understanding of the problem).

[sdemunck]

Ok, thanks for the update I'll keep that in mind.

[xishian]

Wrong sha1 ?

net.sf.sevenzipjbinding.SevenZip

private static String byteArrayToHex(byte[] byteArray) { StringBuilder stringBuilder = new StringBuilder(); for (int i = 0; i < byteArray.length; i++) { stringBuilder.append(Integer.toHexString(0xFF & byteArray[i])); } return stringBuilder.toString(); }

[lfcnassif]

Thanks very much for the library. I also hit this. Any time frame for fixing this?

[borisbrodski]

It's the next challenge for me :). Should be in the next release (may be in Q1 2021)

[lfcnassif]

Thanks for answering. I was able to workaround like @sdemunck but my services are often restarted and that is filling up my temp folder with a lot of lib7-Zip-JBinding.dll (in different folders).

I also tried to initialize before starting the services, so the temp dll would be created before and reused by them, but the same error is thrown. Can't the same dll be shared between different processes? I thought the issue was different processes trying to create the same dll at the same time, but if the dll is already there, what is wrong with this approach? Any tips?

Thanks in advance

[borisbrodski]

Sharing DLLs should be possible. Maybe you are running into some Windows related locking issues?

[lfcnassif]

maybe... but sharing the same dll used to work with previous sevenzipjbinding versions, not sure why extracting the dll before starting the services is not working...

[borisbrodski]

Do you have a stacktrace for me?

[lfcnassif]

The relevant part is below, executed with latest release:

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Error initializing SevenZipJBinding native library: can't copy native library out of a resource file to the temporary location: 'N:\temp\indexador-temp1603484415838\SevenZipJBinding-0EsPGE1ZKaCb\lib7-Zip-JBinding.dll'
    at dpf.sp.gpinf.indexer.parsers.SevenZipParser.getTypes(SevenZipParser.java:91)
    at dpf.sp.gpinf.indexer.parsers.SevenZipParser.<clinit>(SevenZipParser.java:57)
    ... 11 more
Caused by: java.lang.RuntimeException: Error initializing SevenZipJBinding native library: can't copy native library out of a resource file to the temporary location: 'N:\temp\indexador-temp1603484415838\SevenZipJBinding-0EsPGE1ZKaCb\lib7-Zip-JBinding.dll'
    at net.sf.sevenzipjbinding.SevenZip.copyLibraryToFS(SevenZip.java:844)
    at net.sf.sevenzipjbinding.SevenZip.copyOrSkipLibraries(SevenZip.java:566)
    at net.sf.sevenzipjbinding.SevenZip.initSevenZipFromPlatformJARIntern(SevenZip.java:455)
    at net.sf.sevenzipjbinding.SevenZip.initSevenZipFromPlatformJAR(SevenZip.java:340)
    at dpf.sp.gpinf.indexer.parsers.SevenZipParser.getTypes(SevenZipParser.java:73)
    ... 12 more
Caused by: java.io.FileNotFoundException: N:\temp\indexador-temp1603484415838\SevenZipJBinding-0EsPGE1ZKaCb\lib7-Zip-JBinding.dll (The process cannot access the file because it is being used by another process)
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(Unknown Source)
    at java.io.FileOutputStream.<init>(Unknown Source)
    at java.io.FileOutputStream.<init>(Unknown Source)
    at net.sf.sevenzipjbinding.SevenZip.copyLibraryToFS(SevenZip.java:833)
    ... 16 more

I think the check in this line is not working: https://github.com/borisbrodski/sevenzipjbinding/blob/master/jbinding-java/src/net/sf/sevenzipjbinding/SevenZip.java#L559

Maybe because this line https://github.com/borisbrodski/sevenzipjbinding/blob/master/jbinding-java/src/net/sf/sevenzipjbinding/SevenZip.java#L604 should check if length < 0? It is possible the InputStream.read(byte[]) return 0 bytes without reaching the end of the stream, but not sure if this is the case.

[lfcnassif]

I think I found the issue in line https://github.com/borisbrodski/sevenzipjbinding/blob/master/jbinding-java/src/net/sf/sevenzipjbinding/SevenZip.java#L632

It is removing a leading zero in each byte of the hash. So the windows x64 dll sha1 c10cc570b0d7edd03697eb6ac4cbe3d101d34e2a is converted to string c1cc570b0d7edd03697eb6ac4cbe3d11d34e2a

Changing that line to stringBuilder.append(String.format("%1$02x", 0xFF & byteArray[i])); should fix.

[lfcnassif]

just saw @xishian found the issue 2 days ago :)