Crash when recursively extracting the same DLL and not using standard UrlClassLoader

[shartte]

This issue occurs because Loader tries to replace a DLL file that has already been extracted and loaded by another DLL, but only if the codepath for non-JarURLConnection based resources is taken.

As far as I can tell, if the URL is a JarURLConnection, Loader will not try to delete&replace the File, if it already exists, has the same size and modification date. But if the URL is not a JarURLConnection, it will always try to first delete, and then replace the file. See here: https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/Loader.java#L849

This will not work, if different DLLs are loaded that use the same CRT. (i.e. avutil here loads, which will in turn try to load the javacpp basics).

Stacktrace:

java.lang.UnsatisfiedLinkError: java.io.FileNotFoundException: C:\Users\Sebastian\.javacpp\cache\windows-x86_64\jniavutil.dll (The process cannot access the file because it is being used by another process)
    at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1853) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1423) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1234) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1210) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.ffmpeg.avutil.AVChannelLayout.<clinit>(AVChannelLayout.java:46) ~[ffmpeg-6.0-1.5.9.jar:6.0-1.5.9] {}
    at org.bytedeco.ffmpeg.global.avutil.AV_CHANNEL_LAYOUT_MONO(Native Method) ~[ffmpeg-6.0-1.5.9.jar:6.0-1.5.9] {}
    at org.bytedeco.ffmpeg.global.avutil.<clinit>(avutil.java:4381) ~[ffmpeg-6.0-1.5.9.jar:6.0-1.5.9] {}
    at java.lang.Class.forName0(Native Method) ~[?:?] {}
    at java.lang.Class.forName(Class.java:467) ~[?:?] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1289) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1234) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.load(Loader.java:1226) ~[javacpp-1.5.9.jar:1.5.9] {}
    at appeng.siteexport.WebPExporter.<init>(WebPExporter.java:48) ~[classes/:?] {re:classloading}
    at appeng.siteexport.OffScreenRenderer.captureAsWebp(OffScreenRenderer.java:109) ~[classes/:?] {re:classloading}
    at appeng.siteexport.SiteExporter.renderAndWrite(SiteExporter.java:563) ~[classes/:?] {re:classloading,pl:runtimedistcleaner:A}
    at appeng.siteexport.SiteExporter.processItems(SiteExporter.java:477) ~[classes/:?] {re:classloading,pl:runtimedistcleaner:A}
    at appeng.siteexport.SiteExporter.export(SiteExporter.java:339) ~[classes/:?] {re:classloading,pl:runtimedistcleaner:A}
    at appeng.siteexport.SiteExporter.export(SiteExporter.java:172) ~[classes/:?] {re:classloading,pl:runtimedistcleaner:A}
    at appeng.siteexport.SiteExporter.lambda$initialize$0(SiteExporter.java:140) ~[classes/:?] {re:classloading,pl:runtimedistcleaner:A}
    at net.minecraftforge.eventbus.EventBus.doCastFilter(EventBus.java:260) ~[eventbus-6.0.5.jar:?] {}
    at net.minecraftforge.eventbus.EventBus.lambda$addListener$11(EventBus.java:252) ~[eventbus-6.0.5.jar:?] {}
    at net.minecraftforge.eventbus.EventBus.post(EventBus.java:315) ~[eventbus-6.0.5.jar:?] {}
    at net.minecraftforge.eventbus.EventBus.post(EventBus.java:296) ~[eventbus-6.0.5.jar:?] {}
    at net.minecraftforge.event.ForgeEventFactory.onPostClientTick(ForgeEventFactory.java:939) ~[forge-1.20.1-47.1.54_mapped_official_1.20.1-recomp.jar:?] {re:classloading}
    at net.minecraft.client.Minecraft.tick(Minecraft.java:1875) ~[forge-1.20.1-47.1.54_mapped_official_1.20.1-recomp.jar:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:ae2.mixins.json:PickColorMixin,pl:mixin:A,pl:runtimedistcleaner:A}
    at net.minecraft.client.Minecraft.runTick(Minecraft.java:1112) ~[forge-1.20.1-47.1.54_mapped_official_1.20.1-recomp.jar:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:ae2.mixins.json:PickColorMixin,pl:mixin:A,pl:runtimedistcleaner:A}
    at net.minecraft.client.Minecraft.run(Minecraft.java:718) ~[forge-1.20.1-47.1.54_mapped_official_1.20.1-recomp.jar:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:ae2.mixins.json:PickColorMixin,pl:mixin:A,pl:runtimedistcleaner:A}
    at net.minecraft.client.main.Main.main(Main.java:218) ~[forge-1.20.1-47.1.54_mapped_official_1.20.1-recomp.jar:?] {re:classloading,pl:runtimedistcleaner:A}
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] {}
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?] {}
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] {}
    at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] {}
    at net.minecraftforge.fml.loading.targets.CommonLaunchHandler.runTarget(CommonLaunchHandler.java:126) ~[loader-47.1.41.jar:47.1] {}
    at net.minecraftforge.fml.loading.targets.CommonLaunchHandler.clientService(CommonLaunchHandler.java:114) ~[loader-47.1.41.jar:47.1] {}
    at net.minecraftforge.fml.loading.targets.ForgeClientUserdevLaunchHandler.runService(ForgeClientUserdevLaunchHandler.java:19) ~[loader-47.1.41.jar:47.1] {}
    at net.minecraftforge.fml.loading.targets.CommonLaunchHandler.lambda$launchService$4(CommonLaunchHandler.java:108) ~[loader-47.1.41.jar:47.1] {}
    at cpw.mods.modlauncher.LaunchServiceHandlerDecorator.launch(LaunchServiceHandlerDecorator.java:30) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:53) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:71) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.Launcher.run(Launcher.java:108) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.Launcher.main(Launcher.java:78) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.BootstrapLaunchConsumer.accept(BootstrapLaunchConsumer.java:26) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.modlauncher.BootstrapLaunchConsumer.accept(BootstrapLaunchConsumer.java:23) ~[modlauncher-10.0.10.jar:?] {}
    at cpw.mods.bootstraplauncher.BootstrapLauncher.main(BootstrapLauncher.java:141) ~[bootstraplauncher-1.1.2.jar:?] {}
Caused by: java.io.FileNotFoundException: C:\Users\Sebastian\.javacpp\cache\windows-x86_64\jniavutil.dll (The process cannot access the file because it is being used by another process)
    at java.io.FileOutputStream.open0(Native Method) ~[?:?] {}
    at java.io.FileOutputStream.open(FileOutputStream.java:293) ~[?:?] {}
    at java.io.FileOutputStream.<init>(FileOutputStream.java:235) ~[?:?] {}
    at java.io.FileOutputStream.<init>(FileOutputStream.java:184) ~[?:?] {}
    at org.bytedeco.javacpp.Loader.extractResource(Loader.java:850) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.cacheResource(Loader.java:691) ~[javacpp-1.5.9.jar:1.5.9] {}
    at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1691) ~[javacpp-1.5.9.jar:1.5.9] {}
    ... 43 more

[saudet]

What does it use that isn't JarURLConnection?

[shartte]

It's a bit peculiar. This is in a Minecraft Modding Context. The major modding framework uses some auto-modularization classloader which in turn uses custom URLConnection for retrieving resources from the jar files.

p.s.: I could probably implement URLConnection lastModified / contentLength for that particular URL connection, but even if I did, it doesn't seem like it would be used by Loader.

[saudet]

Well, the idea is to get that information from somewhere, yes. Why wouldn't it work?

[shartte]

The current code never sets the modification date of the file if it takes that codepath in Loader.

[saudet]

File.setLastModified() gets called, yes.

[shartte]

I'll try to get you a better error description, but when I had a breakpoint set at the location outlined above (where it throws the access denied error), it was recursively extracting an existing file.

It does set it here (In the JarUrlConnection branch) https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/Loader.java#L812

But it doesn't set it here: https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/Loader.java#L850

Although looking at the call-tree it does try to set it in the parent method.

[saudet]

But it doesn't set it here: https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/Loader.java#L850

No, because that information isn't available there. Where can we get that information in the case of your URLConnection?

[shartte]

But it doesn't set it here: https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/Loader.java#L850

No, because that information isn't available there. Where can we get that information in the case of your URLConnection?

Theoretically via

https://docs.oracle.com/javase/8/docs/api/java/net/URLConnection.html#getLastModified--

(While that method is not actually implemented for the custom URLConnection, I could do that).

[saudet]

Right, so please try to do that, yes.

[shartte]

Implementing getLastModified & getContentLength in the custom class-loader does indeed solve this problem.