opensourceBIM / IfcOpenShell-BIMserver-plugin

IfcOpenShell BIMserver plugin
Other
51 stars 28 forks source link

ERROR org.bimserver.BimServer - java.nio.file.FileSystemException: E:\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar: The process cannot access the file because it is being used by another process. #9

Open stegh opened 5 years ago

stegh commented 5 years ago

Hello,

we've had issues with installing BIMserver on machines without internet connection, maybe it's "our fault", - I'm not sure if installing on machines without internet connection is supported by BIMserver. (and to be honest; we're not entirely sure if the behaviour is related to missing internet connection).

On machines with internet connection some downloads start after this line...

[BimServer.java:1091] Initializing database dependant logic...

ifcopenshellplugin is downloaded from AWS, e.g. this line:

INFO [PluginInterface.installPluginBundle] o.i.IfcGeomServerClient [IfcGeomServerClient.java:193] Downloading from https://s3.amazonaws.com/ifcopenshell-builds/IfcGeomServer-v0.6.0-4f8b430-win64.zip

On machines without internet connection we get this error:

ERROR org.bimserver.BimServer - java.nio.file.FileSystemException: E:\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar: The process cannot access the file because it is being used by another process.

...here in German (and long version ;)):

ERROR [main] o.b.BimServer [BimServer.java:1228] java.nio.file.FileSystemException: C:\Program Files (x86)\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar: Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird. at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source) at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source) at java.nio.file.Files.delete(Unknown Source) at org.bimserver.plugins.PluginManager.loadPlugin(PluginManager.java:1428) at org.bimserver.plugins.PluginManager.loadFromPluginDir(PluginManager.java:1398) at org.bimserver.BimServer.initDatabaseDependantItems(BimServer.java:1226) at org.bimserver.BimServer.start(BimServer.java:755) at org.bimserver.web.WarServerInitializer.contextInitialized(WarServerInitializer.java:116) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5105) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415) at java.util.concurrent.FutureTask.run(Unknown Source) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.util.concurrent.AbstractExecutorService.submit(Unknown Source) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941) at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:839) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415) at java.util.concurrent.FutureTask.run(Unknown Source) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.util.concurrent.AbstractExecutorService.submit(Unknown Source) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941) at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:258) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:770) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.startup.Catalina.start(Catalina.java:671) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)

I tried to find out about the lock / sharing using https://live.sysinternals.com/Procmon.exe

Filtering PATH ends with org.opensourcebim.ifcopenshellplugin-0.5.38.jar

The error occurs at every (re)start of the BIMserver service / Tomcat service.

You can see a SHARING VIOLATION, - but I don't know why...


Event Class: File System Operation: CreateFile Result: SHARING VIOLATION Path: E:\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar TID: 6464 Duration: 0.0000367 Desired Access: Read Attributes, Delete Disposition: Open Options: Non-Directory File, Open Reparse Point Attributes: n/a ShareMode: Read, Write, Delete AllocationSize: n/a

Any help/suggestions appreciated ;)

Regards, Thomas

stegh commented 5 years ago

I guess the issue is related to the following code:

https://github.com/opensourceBIM/IfcOpenShell-BIMserver-plugin/blob/master/src/org/ifcopenshell/IfcGeomServerClient.java#L156

rubendel commented 5 years ago

I don't really understand why you are getting this specific error, but I can explain how the IfcOpenShell plugin is working now. Basically a specific commit-hash is hardcoded in the plugin, and it will download the correct version of the binaries for your platform from S3. The main reasons for this are:

So basically, although BIMserver does provide a solution for computers with no internet connection, currently the IfcOpenShell plugin does not. We plan to provide a special build of the plugin in the future.

The only quick solution I see for now would be to put the correctly named executable directly in the correct folder before you start the BIMserver. Currently that would be home\tmp\org.opensourcebim.ifcopenshellplugin\IfcGeomServer-v0.6.0-4380e1d-win64.exe for 64bit windows.

stegh commented 5 years ago

Thanks, I tried to do this as well:

put the correctly named executable directly in the correct folder before you start the BIMserver. Currently that would be home\tmp\org.opensourcebim.ifcopenshellplugin\IfcGeomServer-v0.6.0-4380e1d-win64.exe

but it didn't work ...it's always the same: without internet connectivity (direct, not via proxy!) it works, but on systems without internet connectivity or via proxies it does not work...

To reproduce this issue on our servers I started with a new virtual machine using VMware vSphere Client Console, - no network connection (deactivated network adapter), instead using hosts file entry:

127.0.0.1 qm27159.zentrale.etask.de The server has no network, no DNS, no Internet access, - only localhost.

I could reproduce the issue, in bimserver.log there is this error 2 times:

java.nio.file.FileSystemException: C:\Program Files (x86)\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar: Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird. English: The process cannot access the file because it is being used by another process.

Next steps:

Results from bimserver.log:

My conclusion is: If there is working internet connectivity, the error does not occur.

However the problem is not solved on two other systems; - I gave Internet access (via proxy) to one of those systems and tested again, - didn't help there.

I tried to download this file in browsers on the server, this works: https://s3.amazonaws.com/ifcopenshell-builds/IfcGeomServer-v0.6.0-4380e1d-win64.zip

Proxy settings in Java are ok as well (I tried all possibilites), but it looks like the proxy is not used by Bimserver, - you can see this in the following logs; here we go...

etask.bimserver-stderr.2019-01-31.log

31-Jan-2019 14:01:38.334 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [E:\eTASK\BIMserver\webapps\ROOT] has finished in [16] ms 31-Jan-2019 14:01:49.568 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. 31-Jan-2019 14:02:36.985 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"] 31-Jan-2019 14:02:37.001 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-172.22.154.27-80"] 31-Jan-2019 14:02:37.001 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-openssl-nio-172.22.154.27-443"] 31-Jan-2019 14:02:37.017 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 59461 ms nManager.java:1398) at org.bimserver.BimServer.initDatabaseDependantItems(BimServer.java:1226) at org.bimserver.BimServer.start(BimServer.java:755) at org.bimserver.web.WarServerInitializer.contextInitialized(WarServerInitializer.java:116) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5105) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415) at java.util.concurrent.FutureTask.run(Unknown Source) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.util.concurrent.AbstractExecutorService.submit(Unknown Source) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941) at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:839) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415) at java.util.concurrent.FutureTask.run(Unknown Source) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.util.concurrent.AbstractExecutorService.submit(Unknown Source) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941) at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:258) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:770) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.startup.Catalina.start(Catalina.java:671) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495) Caused by: java.net.ConnectException: Connection timed out: connect at java.net.DualStackPlainSocketImpl.connect0(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at sun.security.ssl.SSLSocketImpl.connect(Unknown Source) at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source) at sun.net.NetworkClient.doConnect(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.protocol.https.HttpsClient.(Unknown Source) at sun.net.www.protocol.https.HttpsClient.New(Unknown Source) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source) at java.net.URL.openStream(Unknown Source) at org.ifcopenshell.IfcGeomServerClient.getExecutable(IfcGeomServerClient.java:171) ... 37 more

and ultimately again:

bimserver.log

34153 [main] ERROR org.bimserver.BimServer - java.nio.file.FileSystemException: E:\eTASK\BIMserver\Database\plugins\org.opensourcebim.ifcopenshellplugin-0.5.38.jar: The process cannot access the file because it is being used by another process.

at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
at java.nio.file.Files.delete(Unknown Source)
at org.bimserver.plugins.PluginManager.loadPlugin(PluginManager.java:1428)
at org.bimserver.plugins.PluginManager.loadFromPluginDir(PluginManager.java:1398)
at org.bimserver.BimServer.initDatabaseDependantItems(BimServer.java:1226)
at org.bimserver.BimServer.start(BimServer.java:755)
at org.bimserver.web.WarServerInitializer.contextInitialized(WarServerInitializer.java:116)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5105)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:839)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:258)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:770)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:671)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)

Regards, Thomas

rubendel commented 5 years ago

Maybe I missed it, but which version of BIMserver is this? Because the line numbers don't correspond with what I have here.

stegh commented 5 years ago

Maybe I missed it, but which version of BIMserver is this? Because the line numbers don't correspond with what I have here.

Sorry, it's 1.5.117

stegh commented 5 years ago

I don't understand how Bimserver, or rather IfcOpenShell (or Java?!) handles proxies.

From logs above:

Caused by: java.net.ConnectException: Connection timed out: connect

I'd be perfectly happy to be able to use the systems with internet connectivity (so what ;))

rubendel commented 5 years ago

Yeah the proxies just don't work in BIMserver, it's too much a hassle to test everything, I actually struggled to setup a computer without internet connection, it's kinda hard to reach those...

In my opinion routers are for routing traffic, they are really good at that. Firewalls are for blocking/allowing stuff to pass through. Proxies (for outgoing connections) should have never existed, they are playing at the application level of the OSI model, but don't add any functionalities (like an application should). But that's just my rant, doesn't really help you ;).

I did do a few changes to the management of resources on the 23rd of November 2018 (e0d9c1c7bff87d4da9dfca8c00a7ca64636c140d), which is one day after 1.5.117 was released. So you might have better luck with a newer version on this front. Since ther error is about a file being locked, it could have something to do with that.

When I have time I'll at least test the suggestion I gave locally.

stegh commented 5 years ago

Thanks a lot! That explains it and helps. We should be able to resolve the issue with this information.

stegh commented 5 years ago

I was able to verify that there is indeed an issue with BIMserver 1.5.117 and IfcOpenShell Plugin that makes it impossible to run BIMserver on machines without (direct) internet connectivity (proxy servers are not supported by BIMserver).

Workaround: I did a really dirty "hack" in order to work around this issue, - which involves hosts file and "faking" S3 AWS URL, - not nice, but it works, like this:

Prerequisites:

You now should have working URLs (reachable from BIMserver machine):

...and BIMserver 1.5.117 works with this "faked" s3.amazonaws.com URL ;)

31-Jan-2019 23:13:43.534 INFORMATION [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
31-Jan-2019 23:14:13.722 INFORMATION [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
31-Jan-2019 23:14:13.737 INFORMATION [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-127.0.0.1-80"]
31-Jan-2019 23:14:13.737 INFORMATION [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-openssl-nio-127.0.0.1-443"]
31-Jan-2019 23:14:13.737 INFORMATION [main] org.apache.catalina.startup.Catalina.start Server startup in 44113 ms
15140 [main] INFO  org.bimserver.BimServer - Done initializing database dependant logic
15859 [main] INFO  o.b.s.websockets.jsr356.Jsr356Impl - WebSocket context initialized
rubendel commented 5 years ago

Wow, thanks for the extensive update, could be useful for other people in the same situation. I'll try to remember to place a comment in this issue when we get to setting-up a special variant of the IOS plugin with the binaries embedded.

cacu100 commented 5 years ago

Thanks @rubendel rubendel and @stegh stegh. I'm using BIMserver 1.5.117 war over Tomcat. It was running well before the installation of 1.5.125 in parallel in the same server. I read the @klacol suggestion on https://github.com/opensourceBIM/BIMserver/issues/943#issuecomment-467864583. I guess that the things are different for war deployments. What's the best recommendation to fix this issue for 1.5.117 war installations with BIMview

rubendel commented 5 years ago

Using 1.5.125 is the best recommendation imo, I assume you are not using it in production so it can't be that hard to switch.

cacu100 commented 5 years ago

Thanks @rubendel for your soon reply. I tested the 1.5.125 version but when I try to download the gltf model file using the API, it fails. This feature works well in 1.5.117. Are you planning a new version to fix the generation gltf issue

rubendel commented 5 years ago

Please report a bug in the correct issue tracker, "it fails" is not enough information.

cacu100 commented 5 years ago

Thanks @rubendel. I uploaded the same ifc file on both versions 1.5.117 and 1.5.125. The glb file generated throws the API using the download method on 1.5.125 generates a file that I can't read using THREE.GLTFLoader. The error displayed is : JSON.parse: unexpected non-digit at line 1 column 249274 of the JSON data It works with a glb model generated using version 1.5.117. I tested the same code with version 1.125 and shows the related error message. The zip file contains the glb files generated with 1.5.125, 1.5.117 and the original ifc uploaded on the server Prototipo.zip This is my code to load the model on my visualizer app.

let loader = new THREE.GLTFLoader(); loader.setDRACOLoader( new THREE.DRACOLoader() ); loader.load(this.BIMServerUrl+"/download?token="+this.token+"&topicId="+topicId , // onLoad callback ( gltf ) => { let scene = gltf.scene || gltf.scenes[0]; let clips = gltf.animations || []; this.setContent(scene, clips); }, // onProgress callback function ( xhr ) { // console.log( (xhr.loaded / xhr.total * 100) + '% loaded' ); }, // onError callback function ( err ) { console.error( err ); }); Thanks again