search

opensourceBIM / BIMserver

The open source BIMserver platform
GNU Affero General Public License v3.0
1.54k stars 608 forks source link

uploading .IFC of sizse 500 mb with 5gb of heap memory incurs connection reset error #267

Closed sanjeevghimire closed 8 years ago

sanjeevghimire commented 8 years ago

uploading .IFC of size 500 mb with 5gb of heap memory incurs connection reset error. I turned off generate geometry during checkin, "it gave no endpoint 6 found". Before that it gave me "no endpoint 4 found." Any help is appreciated

thanks Sanjeev

sanjeevghimire commented 8 years ago

And can you give us insight whats the minimum required for uploading atleast 500mb of IFC file? All our clients used greater than 500mb size of the IFC file.

sanjeevghimire commented 8 years ago

any help on this is appreciated? and how can we scale BIM server loadbalance or is it jut increasing memory size?

rubendel commented 8 years ago

If the error is that the "connection is reset", without more information this implies something is wrong network wise. Please have a look in the log files, and confirm you actually get errors about the heap size.

The memory requirements are not necessarily linearly related to the size of the IFC file, some files generate huge amounts of geometry, some almost nothing. But in general, a very broad rule of thumb is you need at least 17 times the amount of memory, so 8.5 GB in this case. We are working on lowering the memory usage, but it wil always be quite high (and faster the more memory you have)

sanjeevghimire commented 8 years ago

Thanks Ruben. the connection reset was more related teh websocket. i see lot of places in logs giving web socket connection issue. stack trace is below.

24-11-2015 17:13:22 ERROR (org.bimserver.servlets.websockets.jsr356.Jsr356Impl.java:92) java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at org.apache.coyote.http11.upgrade.BioServletInputStream.doRead(BioServletInputStream.java:37) at org.apache.coyote.http11.upgrade.AbstractServletInputStream.read(AbstractServletInputStream.java:129) at org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:47) at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsReadListener.onDataAvailable(WsHttpUpgradeHandler.java:203) at org.apache.coyote.http11.upgrade.AbstractServletInputStream.onDataAvailable(AbstractServletInputStream.java:203) at org.apache.coyote.http11.upgrade.AbstractProcessor.upgradeDispatch(AbstractProcessor.java:93) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:623) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

do you know if we can run the BerkeleyDB separately /remotely like other databases so that we can loadbalance bimserver??

Appreciated and thanks Sanjeev

sanjeevghimire commented 8 years ago

BTW IFCOpenShell takes forever to generate the geometry. Is there any other way this can be reduced?

rubendel commented 8 years ago

No, BerkeleyDB cannot be run outside of BIMserver, basically we see BIMserver as a specialized database server, your application is the application using it. Load balancing is not possible at the moment.

The other engine that is included in the build is a lot faster, but does not generate colors/materials. You can also run the engine on multiple processes. Howeverth this is is experimental. You can enable it by going to "Server Settings" and changing "Max Render Engine Processes" to for example 4 (the amount of cores available).

sanjeevghimire commented 8 years ago

I don't see "Server Settings" and changing "Max Render Engine Processes" in the version I am using. I am using the latest version though.

rubendel commented 8 years ago

In the top menu click "Server", then "Server Settings", then there is a setting "Max Render Engine Processes (BETA)".