defold / editor2-issues

DEPRECATED
45 stars 4 forks source link

Java heap space in build time #443

Closed tarnumius closed 7 years ago

tarnumius commented 7 years ago

Expected behaviour

Start project

Actual behaviour

Receieve java heap space and run old version of project build :) Java memory in use about 2.5Gb, but my system has 8Gb

Steps to reproduce

Build project log in attach log_send2.txt


Defold version1.2.96
Defold sha9a3c683d14e3b3e375fc494e83a32523d550771b
Build time2017-01-27T17:45:00.623594
OS nameWindows 7
OS version6.1
OS archamd64
Java version1.8.0_102-b14
Error57fe7dd852174fb2b12cd8edf22f0bf7
ragnard-king commented 7 years ago

Hi @tarnumius! Thanks for your report.

Two things:

  • could you verify the JVM version in use by running this command from the root of your Defold installation: packages\jre\bin\java -version
  • are you able to share this project (or a smaller reproducing case) with us?

From what I can see, you're running out of memory when building an atlas (although the culprit could be somewhere else). Are you able to build the project sometimes, or consistently not at all?

tarnumius commented 7 years ago
  1. I check version
  2. Yes i can. I make java heap version of project, connected with different version of graphics content, and say you later.
ragnard-king commented 7 years ago

@tarnumius Ok, excellent, thanks for your help. If you want to share a reproducing project with me privately (ie. not post on github) you can contact me on ragnar.dahlen@king.com.

tarnumius commented 7 years ago

java version "1.8.0_102" Java(TM) SE Runtime Environment (build 1.8.0_102-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

Project I send to you email

tarnumius commented 7 years ago

I send email.

ragnard-king commented 7 years ago

Got it, thanks! We will look into this.

ragnard-king commented 7 years ago

Could you provide the output of the following command?

packages\jre\bin\java -XX:+PrintFlagsFinal -version | findstr HeapSize
tarnumius commented 7 years ago

All strings with HeapSize: uintx ErgoHeapSizeLimit = 0 {product} uintx HeapSizePerGCThread = 87241520 {product} uintx InitialHeapSize := 134217728 {product} uintx LargePageHeapSizeThreshold = 134217728 {product} uintx MaxHeapSize := 2130706432 {product}

tarnumius commented 7 years ago

btw - first editor build project, but have error in dmengine.exe after start

ragnard-king commented 7 years ago

That output tells me that the JVM will be limited to ~2G of heap. You can try manually increasing this by editing the file called "config" in your Defold directory, and add -Xmx4G first on the vmargs line, so it looks like:

vmargs = -Xmx4g,...

Regarding the dmengine error, I got an error because of nested comments some script files, ie. nested [[, but removing that allowed me to build successfully.

tarnumius commented 7 years ago

about dmengine - thank you. I try up heap

tarnumius commented 7 years ago

Ragnar big thanks for help.

tarnumius commented 7 years ago

But I have question - why editor required how many memory? My graphics pack not very big.

ragnard-king commented 7 years ago

We're still investigating this issue. The new editor does require more memory in general, a tradeoff to get much better performance for some operations, but I agree that for this project and graphics pack the usage is unreasonable. Will keep this issue updated, thanks for reporting!

tarnumius commented 7 years ago

Editor 2 cool btw! Thank you too.

tarnumius commented 7 years ago

I make some investigation. If I open main.collection on scene - Editor allocated 4GB memory without any other actions. Its strange :(

After I try build project with open main.collection - Editor allocate 5.2Gb and crushed again :)

tarnumius commented 7 years ago

From first editor. I delete all comments in scripts

Assertion failed: dmMemory::RESULT_OK == dmMemory::AlignedMalloc((void**)&message_buffer, 16, message_buffer_size), file ..\src\ddf\ddf.cpp, line 175

tarnumius commented 7 years ago

And after make windows bundle project crushed. In archive bundle https://drive.google.com/file/d/0B9YzeJRaLjGeTF9ITGxzYlpwcWc/view?usp=sharing

tarnumius commented 7 years ago

Hi guys.

I cannot continue my work :( I load second pack graphics and I can make only one build :) After one build memory not clear and java process allocate 4-5GB, after try second build - java process allocate more memory and die(all build without open main.collection).

All game objects, scripts, and atlas I make in external text editor :) If I open main.collection Editor allocate more 4g memory and build impossible :)

May be you can make clear memory button or something :)

Sorry for annoying, but its really strange behavior

tarnumius commented 7 years ago

I see strange behavior with files _generated_f7237a81.texturec after build. Atlas with images weight - 1.2Mb, but after build texturec file its file weight 171M

More strange behavior pack weight 600k, texture after build - 171M

Both packs have anim sequence

Atlas with 1 png file weight 55k make texturec file weight 21M

ragnard-king commented 7 years ago

One major feature we have not yet implemented in editor2 is texture profiles, enabling compression of image data. The is the reason you're seeing large texturec files on disk, much larger than your source image data (which is compressed). This is absolutely on the to-do list, but if you need this to work right now your probably better of using editor1 for this project until we've implemented support in editor2.

We also currently don't do a good job of reclaiming memory used by scenes with lots of image data, and I'll create an internal issue to track that.

Thanks for your patience and great feedback testing editor2!

tarnumius commented 7 years ago

Thank you for excellent and easy engine. Ok I read about texture profiles and use Editor1. And wait implemented in Editor2 :)

ragnard-king commented 7 years ago

A fix that improves our memory usage for projects that have lots and/or large images will be available in an update later today. Closing this now as I believe that was the root cause for this issue.