search

ChristophSchranz / Tweaker-3

A performant auto-rotate software for objects in 3D Printing, whose parameter are trained by an evolutionary algorithm.
GNU General Public License v3.0
91 stars 25 forks source link

High Memory Usage #8

Closed Renari closed 4 years ago

Renari commented 4 years ago

Giving tweaker a larger stl (180mb) tweaker seems to use a little over 2GB of memory to orient this file. Is there anything that can be done to minimize memory usage?

ChristophSchranz commented 4 years ago

Hi @Renari What version are you using? Some weeks ago I've removed a duplicated object, that should have been around 300MB for your file.

It seems not easy to decrease the memory usage without re-reading the file again for the rotation part and therefore increase the CPU costs and time usage. Currently, there is only the object itself in the memory, as well as an augmented mesh structure. I've now made a commit that deletes this mesh structure as soon as it isn't used any more. This reduced the memory by ~300MB for my 160MB file.

The object itself is used in the rotation part at the end, hence, we should leave it in memory. Maybe you can find a better option in FileHandler.py. This part is quite old and I'm sure the content could be read in a better and more compact way.

ChristophSchranz commented 4 years ago

I've investigated some more and made some quick improvements in this commit where I removed some objects that are not used any more in the writing phase. The problem here, is that most functions are not implemented consistently using in either numpy or built-in lists and without a line-wise replacement, e.g., in https://github.com/ChristophSchranz/Tweaker-3/blob/3aea2d90422076c0486de43c0979379d7bc53e4a/FileHandler.py#L157

The memory peak is reached while writing on the disk: https://github.com/ChristophSchranz/Tweaker-3/blob/3aea2d90422076c0486de43c0979379d7bc53e4a/FileHandler.py#L165 So far, I don't know how to optimize that, however, please feel free to have a look at this file and to open pull requests.

Furthermore, using the extended mode using the flag -x needs some memory, as it has to order a larger derived object.

P.S.: I'll close this issue as it is know rather an optimization discussion.

Renari commented 4 years ago

Thanks Chris, I'll try out the changes you mentioned above and see if it makes a difference for me.