search

openphotogrammetry / meshroomcl

MeshroomCL: An OpenCL implementation of photogrammetry with the Meshroom interface
Other
173 stars 8 forks source link

Exploding usage of RAM #6

Closed Expro closed 3 years ago

Expro commented 3 years ago

Hi,

I have observed very demanding use of RAM on StructureFromMotionCL node. 12x 4K photos consumed 20GB of RAM while computing this node, while same photoset on Meshroom with CUDA consumes 6GB of RAM. Not to mention, CUDA version game much better result, but that's not the point here.

Please look into more efficient memory management. In current state, MeshroomCL is impractical for even medium projects.

Regards, Expro

3D-360 commented 3 years ago

We use as much RAM as available to increase SFM processing speed, but we could change our approach. How much RAM is in your system? How long does SFM take on your 12 images?

revisionarian commented 3 years ago

Hi @Expro, thanks very much for your constructive feedback on MeshroomCL. As you observed, previous versions of MeshroomCL could use an excessive amount of RAM during execution of the StructureFromMotionCL node. This was due to many CPU threads running simultaneously on multi-core systems, and each individual thread could consume up to 3 GB of RAM in the case of high-resolution images.

Today, we have released a new version of the software, MeshroomCL 0.6.0, available here:

https://github.com/openphotogrammetry/meshroomcl/releases

This latest version has better memory management. During execution of the StructureFromMotionCL node, the number of threads will be limited such that no more than 75% of your system RAM will be consumed. Additionally, you have the option of changing the "Max CPU Threads" attribute of the node if you desire to limit the RAM usage further (fewer threads will use less memory, but will take longer time to process).

Please let us know if this new version of MeshroomCL addresses your concerns about memory usage, and if you have any other constructive criticism or ideas for the software. Thanks!

Expro commented 3 years ago

Hi @Expro, thanks very much for your constructive feedback on MeshroomCL.

It's you I should thank. I wished for Meshroom with OpenCL support for long time + I believe OpenCL needs flagship app to finally grab attention in compute world, and MeshroomCL could be that app.

Please let us know if this new version of MeshroomCL addresses your concerns about memory usage

I'm happy to report that 0.6 computed successfully same project without neither hitting swap and crashing VM not need to raise RAM to 24GB. In short: this issue was successfully resolved.

if you have any other constructive criticism or ideas for the software

I will. For now I think it would be nice to be able to configure this 75% cap in GUI. I would happy raise it to 90% for this project if it meant shorter compute time.