Open willfig opened 7 years ago
I cannot replicate this on my local machine, it seems to handle the mesh fine, albeit slowly. However, there could be differences between the way the OSX and Windows versions of the pypy interpreter handle the memory heap size. In any script, perhaps we might need to bump it up, especially when using large meshes such as this.
I will also look at limiting what is stored in memory for the vertices and faces, which is what is causing the error.
Did some reading. Pypy is 32 bit and as with all Windows 32bit apps, will only have access to max 2 GB ram. There is not 64bit version. I made sure to install 64bit python but it runs much slower as you know. Solutions:
Would something like that work?
I'll have a think about the data structures we are using. Unless a file is ~2BG this should not really be an issue, unless the python garbage collection is causing issues.
My latest thoughts are to try and parallelise the code, which seems to work from my initial testing. We could add another input parameter to the script - the number of cores on the computer (default to say 4). Then we might be able to get away without using pypy and run 64-bit python. What do you think of this approach @willfig ?
I've been playing around with parallelising the code and it's a bit of a headache. Might need to restructure the code. Perhaps running Linux on a virtual machine might be easier?
Hold off on this. I talked with Mitch from ACFR about this a bit. He has a lot of code already that does various versions of this. He suggested we may be able to use some of his core bits, coded in C and made available to python. So we could still use python for the main execution. In any case, he was going to have a look at his code and send something through as a test.
In the meantime, I did ask for a linux server. They just got back to me. If I got this up and running, would you be able to log on and install what we need to run the code? Is that easy?
Yep, can do.
ok, i'll ask them to build it.
I've got the script up and running on one of the high performance computers and I think I'm having memory issues. I grabbed a few Lizard Island meshes and one from One Tree Island. The One Tree one worked but the other two failed with errors as in the screen shot. The computer I was using has 32GB of RAM. I'll see if I can upload them to cloudstore.