search

CadQuery / cadquery

A python parametric CAD scripting framework based on OCCT
https://cadquery.readthedocs.io
Other
2.93k stars 276 forks source link

Can one limit the number of cores or threads to run in parallel? #1600

Open rfryeSigma opened 3 weeks ago

rfryeSigma commented 3 weeks ago

I am running cadquery in a LatticeQuery environment which defines the function eachpointAdaptive similar to https://github.com/CadQuery/cadquery/issues/628#issuecomment-807493984. The script runs quite fast since shapes.py sets SetRunParallel true, and it recruits all processors.

But I am running on a server where it is not polite to hog the whole machine. Is there a way to tell cadquery to limit the maximum number of cores or threads to run in parallel?

adam-urbanczyk commented 3 weeks ago

I think like so, but please test and report back:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s()
pool.SetNbDefaultThreadsToLaunch(N_THREADS)
rfryeSigma commented 3 weeks ago

I just tried it, and it is still using all cores despite telling it to use only 3 out of the 10 on my MAC. This is my script and some of the output:

(lq) rogerfrye@Rogers-Laptop LatticeQuery % python Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:54:21) [Clang 16.0.6 ] on darwin Type "help", "copyright", "credits" or "license" for more information.

import cadquery as cq from lq.topologies.bcc import bcc_heterogeneous_lattice import OCP pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s() pool.SetNbDefaultThreadsToLaunch(3) unit_cell_size = 10 min_strut_diameter = 0.8 max_strut_diameter = 5. min_node_diameter = 0.88 max_node_diameter = 5.5 Nx = 4 Ny = 4 Nz = 20 cq.Workplane.bcc_heterogeneous_lattice = bcc_heterogeneous_lattice result = bcc_heterogeneous_lattice(unit_cell_size, ... min_strut_diameter, ... max_strut_diameter, ... min_node_diameter, ... max_node_diameter, ... Nx, Ny, Nz, ... topology = 'bcc', ... rule = 'parabola' ... ) [0.4 0.89861496 1.34182825 1.72963989 2.06204986 2.33905817 2.56066482 2.72686981 2.83767313 2.89307479 2.89307479 2.83767313 2.72686981 2.56066482 2.33905817 2.06204986 1.72963989 1.34182825 0.89861496 0.4 ] Datapoints generated Building an element of an array... Building an element of an array... Success!

Building an element of an array... Success!

....

adam-urbanczyk commented 3 weeks ago

Other ideas:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s()
pool.Init(N_THREADS)

or try setting the TBB_NUM_THREADS env var, if your OCCT is built with TBB.

adam-urbanczyk commented 3 weeks ago

Based on the OCCT code, maybe even like so:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s(N_THREADS)

https://github.com/Open-Cascade-SAS/OCCT/blob/4c8faa5e2dd0d3e4bd7039c908016d8204889c09/src/OSD/OSD_ThreadPool.hxx#L57

rfryeSigma commented 3 weeks ago

The solution turned out to be to set the number of threads before importing cadquery: import OCP pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s(1) import cadquery as cq

Thank you!