Suggestion for those experiencing slow runtimes

[jdebacker]

@rickecon has noted a high variance in the run times of OG-PHL depending on one's compute environment.

One suggestion is to change line 27 of the run_og_phl.py script too:

client = Client(n_workers=num_workers, processes=False, threads_per_worker=1)

This will use multiple threads to distribute the compute rather than multiple cores/CPUs (seeDask docs on threads vs processes). A potential issue some are running into is that for security it might be difficult to move memory around the computer across multiple CPUs. By using multiple threads on a single CPU, one can avoid this.

Note that one may want to play with the settings for threads_per_worker as well.

[SeaCelo]

@jdebacker We'll test this with some of the participants that are experiencing significant solving times.

FYI, testing this change on an M1 MBP results in a much slower compute time, with low CPU utilization. This is probably only useful for those looking to solve the memory issue you mention.

In this image you can see the original client = (left half) and the revised one (right half). Increasing threads_per_worker doesn't affect my results.

[ARFajardo]

@jdebacker It's running slower than it used to on a windows machine with intel core i7. Before, it uses 100% of the CPU and now it only utilizes about 30%

[pmagnata-primo]

After adding "processes=False" implementation became significantly slower.

[rickecon]

@jdebacker and I spoke yesterday evening, and he suggested trying the following code on Windows machines:

• Set the processes input to False and set the threads_per_worker input to something greater than 1 (try 7).

  client = Client(
  n_workers=num_workers, processes=False, threads_per_worker=7
  )

Somebody with a Windows machine try this and see if it speeds up your runtime for the run_og_phl.py file.

[JustinGSimon]

Hi @jdebacker @rickecon! Similar to runs of @ARFajardo, the code changes for Line 27 (processes=False and threads_per_worker=7) resulted in a slower CPU performance of only around 20-48%. Before the suggested changes in Line 27, my computer used 100% of the CPU capacity.

[ARFajardo]

Hello @rickecon and @jdebacker. It's also running slower in my device with processes=False and threads_per_worker=7. It's only utilizing 40 - 60% of the CPU. It also pauses for a while before stepping into the next iteration for the transition path.