GES-compchem / job-dispatcher

In-home wrapper around Multiprocessing to parallelize the execution of independent functions.
https://ges-compchem.github.io/job-dispatcher/
GNU General Public License v3.0
1 stars 4 forks source link

Fixed race conditions in fetching results #31

Closed GES-mpalermo closed 1 year ago

GES-mpalermo commented 2 years ago

Also changed the naming of multithreating into threading

lbabetto commented 2 years ago

Sembra funzionare tutto!

Solves #30

GES-mpalermo commented 2 years ago

image

ghost commented 2 years ago

La modifica risolve il problema. Ho testato la funzione con il codice:

import logging
import jobdispatcher as jd

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

def compute_square(n: int) -> int:
    return n**2

jobs = []
for n in range(10):
    job = jd.Job(f"job_{n}", compute_square, arguments=[n], cores=1)
    jobs.append(job)

engine = jd.JobDispatcher(jobs, maxcores=8, engine="multiprocessing")
results = engine.run()

for key, value in results.items():
    print(f"{key}:\t{value}")

Unica osservazione: se, come nell'esempio, il logger è settato in modalità debug; un'eccezione TypeError: not all arguments converted during string formatting viene sollevata causa linea 150 in job-dispatcher/jobdispatcher/processor.py. Sostituirei la linea con logger.debug(f"Total number of rebalances: {counter_rebalances}") per risolvere il problema.

GES-mpalermo commented 1 year ago

Merging to main after positive review @GES-lbabetto @GES-ppravatto

image