Open nix1 opened 3 years ago
For now, I just removed creating processes in pyirt/solver/model.py
. It probably became slower, but it worked.
So for instance:
for i in range(num_chunk):
p = mp.Process(target=update, args=(user_ell, user_cnt, chunk_list[i][0], chunk_list[i][1],))
procs.append(p)
if num_chunk > 1:
procs = procs_operator(procs, 1200, self.check_interval)
else:
procs = procs_operator(procs, 7200, 0.1)
becomes:
for i in range(num_chunk):
update(user_ell, user_cnt, chunk_list[i][0], chunk_list[i][1])
A similar situation happens around 4 times in this file.
I'm trying to use PyIRT, however it seems that the library has some issues with how it uses
multiprocessing
. I guess it used to be fine some time ago, I'm currently using Python 3.9.1.It seems that essentially PyIRT passes a local
update
function as atarget
toProcess
. In effect, whenmultiprocessing
tries to fork and spawn a new process, it tries to dump a pickle of the function, which fails withAttributeError: Can't pickle local object 'IRT_MMLE_2PL.__update_theta_distr.<locals>.update'
.Does anyone have a solution or a workaround to this issue?