posebusters: eval: DiffBindFR/DiffBindFR/evaluation/eval.py", line 162, in runner pd_df['smina_score'] = pd_df['docked_lig'].parallel_apply( .... Error: Ran out of input

[rytakahashi]

Dear Sir,

as in the documentation, I tries to repeat posebusters evaluations

python DiffBindFR/evaluation/eval.py -d /home/rtakahashi/software/DiffDock/data/posebusters_benchmark_set -o /home/rtakahashi/Work/ -j dock -n posebusters -lb pb -np 40 -gpu 0 -cpu 8 -bs 16 -eval -rp

It seems this execution is fine, but at the end on analysis

Traceback (most recent call last):
  File "/home/rtakahashi/software/DiffBindFR/DiffBindFR/evaluation/eval.py", line 277, in <module>
    runner(df, args)
  File "/home/rtakahashi/software/DiffBindFR/DiffBindFR/evaluation/eval.py", line 162, in runner
    pd_df['smina_score'] = pd_df['docked_lig'].parallel_apply(
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandarallel/core.py", line 324, in closure
    return wrapped_reduce_function(
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandarallel/core.py", line 199, in closure
    return reduce_function(dfs, extra)
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandarallel/data_types/series.py", line 34, in reduce
    return pd.concat(datas, copy=False)
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 285, in concat
    op = _Concatenator(
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 339, in __init__
    objs = list(objs)
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandarallel/core.py", line 195, in <genexpr>
    get_dataframe_and_delete_file(output_file_path)
  File "/home/rtakahashi/mambaforge/envs/diffbindfr/lib/python3.9/site-packages/pandarallel/core.py", line 189, in get_dataframe_and_delete_file
    data = pickle.load(file_descriptor)
EOFError: Ran out of input

This Dataframe error is usually just index is in the column ... etc. But, I did not figure out quickly. If you give a hit to solve this error, it will be great.

Many thanks,

[rytakahashi]

The above error looks associated with permission issues in druglib/ops/, so that when I changed, I don't see errors. However, now, the calculation stacked at

2024-04-11 19:44:38,926 - Evaluator - INFO - Start to MDN based pose scoring...
Processing 120 jobs, chunk 50000 per iter, total 1 iters...:   0%|          | 0/1 [00:00<?, ?it/s]INFO: Pandarallel will run on 8 workers.
INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.

could you give me a hint, how I could solve this issue?

Many thanks,

[Sinsilcobio]

I am also having the same issue with the process being stuck at this step. Any idea what might be the problem and how to resolve it?

[HBioquant]

Hi, @rytakahashi and @sajad-stack,

This seems to have happened before, and you can refer to this to try to solve it.

Best,

[rytakahashi]

Many thanks for your comment, when I looked at your link of the previous issue. apart of numpy, it looks to me I setup correctly, e.g. pytorch, etc. I have updated numpy 1.23.5. However, still the same issue, The calculations still stack in the MDN. at test_dataset. (Looks like getting into infinite loop ...?)

Just make sure with --no_mdn_scoring, I am able to complete it, but as the paper (supplemental is missing in the archive?), DiffBindFR+MDN has the best results. I would like to see by repeating PoseBusters.

Many thanks

[rytakahashi]

Looking at pandarallel, As your comments, the line 156 parallel_apply , so when I changed to simply ``applyorprogress_apply```seem to be working (do not stack in here-). Also, as your comment 'handle multi-process operation, such as using joblib or the more basic multiprocessing,', however, I have not tried this option yet.

Then, now, when I search I saw parallel_apply at the several places, but their operations are fine. Why only this line get stack is a puzzling. Do you have any insites?

Many thanks,

[HBioquant]

Hi, @rytakahashi,

I tried to reproduce the problem on several machines, but my program worked fine. I haven't been able to find out where the problem came from. Perhaps replacing the pandarallel library with another multi-process python library is a suitable solution.