jamesgleave / DD_protocol

Official repository for the Deep Docking protocol
MIT License
97 stars 31 forks source link

ValueError: Number of processes must be at least 1 #8

Closed xavierchee closed 10 months ago

xavierchee commented 10 months ago

Dear all, i'm running into issue with this error. Tried troubleshooting a few times but no avail. Can i seek some pointers?

Thanks!

I used these codes: python molecular_file_count_updated.py --project_name protein_test --n_iteration 1 --data_directory /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp --tot_process 1 --tot_sampling 3000000

But i get the following error:

molecular_file_count_updated: Parsed Args:
         molecular_file_count_updated:  - Iteration: 1
         molecular_file_count_updated:  - Data Directory: /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp
         molecular_file_count_updated:  - Training Size: 1
         molecular_file_count_updated:  - Validation Size: 3000000
         molecular_file_count_updated: Number Of Files: 0
         molecular_file_count_updated: Minimum Value: 0
         molecular_file_count_updated: Error: Minimum value is less than 1
         molecular_file_count_updated: Reading Files...
Traceback (most recent call last):
  File "/fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/molecular_file_count_updated.py", line 68, in <module>
    with closing(Pool(min_value)) as pool:
                 ^^^^^^^^^^^^^^^
  File "/home/xchee/.conda/envs/conda_vs/lib/python3.11/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xchee/.conda/envs/conda_vs/lib/python3.11/multiprocessing/pool.py", line 205, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1

However, when i check the path /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp using list, i can see all the files:

smiles_all_00.txt smiles_all_34.txt smiles_all_68.txt smiles_all_01.txt smiles_all_35.txt smiles_all_69.txt smiles_all_02.txt smiles_all_36.txt smiles_all_70.txt smiles_all_03.txt smiles_all_37.txt smiles_all_71.txt smiles_all_04.txt smiles_all_38.txt smiles_all_72.txt smiles_all_05.txt smiles_all_39.txt smiles_all_73.txt smiles_all_06.txt smiles_all_40.txt smiles_all_74.txt smiles_all_07.txt smiles_all_41.txt smiles_all_75.txt smiles_all_08.txt smiles_all_42.txt smiles_all_76.txt smiles_all_09.txt smiles_all_43.txt smiles_all_77.txt smiles_all_10.txt smiles_all_44.txt smiles_all_78.txt smiles_all_11.txt smiles_all_45.txt smiles_all_79.txt smiles_all_12.txt smiles_all_46.txt smiles_all_80.txt smiles_all_13.txt smiles_all_47.txt smiles_all_81.txt smiles_all_14.txt smiles_all_48.txt smiles_all_82.txt smiles_all_15.txt smiles_all_49.txt smiles_all_83.txt smiles_all_16.txt smiles_all_50.txt smiles_all_84.txt smiles_all_17.txt smiles_all_51.txt smiles_all_85.txt smiles_all_18.txt smiles_all_52.txt smiles_all_86.txt smiles_all_19.txt smiles_all_53.txt smiles_all_87.txt smiles_all_20.txt smiles_all_54.txt smiles_all_88.txt smiles_all_21.txt smiles_all_55.txt smiles_all_89.txt smiles_all_22.txt smiles_all_56.txt smiles_all_90.txt smiles_all_23.txt smiles_all_57.txt smiles_all_91.txt smiles_all_24.txt smiles_all_58.txt smiles_all_92.txt smiles_all_25.txt smiles_all_59.txt smiles_all_93.txt smiles_all_26.txt smiles_all_60.txt smiles_all_94.txt smiles_all_27.txt smiles_all_61.txt smiles_all_95.txt smiles_all_28.txt smiles_all_62.txt smiles_all_96.txt smiles_all_29.txt smiles_all_63.txt smiles_all_97.txt smiles_all_30.txt smiles_all_64.txt smiles_all_98.txt smiles_all_31.txt smiles_all_65.txt smiles_all_99.txt smiles_all_32.txt smiles_all_66.txt smiles_all_33.txt smiles_all_67.txt

An example of a file content is: ZINC000000000638_1,2,4,13,29,64,80,120,121,145,147,175,301,356,423,433,491,4 94,504,568,649,650,659,661,695,726,728,807,832,849,890,891,892,893,910,926,9 54,967,983,1019 ZINC000000000794_1,1,21,29,33,64,80,104,118,123,124,159,175,207,227,232,234, 329,356,367,386,399,428,572,582,666,695,698,705,726,773,789,807,849,861,926, 946,948,975,988

fgentile89 commented 10 months ago

Hi, This is a strange error. Can you confirm that you are using the protocol version of the python script? You are probably using an older version.

If it doesn't work, can you check what python sees with glob.glob

for f in glob.glob(' /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp' ): files.append(f) print("Number Of Files:", len(files))

Il giorno gio 16 nov 2023 alle ore 20:41 xavierchee < @.***> ha scritto:

Dear all, i'm running into issue with this error. Tried troubleshooting a few times but no avail. Can i seek some pointers?

Thanks!

I used these codes: python molecular_file_count_updated.py --project_name protein_test --n_iteration 1 --data_directory /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp --tot_process 1 --tot_sampling 3000000

But i get the following error:

molecular_file_count_updated: Parsed Args: molecular_file_count_updated: - Iteration: 1 molecular_file_count_updated: - Data Directory: /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp molecular_file_count_updated: - Training Size: 1 molecular_file_count_updated: - Validation Size: 3000000 molecular_file_count_updated: Number Of Files: 0 molecular_file_count_updated: Minimum Value: 0 molecular_file_count_updated: Error: Minimum value is less than 1 molecular_file_count_updated: Reading Files... Traceback (most recent call last): File "/fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/molecular_file_count_updated.py", line 68, in with closing(Pool(min_value)) as pool: ^^^^^^^^^^^^^^^ File "/home/xchee/.conda/envs/conda_vs/lib/python3.11/multiprocessing/context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/xchee/.conda/envs/conda_vs/lib/python3.11/multiprocessing/pool.py", line 205, in init raise ValueError("Number of processes must be at least 1") ValueError: Number of processes must be at least 1

However, when i check the path /fred/oz241/xchee/[sandbox]_Deepdocking/phase_1/Morgan/library_prepared_fp using list, i can see all the files:

smiles_all_00.txt smiles_all_34.txt smiles_all_68.txt smiles_all_01.txt smiles_all_35.txt smiles_all_69.txt smiles_all_02.txt smiles_all_36.txt smiles_all_70.txt smiles_all_03.txt smiles_all_37.txt smiles_all_71.txt smiles_all_04.txt smiles_all_38.txt smiles_all_72.txt smiles_all_05.txt smiles_all_39.txt smiles_all_73.txt smiles_all_06.txt smiles_all_40.txt smiles_all_74.txt smiles_all_07.txt smiles_all_41.txt smiles_all_75.txt smiles_all_08.txt smiles_all_42.txt smiles_all_76.txt smiles_all_09.txt smiles_all_43.txt smiles_all_77.txt smiles_all_10.txt smiles_all_44.txt smiles_all_78.txt smiles_all_11.txt smiles_all_45.txt smiles_all_79.txt smiles_all_12.txt smiles_all_46.txt smiles_all_80.txt smiles_all_13.txt smiles_all_47.txt smiles_all_81.txt smiles_all_14.txt smiles_all_48.txt smiles_all_82.txt smiles_all_15.txt smiles_all_49.txt smiles_all_83.txt smiles_all_16.txt smiles_all_50.txt smiles_all_84.txt smiles_all_17.txt smiles_all_51.txt smiles_all_85.txt smiles_all_18.txt smiles_all_52.txt smiles_all_86.txt smiles_all_19.txt smiles_all_53.txt smiles_all_87.txt smiles_all_20.txt smiles_all_54.txt smiles_all_88.txt smiles_all_21.txt smiles_all_55.txt smiles_all_89.txt smiles_all_22.txt smiles_all_56.txt smiles_all_90.txt smiles_all_23.txt smiles_all_57.txt smiles_all_91.txt smiles_all_24.txt smiles_all_58.txt smiles_all_92.txt smiles_all_25.txt smiles_all_59.txt smiles_all_93.txt smiles_all_26.txt smiles_all_60.txt smiles_all_94.txt smiles_all_27.txt smiles_all_61.txt smiles_all_95.txt smiles_all_28.txt smiles_all_62.txt smiles_all_96.txt smiles_all_29.txt smiles_all_63.txt smiles_all_97.txt smiles_all_30.txt smiles_all_64.txt smiles_all_98.txt smiles_all_31.txt smiles_all_65.txt smiles_all_99.txt smiles_all_32.txt smiles_all_66.txt smiles_all_33.txt smiles_all_67.txt

An example of a file content is:

ZINC000000000638_1,2,4,13,29,64,80,120,121,145,147,175,301,356,423,433,491,4

94,504,568,649,650,659,661,695,726,728,807,832,849,890,891,892,893,910,926,9 54,967,983,1019

ZINC000000000794_1,1,21,29,33,64,80,104,118,123,124,159,175,207,227,232,234,

329,356,367,386,399,428,572,582,666,695,698,705,726,773,789,807,849,861,926, 946,948,975,988

— Reply to this email directly, view it on GitHub https://github.com/jamesgleave/DD_protocol/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJNASLHT7VLNSAXXRSJATYE26DXAVCNFSM6AAAAAA7PCIZ2CVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TQMBZG4ZTMMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Francesco Gentile

xavierchee commented 10 months ago

Thanks Francesco for your very prompt response!

Hello, i downloaded the code from https://github.com/jamesgleave/DD_protocol/blob/main/scripts_1/molecular_file_count_updated.py

I've redownload the code but i still get the same error message.

I wrote two scripts: Script 1

import os

# Get a list of all files in current directory
files = os.listdir('/fred/oz241/xchee/[sandbox]_Deepdocki
ng/phase_1/Morgan/library_prepared_fp')
print("Number Of Files:", len(files))

Number of files : 100

Script 2

import glob
import os

# Get a list of all files in current directory
files = [f for f in glob.glob('/fred/oz241/xchee/[sandbox
]_Deepdocking/phase_1/Morgan/library_prepared_fp') if os.
path.isfile(f)]
print("Number Of Files:", len(files))

Number of files : 0

Thanks! It seems that for some reasons Python is not seeing those files.

fgentile89 commented 10 months ago

I think it is due to the fact that your path has squared brackets in it ([]) which is interpreted differently by glob.glob (character classes). If you can modify your path name (eg renaming the [sandbox]_Deepdocking to sandbox_Deepdocking, and change the command accordingly) it should fix the problem. There are also other solutions

Francesco

Il giorno gio 16 nov 2023 alle ore 21:58 xavierchee < @.***> ha scritto:

Thanks Francesco for your very prompt response!

Hello, i downloaded the code from https://github.com/jamesgleave/DD_protocol/blob/main/scripts_1/molecular_file_count_updated.py

I've redownload the code but i still get the same error message.

I wrote two scripts: Script 1

import os

Get a list of all files in current directory

files = os.listdir('/fred/oz241/xchee/[sandbox]_Deepdocki ng/phase_1/Morgan/library_prepared_fp') print("Number Of Files:", len(files))

Number of files : 100

Script 2

import glob import os

Get a list of all files in current directory

files = [f for f in glob.glob('/fred/oz241/xchee/[sandbox ]_Deepdocking/phase_1/Morgan/library_prepared_fp') if os. path.isfile(f)] print("Number Of Files:", len(files))

Number of files : 0

Thanks! It seems that for some reasons Python is not seeing those files.

— Reply to this email directly, view it on GitHub https://github.com/jamesgleave/DD_protocol/issues/8#issuecomment-1815672133, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJNAUIMFQUGGBUZX7SWYTYE3HF7AVCNFSM6AAAAAA7PCIZ2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGY3TEMJTGM . You are receiving this because you commented.Message ID: @.***>

-- Francesco Gentile

xavierchee commented 10 months ago

Hi Francesco, you are absolutely right! The [ ] does mess up glob.glob

Thanks again for the prompt response! I would have never figure it out.

fgentile89 commented 10 months ago

Glad that it worked :-)

Il giorno gio 16 nov 2023 alle ore 23:02 xavierchee < @.***> ha scritto:

Hi Francesco, you are absolutely right! The [ ] does mess up glob.glob

Thanks again for the prompt response! I would have never figure it out.

— Reply to this email directly, view it on GitHub https://github.com/jamesgleave/DD_protocol/issues/8#issuecomment-1815714039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJNAQU4MA7IWLMLGMYSWLYE3OUNAVCNFSM6AAAAAA7PCIZ2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVG4YTIMBTHE . You are receiving this because you commented.Message ID: @.***>

-- Francesco Gentile