Problem while running Tinker

[flatstik]

PyDP4.py -f mmff -w gnomesw -m t -s b3lyp --InChI Inchis.inchi experimental_nmr_files

Setting up Tinker files...
Tinker input for InChI_Mol_0_0 prepared.

Running Tinker...
/appl/soft/chem/tinker/bin/scan InChI_Mol_0_0 /appl/soft/chem/tinker/params/mmff.prm 0 10 20 0.00001 | tee ./InChI_Mol_0_0.tout
Tinker job 1 of 1 completed.
InChI_Mol_0_0 is matching conformational search output for InChI_Mol_0_0
Reading InChI_Mol_0_0
Tinker output InChI_Mol_0_0 is corrupted, aborting.

The /appl/soft/chem/tinker/bin/scan InChI_Mol_0_0 /appl/soft/chem/tinker/params/mmff.prm 0 10 20 0.00001 | tee ./InChI_Mol_0_0.tout gives following

 INITROT  --  Rotation about     6     7 occurs more than once in Z-matrix

 Tinker is Unable to Continue; Terminating the Current Calculation

[flatstik]

The problem was in the Inchi format. After I switched to --Smiles, it worked

[flatstik]

Additional:

Reading experimental NMR data...
[PosixPath('experimental_nmr_files/dada/1/Proton')]
Processing Proton Spectrum
minimisation time = 0.3785014748573303 mins
     number of protons = 13
     number of labile protons = 0
the best fit number of protons is 14
/scratch/project_2003067/DP5/DP5/Proton_processing.py:1563: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  whdel = np.where(picked_peaks == peaks_to_remove)
/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/numpy/lib/function_base.py:4454: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  arr = asarray(arr)

Assigning proton spectrum...

Plotting proton spectrum...

Calculating DP5 probabilities...
[[[]]]
multiprocess.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 81, in kde_probfunction
    scaled_shifts = ScaleNMR(conf_shifts, dp5Data.Cexp[iso])
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 317, in ScaleNMR
    slope, intercept, r_value, p_value, std_err = stats.linregress(expShifts,
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/scipy/stats/_stats_mstats_common.py", line 146, in linregress
    raise ValueError("Inputs must not be empty.")
ValueError: Inputs must not be empty.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 835, in <module>
    main(settings)
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 493, in main
    DP5data = DP5.kde_probs(Isomers, DP5data, 0.025)
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 136, in kde_probs
    dp5Data.ScaledAtomProbs[iso][ind1] = res[ind1].get()
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/multiprocess/pool.py", line 771, in get
    raise self._value
ValueError: Inputs must not be empty.

And if I do the run again, I get:

Traceback (most recent call last): File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 835, in main(settings) File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 501, in main DP5data = DP5.UnPickle_res(DP5data, settings) File "/scratch/project_2003067/DP5/DP5/DP5.py", line 400, in UnPickle_res data_dic = pickle.load(open(Path(Settings.OutputFolder) / "dp5" / "data_dic.p","rb")) FileNotFoundError: [Errno 2] No such file or directory: 'dp5/data_dic.p'

[flatstik]

I cloned the package again and now I get following for PyDP4.py -f mmff -w gnomesw -m t -s b3lyp --Smiles smiles.list experimental_nmr_files

Running Gaussian DFT geometry optimizations locally...
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp001.com > Smiles_Mol_0_0ginp001.out
Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 835, in <module>
    main(settings)
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 317, in main
    Isomers = DFT.RunOptCalcs(Isomers, settings)
  File "/scratch/project_2003067/DP5/DP5/Gaussian.py", line 254, in RunOptCalcs
    Completed = RunCalcs(GausJobs, settings)
  File "/scratch/project_2003067/DP5/DP5/Gaussian.py", line 308, in RunCalcs
    outp = subprocess.check_output(GausPrefix + " < "  + f + ' > ' + f[:-3] + 'out', shell=True)
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp001.com > Smiles_Mol_0_0ginp001.out' returned non-zero exit status 1.

Which is understandable, because only these files are generated:

Smiles_Mol_0_0.arc Smiles_Mol_0_0.key Smiles_Mol_0_0.sdf Smiles_Mol_0_0.tout Smiles_Mol_0_0.xyz Smiles_Mol0.sdf

I also get this notification:

Reading Smiles_Mol_0_0
/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/openbabel/__init__.py:14: UserWarning: "import openbabel" is deprecated, instead use "from openbabel import openbabel"
  warnings.warn('"import openbabel" is deprecated, instead use "from openbabel import openbabel"')

@HowarthA

[HowarthA]

I shall try to debug the first issue, this might be easier if you could send me your input files, the problem doesn't look to large. As for the error when you rerun the code, the first time you run the code DP5 makes an output file to store everything so its faster if you run it again. Because the calculation didn't finish properly DP5 looks for this file and doesn't find it. Delete the "dp5" folder to remove this problem.

[HowarthA]

for the second issue, does the program make an "opt" folder in your output location?

[flatstik]

I shall try to debug the first issue, this might be easier if you could send me your input files, the problem doesn't look to large. As for the error when you rerun the code, the first time you run the code DP5 makes an output file to store everything so its faster if you run it again. Because the calculation didn't finish properly DP5 looks for this file and doesn't find it. Delete the "dp5" folder to remove this problem.

I removed everything, still the same issue for "PyDP4.py -f mmff -w gnomesw -m t -s b3lyp --Smiles smiles.list experimental_nmr_files_path_here"

examples.tar.gz

for the second issue, does the program make an "opt" folder in your output location?

Yes, and there are the following files: Smiles_Mol_0_0ginp001.chk Smiles_Mol_0_0ginp001.com Smiles_Mol_0_0ginp001.out

[HowarthA]

hmm, interesting thank you. I'll finally have a look at this now, sorry its taken a while.

[flatstik]

hmm, interesting thank you. I'll finally have a look at this now, sorry its taken a while.

Ah, there's problem with the smiles.list.. I accidentally tarred everything in that. but the contents of it should be

NCCc1ccccc1
CC1NCCc2ccccc12

[flatstik]

Interestingly, if I run it without "-s b3lyp" It can find the files and runs normally

[HowarthA]

the -s command is to specify the solvent, like "-s chloroform" run PyDP4.py -h for more help with the input switches. I haven't tested this fully, but I have a feeling the issue is with how the NMR files are arranged. The NMR folder should be arranged as below and the proton and carbon folders should be named this way.

-NMRfolder

• \Carbon -raw/loose files
• \Proton -raw/loose files

I have done this in this example

DP5_example.zip .

[flatstik]

Yes, I wanted to use DMSO as the solvent. Anyway, now that I ran it without solvent, I get this:

Converting DFT data to NMR shifts...
WARNING: NMR shift calculation currently ignores the instruction to exclude atoms from analysis
C shifts for isomer 0: 
24.858, 54.931, 40.471, 33.342, 141.095, 133.305, 130.040, 129.535, 131.731, 145.845
H shifts for isomer 0: 
1.785, 0.860, 1.321, 4.351, 0.659, 2.813, 3.358, 2.535, 3.092, 7.174, 7.240, 7.237, 7.109

Reading experimental NMR data...
[PosixPath('experimental_nmr_files/Proton/format.temp')]
Loading NMR data from [PosixPath('experimental_nmr_files/Proton/format.temp')]
['EDIT_PAR COMMAND FILE']
Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 835, in <module>
    main(settings)
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 397, in main
    NMRData = NMR.NMRData(settings)
  File "/scratch/project_2003067/DP5/DP5/NMR.py", line 93, in __init__
    self.ExpNMRFromDesc()
  File "/scratch/project_2003067/DP5/DP5/NMR.py", line 125, in ExpNMRFromDesc
    self.Clabels, self.Cshifts = self.ParseExp(Cexp)
  File "/scratch/project_2003067/DP5/DP5/NMR.py", line 150, in ParseExp
    expShifts = [float(x) for x in ShiftData]
  File "/scratch/project_2003067/DP5/DP5/NMR.py", line 150, in <listcomp>
    expShifts = [float(x) for x in ShiftData]
ValueError: could not convert string to float: 'EDIT_PAR COMMAND FILE'

it looks the "format.temp" file containing "EDIT_PAR COMMAND FILE' which is in the nmr data...

[flatstik]

I have done this in this example

DP5_example.zip .

I tried to run the examples. It yielded this error:

PyDP4.py -f mmff -w gnomesw -s dimethylsulfoxide -m t --Smiles huh/DP5_example/smiles.list.txt huh/DP5_example/NMR_folder/
==========================
PyDP4 script,
integrating Tinker/MacroModel,
Gaussian/NWChem and DP4
v1.0

Copyright (c) 2015-2019 Kristaps Ermanis, Alexander Howarth, Jonathan M. Goodman
Distributed under MIT license
==========================

Settings read from settings.cfg:
  SCHRODINGER: /appl/soft/chem/schrodinger/2021.3
  TinkerPath: /appl/soft/chem/tinker
  NWChemPath: /appl/soft/chem/nwchem/6.8.1/bin/nwchem
  GausPath: /appl/soft/chem/gaussian/G16RevC.01_new/g16/g16
[]
huh/DP5_example/NMR_folder/
NMR_path
Current working directory: /scratch/project_2003067/DP5/DP5_example
Initial input files: []
NMR file: [PosixPath('huh/DP5_example/NMR_folder/Proton')]
Workflow: gnomesw

Generating diastereomers...
Generated input files: ['Smiles_Mol_0_0', 'Smiles_Mol_1_0']

Assuming all computations are done? ...  False
Using preexisting DFT data? ...  False
Tinker

Setting up Tinker files...
Tinker input for Smiles_Mol_0_0 prepared.
Tinker input for Smiles_Mol_1_0 prepared.

Running Tinker...
Output files for Smiles_Mol_0_0 already exist, skipping.
Output files for Smiles_Mol_1_0 already exist, skipping.
Smiles_Mol_0_0 is matching conformational search output for Smiles_Mol_0_0
Reading Smiles_Mol_0_0
/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/openbabel/__init__.py:14: UserWarning: "import openbabel" is deprecated, instead use "from openbabel import openbabel"
  warnings.warn('"import openbabel" is deprecated, instead use "from openbabel import openbabel"')
Number of accepted conformers by energies: 5
Smiles_Mol_1_0 Smiles_Mol_0_0 Nope
Smiles_Mol_0_0 Smiles_Mol_1_0 Nope
Smiles_Mol_1_0 is matching conformational search output for Smiles_Mol_1_0
Reading Smiles_Mol_1_0
Number of accepted conformers by energies: 1

Pruning conformers...
Smiles_Mol_0_0: 5 conformers after pruning with 0A RMSD cutoff
Smiles_Mol_1_0: 1 conformers after pruning with 0A RMSD cutoff

Setting up geometry optimization calculations...

Running geometry optimization calculations...

Running Gaussian DFT geometry optimizations locally...
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp001.com > Smiles_Mol_0_0ginp001.out
Gaussian job 1 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp002.com > Smiles_Mol_0_0ginp002.out
Gaussian job 2 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp003.com > Smiles_Mol_0_0ginp003.out
Gaussian job 3 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp004.com > Smiles_Mol_0_0ginp004.out
Gaussian job 4 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp005.com > Smiles_Mol_0_0ginp005.out
Gaussian job 5 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_1_0ginp001.com > Smiles_Mol_1_0ginp001.out
Gaussian job 6 of 6 completed.
6 Gaussian jobs completed successfully.

Reading DFT optimized geometries...

Setting up energy calculations...

Running energy calculations...

Running Gaussian DFT energy calculations locally...
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp001.com > Smiles_Mol_0_0ginp001.out
Gaussian job 1 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp002.com > Smiles_Mol_0_0ginp002.out
Gaussian job 2 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp003.com > Smiles_Mol_0_0ginp003.out
Gaussian job 3 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp004.com > Smiles_Mol_0_0ginp004.out
Gaussian job 4 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp005.com > Smiles_Mol_0_0ginp005.out
Gaussian job 5 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_1_0ginp001.com > Smiles_Mol_1_0ginp001.out
Gaussian job 6 of 6 completed.
6 Gaussian jobs completed successfully.

Reading data from the output files...
Energies: 
Smiles_Mol_0_0: [-366.197214745, -366.197883219, -366.196659612, -366.197404289, -366.19613879]
Smiles_Mol_1_0: [-443.612476661]

Setting up NMR calculations...

Running NMR calculations...

Running Gaussian DFT NMR calculations locally...
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp001.com > Smiles_Mol_0_0ginp001.out
Gaussian job 1 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp002.com > Smiles_Mol_0_0ginp002.out
Gaussian job 2 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp003.com > Smiles_Mol_0_0ginp003.out
Gaussian job 3 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp004.com > Smiles_Mol_0_0ginp004.out
Gaussian job 4 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_0_0ginp005.com > Smiles_Mol_0_0ginp005.out
Gaussian job 5 of 6 completed.
/appl/soft/chem/gaussian/G16RevC.01_new/g16/g16 < Smiles_Mol_1_0ginp001.com > Smiles_Mol_1_0ginp001.out
Gaussian job 6 of 6 completed.
6 Gaussian jobs completed successfully.

Reading data from the output files...
Smiles_Mol_0_0ginp001.out 20
Smiles_Mol_0_0ginp002.out 20
Smiles_Mol_0_0ginp003.out 20
Smiles_Mol_0_0ginp004.out 20
Smiles_Mol_0_0ginp005.out 20
Smiles_Mol_1_0ginp001.out 24
Shieldings: 
Smiles_Mol_0_0: 
[227.7226, 135.9834, 140.6408, 41.2866, 54.6, 54.9249, 57.5859, 54.9249, 54.6, 31.4904, 31.4904, 29.3302, 29.3302, 29.3618, 29.3618, 24.6724, 24.5632, 24.6648, 24.5632, 24.6724]
[236.8067, 139.0096, 143.4574, 40.724, 53.3368, 55.6523, 57.3108, 54.3826, 54.1919, 31.6266, 32.0611, 29.5183, 29.048, 29.617, 29.175, 24.7129, 24.5725, 24.6095, 24.4877, 24.6373]
[230.6835, 136.5512, 145.3966, 40.472, 54.4241, 55.0944, 57.4607, 54.9556, 54.1836, 31.196, 31.6251, 29.0264, 29.4506, 29.4326, 29.2383, 24.6668, 24.5711, 24.6477, 24.5499, 24.5984]
[235.4631, 139.9976, 146.5413, 40.332, 53.2518, 55.5416, 57.2077, 54.4654, 54.4438, 31.5766, 32.0352, 29.0501, 29.2034, 29.3643, 29.154, 24.6409, 24.5673, 24.6145, 24.4943, 24.6262]
[235.1279, 138.5802, 146.8167, 38.6611, 52.2061, 56.2759, 57.8835, 55.1551, 54.8817, 31.9147, 31.5932, 29.4559, 28.6527, 29.5531, 29.2537, 24.6563, 24.5989, 24.6428, 24.5187, 24.6201]
Smiles_Mol_1_0: 
[162.8952, 132.93, 212.5706, 147.4285, 156.4572, 46.289, 53.9347, 57.7416, 57.9845, 56.2357, 40.3121, 30.7673, 30.7988, 30.5589, 27.9688, 31.0354, 29.0517, 28.9329, 29.3373, 29.4602, 24.7993, 24.7425, 24.7405, 24.8413]
Energies: 
Smiles_Mol_0_0: [-366.197214745, -366.197883219, -366.196659612, -366.197404289, -366.19613879]
Smiles_Mol_1_0: [-443.612476661]

Setting TMS computational NMR shielding constant references
Setting TMS references to 188.622225 and 32.1198166667

Converting DFT data to NMR shifts...
WARNING: NMR shift calculation currently ignores the instruction to exclude atoms from analysis
C shifts for isomer 0: 
50.270, 44.568, 148.066, 135.038, 133.185, 131.255, 134.029, 134.274
H shifts for isomer 0: 
0.560, 0.253, 2.807, 2.961, 2.637, 2.902, 7.441, 7.549, 7.492, 7.607, 7.484
C shifts for isomer 1: 
25.732, 55.703, 41.201, 32.171, 142.360, 134.713, 130.905, 130.662, 132.412, 148.338
H shifts for isomer 1: 
1.353, 1.321, 1.561, 4.151, 1.084, 3.068, 3.187, 2.783, 2.660, 7.321, 7.378, 7.380, 7.279

Reading experimental NMR data...
[PosixPath('huh/DP5_example/NMR_folder/Proton')]
Processing Proton Spectrum
minimisation time = 0.2390958070755005 mins
     number of protons = 11
==============================
*** Open Babel Error  in ReadFile
  Cannot read from Smiles_Mol_0_0
     number of labile protons = 0
the best fit number of protons is 14
/scratch/project_2003067/DP5/DP5/Proton_processing.py:1563: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  whdel = np.where(picked_peaks == peaks_to_remove)
/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/numpy/lib/function_base.py:4454: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  arr = asarray(arr)

Assigning proton spectrum...

Plotting proton spectrum...

Calculating DP5 probabilities...
[[[], [], [], [], []], [[]]]
multiprocess.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 81, in kde_probfunction
    scaled_shifts = ScaleNMR(conf_shifts, dp5Data.Cexp[iso])
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 317, in ScaleNMR
    slope, intercept, r_value, p_value, std_err = stats.linregress(expShifts,
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/scipy/stats/_stats_mstats_common.py", line 146, in linregress
    raise ValueError("Inputs must not be empty.")
ValueError: Inputs must not be empty.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 835, in <module>
    main(settings)
  File "/scratch/project_2003067/DP5/DP5/PyDP4.py", line 493, in main
    DP5data = DP5.kde_probs(Isomers, DP5data, 0.025)
  File "/scratch/project_2003067/DP5/DP5/DP5.py", line 136, in kde_probs
    dp5Data.ScaledAtomProbs[iso][ind1] = res[ind1].get()
  File "/scratch/project_2003067/DP5/DP5_env/lib/python3.9/site-packages/multiprocess/pool.py", line 771, in get
    raise self._value
ValueError: Inputs must not be empty.

[HowarthA]

This is really strange, its clearly very nearly there. Could you send me a copy of the example with the completed calculations? This will make it quicker to debug

[flatstik]

This is really strange, its clearly very nearly there. Could you send me a copy of the example with the completed calculations? This will make it quicker to debug

You can download the example.tar.gz at https://mega.nz/file/7U92FCLT#5lCzql1e2rfkIIIQKjedhEbasjzfqZbUeukm1Z0qH9Q

[HowarthA]

I should have noticed this before, this is not a bug in the software. As stated in the paper, DP5 runs with carbon data only. There is no carbon data in your example file, so DP5 has nothing to calculate, this is why the inputs to the multiprocessing function are empty. Try another example with carbon data and see if this works.