Refmac5 does not produce output

Hi,

I have tried to use servalcat on one of our ubuntu workstations (20.04) and run into an issue with refmac. Firstly, I installed the stand-alone ccp4 package and sourced the bash-script. Then, I created a conda environment to install numpy and pandas with conda create -n servalcat -c conda-forge numpy pandas. To install servalcat, I activated the conda environment and installed with pip install servalcat.

I tried running a refine_spa job with two half-maps, a binary mask and my model.pdb. I added hydrogens to the model using phenix.reduce. The first issue I faced was that some hydrogens were not accepted by servalcat, so I tried running it with a model lacking hydrogens, to see if it would work. With this, the program seems to run normally, until it launches refmac5. Here is a relevant snippet from the log:

Running REFMAC5.. refmac5 hklin masked_fs_obs.mtz hklout shifted_TEST.mtz xyzin shifted.pdb xyzout shifted_TEST.pdb libin merged_ligands.cif <<__eof__ > shifted_TEST.log labin FP=Fout PHIB=Pout make hydr all make hout no solvent no scale lssc isot source em mb ncycle 10 reso 2.05 weight auto 2.00e-01 ncsr local pdbo keep auth eof Starting Refmac 5.8.0267 (PID: 2200424) Fortran runtime error: Bad value during integer read Backtrace for this error: REFMAC5 finished with exit code= 2 Traceback (most recent call last): File "/home/XXX/Software/miniconda3/envs/servalcat/bin/servalcat", line 8, in sys.exit(main()) File "/home/XXX/Software/miniconda3/envs/servalcat/lib/python3.10/site-packages/servalcat/command_line.py", line 111, in main modules[args.command].main(args) File "/home/XXX/Software/miniconda3/envs/servalcat/lib/python3.10/site-packages/servalcat/spa/run_refmac.py", line 313, in main refmac_summary = refmac.run_refmac() File "/home/XXX/Software/miniconda3/envs/servalcat/lib/python3.10/site-packages/servalcat/utils/refmac.py", line 359, in run_refmac raise RuntimeError("REFMAC5 did not produce output files.") RuntimeError: REFMAC5 did not produce output files.

Can you tell me, how to solve this issue? Thank you in advance!

Thanks for your feedback, and sorry for the inconvenience!

I added hydrogens to the model using phenix.reduce. The first issue I faced was that some hydrogens were not accepted by servalcat,

Probably hydrogen atom names by phenix.reduce were incompatible with those in CCP4 monomer library. However, if you just want Refmac to consider hydrogen atoms during the refinement, it is done by default (this is the default behaviour of Refmac, where hydrogen atoms are internally added and removed before output).

With this, the program seems to run normally, until it launches refmac5. Here is a relevant snippet from the log:

I need to look at the Refmac log file. Could you paste some last lines of shifted_TEST.log?

Could you paste some last lines of shifted_TEST.log?

Here you go:

Input file :shifted.pdb

--- LIBRARY OF MONOMERS --- _lib_name mon_lib _lib_version 5.52 _lib_update 17/08/20

NUMBER OF MONOMERS IN THE LIBRARY : 30268 with complete description : 30268 NUMBER OF MODIFICATIONS : 117 NUMBER OF LINKS : 121 I am reading libraries. Please wait.

energy parameters

monomer"s description (links & mod ) At line 205 of file /home/jenkins/workspace/CCP4/sl6_devtoolset3/devtools/checkout/refmac5.8/make_PDB.f Fortran runtime error: Bad value during integer read

Backtrace for this error:

/home/XXX/Software/ccp4-7.1/bin/../lib/libgfortran.so.3(+0x13f89) [0x7fb99dcacf89]

/home/XXX/Software/ccp4-7.1/bin/../lib/libgfortran.so.3(+0x1582e) [0x7fb99dcae82e]

/home/XXX/Software/ccp4-7.1/bin/../lib/libgfortran.so.3(_gfortran_generate_error+0x121) [0x7fb99dcaea81]

/homeXXX/Software/ccp4-7.1/bin/../lib/libgfortran.so.3(+0xbbb54) [0x7fb99dd54b54]

/home/XXX/Software/ccp4-7.1/bin/../lib/libgfortran.so.3(+0xbf397) [0x7fb99dd58397]

refmac5() [0x72ede3]

refmac5() [0x7347db]

refmac5() [0x530fa5]

refmac5() [0x532996]

refmac5() [0x47972f]

refmac5() [0x40537d]

/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7fb99d9640b3]

refmac5() [0x405ae5]

Thanks. So there is something wrong with the model file (or more likely with Refmac5). Can we see your shifted.pdb? If it is unpublished data, please send it to us (email address may be found in our paper) and we will keep it confidential.

The Model is PDB 7K00. It was fetched in ChimeraX and aligned to my map, then saved as .cif. I opened this in coot and saved it from there as .pdb to add the hydrogens (as mentioned above this is the version from coot where no hydrogens were added, yet).

shifted.pdb.zip

OK, this PDB file contains 2-letter chain IDs that are not supported by Refmac5. I will fix Servalcat to give cif file to Refmac if there are 2-letter chain IDs. In the meantime, can you try to give a cif file to Servalcat (with --model). Then cif files are used internally.

Thanks for the help! It is running now, I assume this will take a while?

Yes it would, as this is a relatively huge complex. Please let me know if you have any further problems.

Well it stopped again, due to missing ligand restraints. I will try tomorrow and see if I can resolve this, else I can always mutate to the parental residue. But this seems to be not due to servalcat, so thanks again for the help! Here is the message, just in case:

ERROR : atom :P 3TD 1915 a is absent in the library ERROR : atom :N1 3TD 1915 a is absent in the library ERROR : atom :C2 3TD 1915 a is absent in the library ERROR : atom :O2 3TD 1915 a is absent in the library ERROR : atom :N3 3TD 1915 a is absent in the library ERROR : atom :C4 3TD 1915 a is absent in the library ERROR : atom :O4 3TD 1915 a is absent in the library ERROR : atom :C5 3TD 1915 a is absent in the library ERROR : atom :C6 3TD 1915 a is absent in the library ERROR : atom :C1' 3TD 1915 a is absent in the library ERROR : atom :C10 3TD 1915 a is absent in the library ERROR : atom :C2' 3TD 1915 a is absent in the library ERROR : atom :O2' 3TD 1915 a is absent in the library ERROR : atom :C3' 3TD 1915 a is absent in the library ERROR : atom :O3' 3TD 1915 a is absent in the library ERROR : atom :C4' 3TD 1915 a is absent in the library ERROR : atom :O4' 3TD 1915 a is absent in the library ERROR : atom :C5' 3TD 1915 a is absent in the library ERROR : atom :O5' 3TD 1915 a is absent in the library ERROR : atom :OP1 3TD 1915 a is absent in the library ... and more ... WARNING : 3TD a 1915 a : first atom of the tree is absent WARNING : 3TD a 1915 a : last atom of the tree is absent Number of chains : 56 Total number of monomers : 17556 Number of atoms : 254373 Number of missing atoms : 0 Number of rebuilt atoms : 93962 Number of unknown atoms : 0 Number of deleted atoms : 0 I am writing new library: /tmp/XXX/refmac5_temp1.15127_lib.cif

Important, Important, Important!!!!!

Your coordinate file has a ligand which has either minimum or no description in the library A new ligand description has been added to /tmp/XXX/refmac5_temp1.15127_lib.cif Picture of the new ligand can be viewed using postscript file. See above Check description in this file and, if satisfied, use it as the input library Otherwise either edit bond orders manually or use JLigand to view and edit the ligand and create a library entry by running acedrg It is strongly recommended that dictionary entry should be checked carefully before using it If you are happy with the library description then use the keyword (MAKE CHECK NONE) I.e. do not check correctness of the coordinates ===> Error: New ligand has been encountered. Stopping now Refmac: New ligand has been encountered. Stopping now Refmac: New ligand has been encountered. Stopping now Times: User: 386.0s System: 0.4s Elapsed: 6:27

This is strange.. as Servalcat checks if all atoms in the model are defined by the CCP4 monomer library and user-supplied definitions. It seems you gave cif files with --ligand, but they did not include 3TD? If included, how did you prepare the file?

I downloaded the .cif files for 3TD, 4D4, D2T and MS6 from PDB Ligand Expo (http://ligand-expo.rcsb.org/ld-search.html) and dumped them in a folder called "cif", then used the parameter --ligand cif/* The file created by refmac "refmac5_temp1.15127_lib.cif" looks like MS6 but with one double-bonded Sulfur atom when I open it in the coot ligand builder. Will check, if that is off in the model file as well.

Oh those cif files provided by PDB only include atom names and ideal geometry, so are not ligand definitions (dictionaries) that can be used in refinement. You need to create a dictionary e.g. (as a CCP4 solution)

acedrg -c 3TD.cif -o acedrg_3TD

and give acedrg_3TD.cif to --ligand. You have many cif files, so you can do this at once in your cif directory

for f in *.cif
do
 acedrg -c $f -o acedrg_${f/.cif/}
done

and give --ligand cif/acedrg* to servalcat.

I also need to fix Servalcat to raise error when such cif files are provided. Thanks.

Thank you a lot for the explanation, indeed I am still learning how to deal with restraints correctly! At least it was useful in improving your software ;)

Definitely. I appreciate your feedback!

One more thing: Is it refmac supposed to run only on one core, or can it be parallelized? At the moment it is only using one cpu for me.

Yes, it only uses a single CPU core.

Sorry I gave a wrong explanation. The problem is not in chain IDs (actually they are all single-letter) but in non-integer residue number.

Now Servalcat (ver. 0.2.28)

switches to mmcif if max residue number > 9999
fails if given ligand cif file does not contain restraints

keitaroyam / servalcat

Refmac5 does not produce output #3

Input file :shifted.pdb

--- LIBRARY OF MONOMERS --- _lib_name mon_lib _lib_version 5.52 _lib_update 17/08/20