VuisterLab / cing

Automated Validation of NMR Structures
http://nmr.le.ac.uk
2 stars 4 forks source link

FC crashed on reading complex #281

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. Look at:
http://nmr.cmbi.ru.nl/NRG-CING/data/ki/2kid/log_validateEntry/2kid_2011-02-04_05
-28-22.log
repeated below.

Note that even though this entry has CS they were not yet added. The input is 
straight up DOCR CCPN XML.

Might be nice to fix this problem while we are waiting for sf.net to come back 
up.

================================================================================
======================
| CING: Common Interface for NMR structure Generation version 0.95 (r921)       
AW,JFD,GWV 2004-2011 |
================================================================================
======================
User: jd         on: nmr.cmbi.umcn.nl (darwin/32bit/8cores/2.6.5) at:  (8780) 
Fri Feb  4 05:29:17 2011
DEBUG: Using:
DEBUG: inputDir:             
file:///Library/WebServer/Documents/NRG-CING/input/ki
DEBUG: outputDir:            /Library/WebServer/Documents/NRG-CING/data/ki/2kid
DEBUG: pdbConvention:        .
DEBUG: restraintsConvention: .
DEBUG: archiveType:          BY_CH23
DEBUG: projectType:          CCPN
DEBUG: modelCount:           None
DEBUG: storeCING2db:         True
DEBUG: ranges:               cv
Enforcing a redo
copying file: //Library/WebServer/Documents/NRG-CING/input/ki/2kid.tgz to: 
2kid.tgz
Moving CCPN directory from [linkNmrStarData] to [2kid]
==> Importing data from Ccpn project "2kid"
Error loading file for:  <ccp.molecule.ChemComp.NonStdChemComp ['protein', 
'Phq']>
Reading:  <open file 
'/Volumes/tera4/NRG-CING/data/ki/2kid/2kid/ccp/molecule/ChemComp/protein+Phq+2ki
d_user_2010-12-08-13-35-37-016_00001.xml', mode 'r' at 0x7fab128>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Error loading file for:  <ccp.molecule.MolSystem.MolSystem ['2kid']>
Reading:  <open file 
'/Volumes/tera4/NRG-CING/data/ki/2kid/2kid/ccp/molecule/MolSystem/2kid+2kid_user
_2010-12-08-13-34-39-344_00002.xml', mode 'r' at 0x7732390>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Error loading file for:  <ccp.nmr.NmrConstraint.NmrConstraintStore [1]>
Reading:  <open file 
'/Volumes/tera4/NRG-CING/data/ki/2kid/2kid/ccp/nmr/NmrConstraint/1+2kid_user_201
0-12-08-13-34-39-371_00001.xml', mode 'r' at 0x6500f40>
Last xml tag read:  NMRC.FixedAtomSet.atoms
Parser state was:  reading
Current object was:  <ccp.nmr.NmrConstraint.FixedAtomSet [1, 1]>
CING started at : Fri Feb  4 05:29:14 2011
CING stopped at : Fri Feb  4 05:29:24 2011
CING took       : 9.995 s

Traceback (most recent call last):
  File "/Users/jd/workspace35/cingStable/python/cing/Scripts/validateEntry.py", line 355, in <module>
    status = main(*sys.argv[1:])
  File "/Users/jd/workspace35/cingStable/python/cing/Scripts/validateEntry.py", line 183, in main
    if not project.initCcpn(ccpnFolder=fileNameTgz, modelCount=modelCount):
  File "/Users/jd/workspace35/cingStable/python/cing/PluginCode/Ccpn.py", line 2341, in initCcpn
    if not ccpn.importFromCcpn(modelCount = modelCount):
  File "/Users/jd/workspace35/cingStable/python/cing/PluginCode/Ccpn.py", line 317, in importFromCcpn
    if not self.importFromCcpnMolecule(modelCount = modelCount):
  File "/Users/jd/workspace35/cingStable/python/cing/PluginCode/Ccpn.py", line 412, in importFromCcpnMolecule
    ccpnConstraintLists.extend(constraintStore.sortedConstraintLists())
  File "/Users/jd/workspace35/ccpn/python/ccp/api/nmr/NmrConstraint.py", line 22236, in sortedConstraintLists
    self.load()
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5030, in load
    self.loadFrom(repositories[0])
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5064, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 267, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace35/ccpn/python/memops/xml/Implementation.py", line 4912, in loadFromStream
    value = clazz.getByKey(oo,vv[1:-1])
  File "/Users/jd/workspace35/ccpn/python/ccp/api/molecule/MolSystem.py", line 1583, in getByKey
    obj1.load()
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5030, in load
    self.loadFrom(repositories[0])
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5064, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 273, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace35/ccpn/python/memops/xml/Implementation.py", line 4993, in loadFromStream
    obj.checkValid()
  File "/Users/jd/workspace35/ccpn/python/ccp/api/molecule/MolSystem.py", line 20447, in checkValid
    ('chemCompVars', (self.linking,  self.descriptor))
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 3974, in getByNavigation
    result.load()
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5030, in load
    self.loadFrom(repositories[0])
  File "/Users/jd/workspace35/ccpn/python/memops/api/Implementation.py", line 5064, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/Users/jd/workspace35/ccpn/python/memops/format/xml/XmlIO.py", line 273, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace35/ccpn/python/memops/xml/Implementation.py", line 4993, in loadFromStream
    obj.checkValid()
  File "/Users/jd/workspace35/ccpn/python/ccp/api/molecule/ChemComp.py", line 26753, in checkValid
    + ": %s" % (self,)
memops.general.Implementation.ApiError: 
ccp.molecule.ChemComp.NonStdChemComp.checkValid:
         constraint protein_DNA_RNA_are_linear_polymers violated: <ccp.molecule.ChemComp.NonStdChemComp ['protein', 'Phq']>

Original issue reported on code.google.com by jurge...@gmail.com on 7 Feb 2011 at 9:55

GoogleCodeExporter commented 9 years ago
Not sure about this one, it's a data model issue - can you send the .tgz of the 
project to Rasmus with the error message? Better person for this one... .

Original comment by wfvran...@gmail.com on 7 Feb 2011 at 10:48

GoogleCodeExporter commented 9 years ago
Rasmus, could you take a look at the attached straight up DOCR CCPN XML.
CING/FC has a problem with it as shown above but also Analysis barfs in the 
same way as per below.

Rasmus, do you want to sign up with Google Code?

jd:stella/cingTmp/ analysis linkNmrStarData/
Model read finished. Duration 2.21501994133 
Model validity check skipped
CcpNmr Analysis Version 2.1. Release 5 (Copyright 2003-2010 CCPN)
Distribution created Wed Aug 11 17:37:18 2010

If you use CcpNmr Analysis, please quote the following reference:

Wim F. Vranken, Wayne Boucher, Tim J. Stevens, Rasmus H. Fogh, Anne 
Pajon, Miguel Llinas, Eldon L. Ulrich, John L. Markley, John Ionides
and Ernest D. Laue (2005). The CCPN Data Model for NMR Spectroscopy: 
Development of a Software Pipeline. Proteins 59, 687 - 696.

A copy of the in-program documentation is located at:
http://www.ccpn.ac.uk/documentation

For how-tos and FAQs visit the CCPN wiki site at:
http://www.ccpn.ac.uk/wiki

To report bugs use the bug tracker at:
http://www.ccpn.ac.uk/bugs

For help and support join the mailing list at:
http://www.ccpn.ac.uk/support

For Python API documentation, tutorials and CCPN web applications visit:
http://www.ccpn.ac.uk/

Error loading file for:  <ccp.molecule.ChemComp.NonStdChemComp ['protein', 
'Phq']>
Reading:  <open file 
'/Users/jd/tmp/cingTmp/linkNmrStarData/ccp/molecule/ChemComp/protein+Phq+2kid_us
er_2010-12-08-13-35-37-016_00001.xml', mode 'r' at 0x28525a0>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Error loading file for:  <ccp.molecule.MolSystem.MolSystem ['2kid']>
Reading:  <open file 
'/Users/jd/tmp/cingTmp/linkNmrStarData/ccp/molecule/MolSystem/2kid+2kid_user_201
0-12-08-13-34-39-344_00002.xml', mode 'r' at 0x28523e8>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Traceback (most recent call last):
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccpnmr/analysis/AnalysisGui.py", line 226, in <module>
    main(projectDir, max_size, glDirect)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccpnmr/analysis/AnalysisGui.py", line 111, in main
    top.initProject(project)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccpnmr/analysis/AnalysisPopup.py", line 1516, in initProject
    Analysis.initProject(self, project)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccpnmr/analysis/Analysis.py", line 303, in initProject
    self.initAtomSetMappings()
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccpnmr/analysis/Analysis.py", line 976, in initAtomSetMappings
    for chain in molSystem.chains:
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccp/api/molecule/MolSystem.py", line 13109, in getChains
    self.load()
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/api/Implementation.py", line 5030, in load
    self.loadFrom(repositories[0])
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/api/Implementation.py", line 5064, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/format/xml/XmlIO.py", line 267, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/xml/Implementation.py", line 4940, in loadFromStream
    obj.checkValid()
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccp/api/molecule/MolSystem.py", line 20447, in checkValid
    ('chemCompVars', (self.linking,  self.descriptor))
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/api/Implementation.py", line 3974, in getByNavigation
    result.load()
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/api/Implementation.py", line 5030, in load
    self.loadFrom(repositories[0])
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/api/Implementation.py", line 5064, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/format/xml/XmlIO.py", line 273, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/memops/xml/Implementation.py", line 4940, in loadFromStream
    obj.checkValid()
  File "/Users/jd/workspace35/ccpnmr/ccpnmr2.1/python/ccp/api/molecule/ChemComp.py", line 26753, in checkValid
    + ": %s" % (self,)
memops.general.Implementation.ApiError: 
ccp.molecule.ChemComp.NonStdChemComp.checkValid:
         constraint protein_DNA_RNA_are_linear_polymers violated: <ccp.molecule.ChemComp.NonStdChemComp ['protein', 'Phq']>

Original comment by jurge...@gmail.com on 7 Feb 2011 at 11:12

Attachments:

GoogleCodeExporter commented 9 years ago
Actually this is a ChemComp issue, once you have done a bit of digging.
The project contains a local ChemComp definition for ChemComp Phq that is 
incorrect and breaks the model constraints. The error comes because the 
ChemComp is defined as molType='protein'; the code quite rightly notices that 
this is inconsistent. There are more problems with the file - for instance it 
has no ChemBond records,and so does not define a molecule but only a bag of 
atoms. I did not look further than that.

The solution is to use a correct ChemComp. Phq is available from the normal 
ChemComp server, you can get it in Analysis by going into M:Molecule:Molecules 
{Small Compounds} and selecting Phq under Mol Type: All. Assuming it is the 
same molecule (it looks like it) this is benzyl chlorocarbonate. 

The most curious thing to my mind is how the incorrect ChemComp got there in 
the first place. According to the guid it was created on December 8 2010 at 
13:35:37 as part of the 2kid project. Whatever way this ChemComp file was 
created should be checked so that 1) we do not create new ChemComps (with 
clashing names and distinct globally unique IDs) for ChemComps we already have 
on the server, and 2) any ChemComps we do create are correct.

I would be interested to hear more, but I am not the right person to 
investigate further. Please let me know.

Rasmus

Original comment by r.h.f...@gmail.com on 8 Feb 2011 at 11:10

GoogleCodeExporter commented 9 years ago
So this is an 'autogenerated' chemComp because the Phq one was not found or 
downloaded. I just tried this project on my computer and it works fine, also 
reading everything back in. Can you check the original project creation log 
file, Jurgen?

It should have gotten the Phq chemComp online (or from the local list of 
chemComps if that's being kept up to date). Works for me in any case.

Original comment by wfvran...@gmail.com on 8 Feb 2011 at 11:59

GoogleCodeExporter commented 9 years ago
ok, I'll test with my development NRG setup in Nijmegen to recreate and read 
back in.
If so, I'll ask Chris at BMRB to do the same.

Thanks guys!

Original comment by jurge...@gmail.com on 8 Feb 2011 at 12:43

GoogleCodeExporter commented 9 years ago
My development setup on nmr.cmbi.ru.nl fails in processing the NRG data.
I'll hold off until I can update CCPN or should I modify something in my setup 
already Wim?
I did check that I'm using the same presets as Chris S.:

'2kid': {

  'authors': ['chris schulte'],
'comment': """
  Warning: Did not map CCPN chain codes ['B']
  Final mapping: [['A', ' ', 2, 0], ['C', ' ', 1, 148]]
""",

  'linkResonances': {

   'keywds': {

      'forceChainMappings': [['A', ' ', 2, 0], ['B', ' ', 1, 699], ['C', ' ', 1, 701]]

      },
    },
  },

Log:

Model read finished. Duration 1.71581101418
Model validity check skipped
Read pdbe.adatah.localConstants.py version 4.5.6
Read recoord2.pdbe.localConstants.py version 1.2.3
length of presetDict 1387
Read recoord2.pdbe.localConstants.py version 1.2.3
Doing 2kid...
  Using preset values...
    >
    >   Warning: Did not map CCPN chain codes ['B']
    >   Final mapping: [['A', ' ', 2, 0], ['C', ' ', 1, 148]]
    >
Traceback (most recent call last):
  File "/Users/jd/workspace35/recoord/python/recoord2/pdbe/linkNmrStarData.py", line 707, in <module>
    LinkNmrStarData(sys.argv)
  File "/Users/jd/workspace35/ccpn/python/pdbe/adatah/Generic.py", line 89, in __init__
    self.catchError(raiseError,timeFlag)
  File "/Users/jd/workspace35/ccpn/python/pdbe/adatah/Generic.py", line 82, in __init__
    self.runSpecific()
  File "/Users/jd/workspace35/recoord/python/recoord2/pdbe/linkNmrStarData.py", line 169, in runSpecific
    self.readNmrStarFile(inNmrStarFile, version = self.originalNmrStarVersion, maxNum = self.numModelsToRead, autoCreateChemComps = True)
  File "/Users/jd/workspace35/ccpn/python/pdbe/adatah/NmrStar.py", line 46, in readNmrStarFile
    **localKeywds)
  File "/Users/jd/workspace35/ccpn/python/ccpnmr/format/converters/DataFormat.py", line 951, in readProject
    self.getFullProject(fileName,**keywds)
  File "/Users/jd/workspace35/ccpn/python/ccpnmr/format/converters/NmrStarFormat.py", line 566, in getFullProject
    chains = self.readSequence(molSystem = molSystem, minimalPrompts = self.minimalPrompts, version = self.version, **localKeywds)
  File "/Users/jd/workspace35/ccpn/python/ccpnmr/format/converters/DataFormat.py", line 1071, in readSequence
    (createMoleculeDict,molSystemBonds) = self.checkMolecules()
  File "/Users/jd/workspace35/ccpn/python/ccpnmr/format/converters/DataFormat.py", line 6035, in checkMolecules
    self.makeChemComp()
  File "/Users/jd/workspace35/ccpn/python/ccpnmr/format/converters/DataFormat.py", line 14308, in makeChemComp
    raise self.FormatConverterError, "  Error: no link code %s, %s" % (linkCode,descriptor)
ccpnmr.format.converters.DataFormat.FormatConverterError: '  Error: no link 
code next, neutral'

Original comment by jurge...@gmail.com on 10 Feb 2011 at 9:24

GoogleCodeExporter commented 9 years ago
Well it works for me... Chris also has problems with the chemComp part, was 
never able to figure out why. You have them in 
data/pdbe/chemComp/archive/ChemComp of the CCPN installation? Or are they 
checked out elsewhere and do you refer the FC to that directory?

Original comment by wfvran...@gmail.com on 10 Feb 2011 at 3:42

GoogleCodeExporter commented 9 years ago
Would you mind looking at my CCPN setup I recall we made an account for you on 
nmr but you can get in thru www right?
My one and only setup is in:

jd:nmr/log/ echo $CCPNMR_TOP_DIR/
/Users/jd/workspace35/ccpn/

Original comment by jurge...@gmail.com on 10 Feb 2011 at 5:19

GoogleCodeExporter commented 9 years ago
So the problem is at the BMRB end, because that's where these CCPN projects 
originate from, right? If so, I'll see if I can still check over there what's 
going on .

Original comment by wfvran...@gmail.com on 21 Feb 2011 at 3:38

GoogleCodeExporter commented 9 years ago
Yes.

Original comment by jurge...@gmail.com on 21 Feb 2011 at 5:58

GoogleCodeExporter commented 9 years ago
Chris, could you reprocess 2kid please?
Hopefully, the new CCPN code should have improved.

Original comment by jurge...@gmail.com on 28 Mar 2011 at 8:21

GoogleCodeExporter commented 9 years ago
It looks like this hadn't crashed the format converter since december. I took a 
closer look at the presetDict and CIF file though and realized that I had 
chains B and C reversed in the presets. Just to make sure, I updated the 
ccpn-chemcomp directory. I'm setting it to fixed - does someone want to varify 
it?

Here are the preset settings that get 2866/2867 conversions:

   'keywds': {

#       'forceChainMappings': [['A', ' ', 1, 0], ['B', ' ', 1, 699], ['C', ' ', 
1, 701]]
      'forceChainMappings': [['A', ' ', 1, 0], ['B', ' ', 1, 700], ['C', ' ', 1, 699]],

       'useCcpnAtomMatching': True,

        'addNameMappings':{
               'Gln': [ ["HG","HG*"] ],
               'Pro': [ ["HG","HG*"] ],
         },
      },

Original comment by schulte....@gmail.com on 28 Mar 2011 at 7:52

GoogleCodeExporter commented 9 years ago
Thanks Chris!

Original comment by jurge...@gmail.com on 29 Mar 2011 at 9:03