FC exporters fail to read XML data

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. run $R/python/recoord2/msd/exportCyana.py on this entry

What is the expected output? What do you see instead?
Model read finished. Duration 2.14677906036
Model validity check skipped
Warning: Backup is being changed from "/big/docr/NRG/link/1lfu/1lfu_backup" to 
"/big/docr/ccpn_tmp/data/recoord/1lfu/linkNmrStarData_backup", updating path
Error loading file for:  <ccp.molecule.MolSystem.MolSystem ['1lfu']>
Reading:  <open file 
'/big/docr/ccpn_tmp/data/recoord/1lfu/linkNmrStarData/ccp/molecule/MolSystem/1lf
u+1lfu_u
ser_2009-01-19-14-47-22-947_00002.xml', mode 'r' at 0x588ce40>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Traceback (most recent call last):
  File "/big/docr/workspace/recoord/python/recoord2/msd/exportCyana.py", line 119, in ?
    exportCyanaInputFiles(origStrucGen,outputDirectory,pdbCode)
  File "/big/docr/workspace/recoord/python/recoord2/msd/exportCyana.py", line 47, in 
exportCyanaInputFiles
    fc.exportFile('sequence','cyana',os.path.join(filePath,"%s.seq" % 
fileRoot),addKeywords=addKeywds)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 292, in 
exportFile
    self.executeIO('export',fileName,informationType,formatName,addKeywords)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 863, in 
executeIO
    keywords = self.getKeywords(funcType,fileName,informationType,formatName,addKeywords)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 939, in 
getKeywords
    self.setDefaultKeywords(keywords,funcType,informationType,formatName,addKeywords)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 1038, in 
setDefaultKeywords
    keywords['chains'] = self.getChains(funcType,addKeywords)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 1121, in 
getChains
    molSystem = self.getMolSystem(funcType)
  File "/big/docr/workspace/ccpn/python/ccpnmr/format/general/Conversion.py", line 1099, in 
getMolSystem
    self.nmrEntry.molSystem = molSystem
  File "/big/docr/workspace/ccpn/python/ccp/api/nmr/NmrEntry.py", line 6221, in 
setMolSystem
    topObjectModify.load()
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5036, in load
    self.loadFrom(repositories[0])
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5070, in 
loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 236, in 
loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 267, in 
loadFromStream
    partialLoad=partialLoad)
  File "/big/docr/workspace/ccpn/python/memops/xml/Implementation.py", line 4833, in 
loadFromStream
    obj.checkValid()
  File "/big/docr/workspace/ccpn/python/ccp/api/molecule/MolSystem.py", line 3223, in 
checkValid
    raise ApiError("""%s.checkValid:
memops.general.Implementation.ApiError: ccp.molecule.MolSystem.Chain.checkValid:
       constraint seqCode_and_seqInsertCode_form_secondary_key_for_residues violated: 
<ccp.molecule.MolSystem.Chain ['1lfu', 'C']>

Original issue reported on code.google.com by jurge...@gmail.com on 20 Jan 2009 at 10:37

GoogleCodeExporter commented 9 years ago

Related bug for Cns exporter in a different entry:

Model read finished. Duration 2.16650891304
Model validity check skipped
Warning: Backup is being changed from "/big/docr/NRG/link/1mit/1mit_backup" to 
"/big/docr/ccpn_tmp/data/recoord/1mit/linkNmrStarData_backup", updating path
Error loading file for:  <ccp.molecule.MolSystem.MolSystem ['1mit']>
Reading:  <open file 
'/big/docr/ccpn_tmp/data/recoord/1mit/linkNmrStarData/ccp/molecule/MolSystem/1mi
t+1mit_user_2009-
01-19-17-44-51-273_00001.xml', mode 'r' at 0x468bf30>
Last xml tag read:  _StorageUnit
Parser state was:  checking validity
Object stack was empty
Error loading file for:  <ccp.nmr.NmrConstraint.NmrConstraintStore [1]>
Reading:  <open file 
'/big/docr/ccpn_tmp/data/recoord/1mit/linkNmrStarData/ccp/nmr/NmrConstraint/1+1m
it_user_2009-01-
19-17-44-51-292_00002.xml', mode 'r' at 0x36b5378>
Last xml tag read:  NMRC.FixedAtomSet.atoms
Parser state was:  reading
Current object was:  <ccp.nmr.NmrConstraint.FixedAtomSet [1, 1]>
Traceback (most recent call last):
  File "/big/docr/workspace/recoord/python/recoord2/msd/exportCns.py", line 105, in ?
    exportCnsInputFiles(origStrucGen,outputDirectory,pdbCode)
  File "/big/docr/workspace/recoord/python/recoord2/msd/exportCns.py", line 47, in exportCnsInputFiles
    constraintLists = strucGen.nmrConstraintStore.findAllConstraintLists(className = constraintListClassName)
  File "/big/docr/workspace/ccpn/python/ccp/api/nmr/Nmr.py", line 84269, in getNmrConstraintStore
    tobj.load()
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5036, in load
    self.loadFrom(repositories[0])
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5070, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 267, in loadFromStream
    partialLoad=partialLoad)
  File "/big/docr/workspace/ccpn/python/memops/xml/Implementation.py", line 4757, in loadFromStream
    val[ii] = clazz.getByKey(oo,vv[1:-1])
  File "/big/docr/workspace/ccpn/python/ccp/api/molecule/MolSystem.py", line 1278, in getByKey
    obj1.load()
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5036, in load
    self.loadFrom(repositories[0])
  File "/big/docr/workspace/ccpn/python/memops/api/Implementation.py", line 5070, in loadFrom
    XmlIO.loadTopObject(repository.url.getDataLocation(), self)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 236, in loadTopObject
    return loadFromStream(open(filePath), topObjId=topObjId, topObject=topObject)
  File "/big/docr/workspace/ccpn/python/memops/format/xml/XmlIO.py", line 273, in loadFromStream
    partialLoad=partialLoad)
  File "/big/docr/workspace/ccpn/python/memops/xml/Implementation.py", line 4833, in loadFromStream
    obj.checkValid()
  File "/big/docr/workspace/ccpn/python/ccp/api/molecule/MolSystem.py", line 3223, in checkValid
    raise ApiError("""%s.checkValid:
memops.general.Implementation.ApiError: ccp.molecule.MolSystem.Chain.checkValid:
       constraint seqCode_and_seqInsertCode_form_secondary_key_for_residues violated: 
<ccp.molecule.MolSystem.Chain ['1mit', 'A']>

Original comment by jurge...@gmail.com on 20 Jan 2009 at 10:40

GoogleCodeExporter commented 9 years ago

Wim, could you take a look at this?

Original comment by jurge...@gmail.com on 21 Jan 2009 at 12:35

GoogleCodeExporter commented 9 years ago

Ah yes. The problem with these is that there are sequence insertion codes in 
the PDB
file. I will have a look, see if I can fix it relatively easily.

Original comment by wfvran...@gmail.com on 22 Jan 2009 at 2:40

GoogleCodeExporter commented 9 years ago

Update: my code reads the PDB file just fine, and as far as I remember this all 
used
to work. However, in the joinedCoord file, you do not store the insertion code, 
which
means that the first two residues (GLY and SER) have the same author sequence 
code.
This creates problems for the mappings, and later on for storage in CCPN.

The most obvious solution here is that we include the insertion code in the 
NMR-STAR
coordinate section (if there are any in the PDB file).

Original comment by wfvran...@gmail.com on 22 Jan 2009 at 3:14

GoogleCodeExporter commented 9 years ago

That's a snag.

I see:
ATOM      1  N   GLY A   1A     -2.327 -16.682  -7.753  1.00  5.59           N  
ATOM      2  CA  GLY A   1A     -3.654 -16.006  -7.759  1.00  5.24           C  
ATOM      3  C   GLY A   1A     -3.612 -14.842  -8.750  1.00  4.42           C  
ATOM      4  O   GLY A   1A     -2.539 -14.446  -9.159  1.00  4.73           O  
ATOM      5  HA2 GLY A   1A     -4.402 -16.726  -8.056  1.00  5.46           H  
ATOM      6  HA3 GLY A   1A     -3.856 -15.644  -6.762  1.00  5.76           H  
ATOM      7  H1  GLY A   1A     -1.590 -16.002  -8.026  1.00  5.69           H  
ATOM      8  H2  GLY A   1A     -2.128 -17.046  -6.800  1.00  5.92           H  
ATOM      9  H3  GLY A   1A     -2.337 -17.472  -8.430  1.00  5.80           H  

Wattos was setup with explicitly not supporting insertion codes. 

I see it's an easy change for us from the mmCIF file:
_pdbx_poly_seq_scheme.pdb_ins_code
A 1 1  GLY 1  1  1  GLY GLY A A
A 1 2  SER 2  1  1  SER SER A .
and the fill in:
_atom_site.pdbx_PDB_ins_code

Eldon, can I have Wattos add a tag like: _pdbx_poly_seq_scheme.pdb_ins_code so 
that the FC can sort this 
out? Please suggest one. Perhaps: 

_Atom_site.PDB_ins_code     CHAR(1)

Would that work for you too Wim? You mentioned it should be in the coordinate 
section, I understand 
because of the way the FC works best.

It will only be a problem for less than half a dozen entries though so I don't 
know if it's worth the efforts?

Original comment by jurge...@gmail.com on 22 Jan 2009 at 7:24

Changed state: Started
Added labels: Entry-1mit

GoogleCodeExporter commented 9 years ago

Ok, I now see the tag
_Atom_site.PDB_ins_code
is already in the dictionary.

I'm going to adjust the Wattos code for this.

Original comment by jurge...@gmail.com on 22 Jan 2009 at 7:27

Added labels: Priority-High, Milestone-Paper-NRG-2009
Removed labels: Priority-Medium

GoogleCodeExporter commented 9 years ago

Wim would you agree an insertion code of " " should be expressed as a . 
(basically a None) in the NMR-STAR 
file? Makes it much faster to work with.

Original comment by jurge...@gmail.com on 22 Jan 2009 at 8:46

GoogleCodeExporter commented 9 years ago

Yep fine by me

Original comment by wfvran...@gmail.com on 22 Jan 2009 at 8:58

GoogleCodeExporter commented 9 years ago

Wim, can you read the attached and try the Cns exporter after that?

Original comment by jurge...@gmail.com on 23 Jan 2009 at 11:14

Attachments:

1mit_wattos.str.gz

GoogleCodeExporter commented 9 years ago

Involved entries can be found by:

cd $dir_export/
find . -name "*export*.log" | xargs grep "Error loading file" | cut -c3-6

Just 6 entries involved:
1ckr
1lfu
1mit
1qe7
7hsc
1ah1

Original comment by jurge...@gmail.com on 29 Jan 2009 at 12:47

GoogleCodeExporter commented 9 years ago

Lowered priority and removed from must do before milestone setting.

Original comment by jurge...@gmail.com on 30 Jan 2009 at 3:10

Added labels: Priority-Medium
Removed labels: Priority-High, Milestone-Paper-NRG-2009

GoogleCodeExporter commented 9 years ago

Edited title and removed entry 1mit.
Solving it for the first entry will solve it for the others.

Original comment by jurge...@gmail.com on 30 Jan 2009 at 3:29

Changed title: FC exporters fail to read XML data
Removed labels: Entry-1mit

GoogleCodeExporter commented 9 years ago

Issue 184 has been merged into this issue.

Original comment by jurge...@gmail.com on 12 Feb 2009 at 12:13

GoogleCodeExporter commented 9 years ago

Also reported in issue 205.

Original comment by jurge...@gmail.com on 9 Apr 2009 at 3:04

Changed state: Duplicate

google-code-export / nmrrestrntsgrid

FC exporters fail to read XML data #159