search

google-code-export / cing

Automatically exported from code.google.com/p/cing
0 stars 0 forks source link

lower casing in iCing #143

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

Wim reporting:

Get this one as well - are you lowercasing filenames, Jurgen?

Error loading file for:  None
Reading:  <open file 
'/Library/WebServer/Documents/tmp/cing/jd3/123456/basp/memops/Implementation/bas
p.xml', mode 'r' at 0x54aede8>
Last xml tag read:  IMPL.MemopsRoot
Parser state was:  reading
Object stack was empty
CING started at : Thu Mar  5 13:22:01 2009
CING stopped at : Thu Mar  5 13:22:07 2009
CING took       : 6.590 s

Traceback (most recent call last):
 File "/Users/jd/workspace34/cing/python/cing/main.py", line 702, in <module>
   main()
 File "/Users/jd/workspace34/cing/python/cing/main.py", line 565, in main
   project.initCcpn(ccpnFolder=options.initCcpn)
 File "/Users/jd/workspace34/cing/python/cing/PluginCode/ccpn.py", line 383, in initCcpn
   ccpnProject = loadProject(ccpnFolder)
 File "/Users/jd/workspace34/ccpn/python/memops/general/Io.py", line 530, in loadProject
   project = XmlIO.loadProject(path, projectName, partialLoad=True)
 File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 205, in loadProject
   return loadProjectFile(filePath, partialLoad=partialLoad)
 File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 217, in loadProjectFile
   partialLoad=partialLoad)
 File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 267, in loadFromStream
   partialLoad=partialLoad)
 File "/Users/jd/workspace34/ccpn/python/memops/xml/Implementation.py", line 4401, in loadFromStream
   + ": %s:%s" % (elemId, topObjId)
memops.general.Implementation.ApiError: TopObject identifier does not fit 
passed=in identifier: BASP:basp

Original issue reported on code.google.com by jurge...@gmail.com on 5 Mar 2009 at 12:58

GoogleCodeExporter commented 9 years ago 
Nop, iCing shouldn't.

Can you send me the test file or may I look it up on the server?

Original comment by jurge...@gmail.com on 5 Mar 2009 at 12:59

GoogleCodeExporter commented 9 years ago 
Tried to look up but it's removed.

Original comment by jurge...@gmail.com on 5 Mar 2009 at 1:02

GoogleCodeExporter commented 9 years ago 
Ah yes I'm doing the PURGE command at the end - I just reran without it, so 
should be
there now...

Original comment by wfvran...@gmail.com on 5 Mar 2009 at 1:05

GoogleCodeExporter commented 9 years ago 
Just for clarity: the original file name in the tar archive is
basp/memops/Implementation/BASP.xml. Could be a Mac upper/lowercase issue... .

Original comment by wfvran...@gmail.com on 5 Mar 2009 at 1:06

GoogleCodeExporter commented 9 years ago 
But the tar is done on your side where it is still upper correct?

Original comment by jurge...@gmail.com on 5 Mar 2009 at 1:22

GoogleCodeExporter commented 9 years ago 
Yep, it is upper cased on my side - but your server is a Mac, no?

Original comment by wfvran...@gmail.com on 5 Mar 2009 at 1:24

GoogleCodeExporter commented 9 years ago 
yes and it's looking good in the archive

give me a second to try this example out in development settings...

jd:nmr/cing/ tar -tvzf $cwd/jd3/123456/basp.tgz
drwxr-xr-x wim/msd           0 2008-12-04 11:42 BASP/
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/nmr/
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccp/nmr/Nmr/
-rw-r--r-- wim/msd    41412819 2008-12-04 11:42 
BASP/ccp/nmr/Nmr/BASP+BASP_wim_2008-12-01-
15-36-11-809_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/nmr/NmrConstraint/
-rw-r--r-- wim/msd     6107881 2008-12-04 11:38 
BASP/ccp/nmr/NmrConstraint/1+BASP_wim_2008-12-
04-10-37-39-189_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/nmr/NmrEntry/
-rw-r--r-- wim/msd        1046 2008-12-04 11:38 
BASP/ccp/nmr/NmrEntry/BASP+BASP_wim_2008-12-04-
10-36-29-176_00001.xml
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccp/general/
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccp/general/Instrument/
-rw-r--r-- wim/msd         469 2008-12-02 16:06 
BASP/ccp/general/Instrument/BASP+BASP_wim_2008-
12-01-15-36-11-808_00001.xml
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccp/general/DataLocation/
-rw-r--r-- wim/msd       16037 2008-12-02 16:06 
BASP/ccp/general/DataLocation/BASP+BASP_wim_2008-
12-01-15-39-29-252_00001.xml
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccp/general/Method/
-rw-r--r-- wim/msd       17860 2008-12-04 11:38 
BASP/ccp/general/Method/BASP+BASP_wim_2008-12-
01-15-36-11-815_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/molecule/
drwxr-xr-x wim/msd           0 2008-12-04 11:42 BASP/ccp/molecule/MolSystem/
-rw-r--r-- wim/msd      264802 2008-12-04 11:42 
BASP/ccp/molecule/MolSystem/Molecularsystem+BASP_wim_2008-12-04-10-37-21-913_000
01.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/molecule/Molecule/
-rw-r--r-- wim/msd       59413 2008-12-04 11:38 
BASP/ccp/molecule/Molecule/CONSERVED_DOMAIN_PROTEIN+BASP_wim_2008-12-04-10-37-21
-
866_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccp/molecule/MolStructure/
-rw-r--r-- wim/msd    22578826 2008-12-04 11:38 
BASP/ccp/molecule/MolStructure/_ccp_molecule_MolSystem_MolSystem___Molecularsyst
em___+1+BASP_wim_
2008-12-04-10-37-23-304_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccpnmr/
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/ccpnmr/Analysis/
-rw-r--r-- wim/msd     1480160 2008-12-04 11:42 
BASP/ccpnmr/Analysis/BASP+BASP_wim_2008-12-01-
15-36-11-811_00001.xml
drwxr-xr-x wim/msd           0 2008-12-04 11:38 BASP/ccpnmr/NameMapping/
-rw-r--r-- wim/msd        6406 2008-12-04 11:38 
BASP/ccpnmr/NameMapping/BASP+BASP_wim_2008-12-
04-10-36-33-319_00001.xml
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/memops/
drwxr-xr-x wim/msd           0 2008-12-02 16:06 BASP/memops/Implementation/
-rw-r--r-- wim/msd      227312 2008-12-04 11:38 
BASP/memops/Implementation/BASP.xml

Original comment by jurge...@gmail.com on 5 Mar 2009 at 1:26

GoogleCodeExporter commented 9 years ago 
Still fails only in iCing setup though....

================================================================================
======================
| CING: Common Interface for NMR structure Generation version 0.86              
AW,JFD,GWV 2004-2009 |
================================================================================
======================

User: root            on: nmr.cmbi.umcn.nl (darwin)                     at: Thu 
Mar  5 15:48:04 2009
CING started at : Thu Mar  5 15:48:01 2009
CING stopped at : Thu Mar  5 15:48:12 2009
CING took       : 11.120 s

Traceback (most recent call last):
  File "/Users/jd/workspace34/cing/python/cing/main.py", line 724, in <module>
    main()
  File "/Users/jd/workspace34/cing/python/cing/main.py", line 587, in main
    project.initCcpn(ccpnFolder=options.initCcpn)
  File "/Users/jd/workspace34/cing/python/cing/PluginCode/Ccpn.py", line 2029, in initCcpn
    if not ccpn.importFromCcpn():
  File "/Users/jd/workspace34/cing/python/cing/PluginCode/Ccpn.py", line 298, in importFromCcpn
    if not self.readCcpnFolder():
  File "/Users/jd/workspace34/cing/python/cing/PluginCode/Ccpn.py", line 216, in readCcpnFolder
    self.ccpnProject = loadProject(ccpnFolder)
  File "/Users/jd/workspace34/ccpn/python/memops/general/Io.py", line 530, in loadProject
    project = XmlIO.loadProject(path, projectName, partialLoad=True)
  File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 205, in loadProject
    return loadProjectFile(filePath, partialLoad=partialLoad)
  File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 217, in loadProjectFile
    partialLoad=partialLoad)
  File "/Users/jd/workspace34/ccpn/python/memops/format/xml/XmlIO.py", line 267, in loadFromStream
    partialLoad=partialLoad)
  File "/Users/jd/workspace34/ccpn/python/memops/xml/Implementation.py", line 4401, in loadFromStream
    + ": %s:%s" % (elemId, topObjId)
memops.general.Implementation.ApiError: TopObject identifier does not fit 
passed=in identifier: BASP:basp

Original comment by jurge...@gmail.com on 5 Mar 2009 at 2:49

GoogleCodeExporter commented 9 years ago 
Work around; use a file name for your CCPN project file that matches the 
project name in the CCPN project. 
E.g. the attached.

Ok, I know why.

Wim, remember in NRG we're creating linkNmrStarData directories?
In the below code in
http://code.google.com/p/cing/source/browse/trunk/cing/python/cing/PluginCode/Cc
pn.py?
spec=svn499&r=499#190

I try to retrieve a decent project name. And then I move the dir to a dir with 
that name. This should be 
improved. Please suggest how to retrieve it in a more general way from the CCPN 
files using short Java code;-
).

Btw, in iCing the project name used in CING is derived from the project 
filename. It's a separate issue from 
the one here.

            # Get a TarFile class.
            ccpnRootDirectory = None # Will become linkNmrStarData at first.
            tar = tarfile.open(self.ccpnFolder, "r:gz")
            for itar in tar:
                tar.extract(itar.name, '.') # itar is a TarInfo object
    #            NTdebug("extracted: " + itar.name)
                if isRootDirectory(itar.name):
                    if not ccpnRootDirectory: # pick only the first one.
                        ccpnRootDirectory = itar.name.replace("/", '')
                        if not ccpnRootDirectory:
                            NTerror("Skipping potential ccpnRootDirectory")
            tar.close()
            if not ccpnRootDirectory:
                NTerror("No ccpnRootDirectory found in gzipped tar file: %s" % self.ccpnFolder)
                return None

            if ccpnRootDirectory != self.project.name:
                move(ccpnRootDirectory, self.project.name)
            ccpnFolder = self.project.name # Now it is a folder.

Original comment by jurge...@gmail.com on 5 Mar 2009 at 3:05

Attachments:

GoogleCodeExporter commented 9 years ago 
OK, the best way to extract the project name is to look at the file:

<directory>/memops/Implementation/<projectName>.xml

so extract <projectName>, and that should always be good!

Original comment by wfvran...@gmail.com on 5 Mar 2009 at 4:16

GoogleCodeExporter commented 9 years ago 
Funny, this problem shows up only if you submit a file basp.tgz with the BASP 
project.
If you just rename the file to basp2.tgz it's fine.

I'll implement the above from comment 10 nevertheless.

Original comment by jurge...@gmail.com on 6 Mar 2009 at 10:57

GoogleCodeExporter commented 9 years ago 
Stupid Java; can't handle long file names in some utility code.

jd:stella/jars/ java com.ice.tar.tar -tvf $CINGROOT/Tests/data/ccpn/basp2.tgz
com.ice.tar.InvalidHeaderException: bad header in block 0 record 0, header 
magic is not 'ustar' or unix-style 
zeros, it is '75-83-657642-1121', or (dec) 75, -83, -65, 76, 42, -1, 121
    at com.ice.tar.TarInputStream.getNextEntry(Unknown Source)
    at com.ice.tar.TarArchive.listContents(Unknown Source)
    at com.ice.tar.tar.instanceMain(Unknown Source)
    at com.ice.tar.tar.main(Unknown Source)
jd:stella

I'll implement this in python but will not help iCing, sorry.

Use the 2 work arounds above.

Original comment by jurge...@gmail.com on 6 Mar 2009 at 12:04

GoogleCodeExporter commented 9 years ago 
Sorry Wim, I also asked Alan about this. When Tim walked by, he offered to help 
too. He was going to ask 
Rasmus; go figure. Didn't want to get 'everybody' involved....

My understanding now is that renaming the toplevel directory is fine and that's 
what I'll do in CING and iCing. 
There seems to be a bug (in my code?) when:

the input tgz like basp.tgz gets upacked to BASP and then moved to basp.

When I use the an input basp2.tgz, unpack to BASP and move to basp2 there is no 
problem. Most likely this is 
related to my filesystem (Mac OS Extended (Journaled)) badly emulating case 
sensitive names or the way my 
python libs go about their business.

I would say we keep this issue closed and forget about it.

Original comment by jurge...@gmail.com on 6 Mar 2009 at 1:33