Missing updates from PDBj

[GoogleCodeExporter]

What steps will reproduce the problem?

Found 77620 but expected 77878 today.

================================================================================
======================
| CING: Common Interface for NMR structure Generation version 0.95 (r1121)      
AW,JFD,GWV 2004-2011 |
================================================================================
======================
User: i          on: nmr (linux/32bit/8cores/2.7.1+)            at: (29702) Wed 
Dec 14 10:42:04 2011
==> Executing script 
"/home/i/workspace/cingStable/python/cing/NRG/weeklyUpdatePdbjMine.py"
Starting $CINGROOT/python/cing/NRG/weeklyUpdatePdbjMine.py
Getting overall number of entry count
Currently found 77440 number of entries in pdbmlplus
Removing previous copy of pdbmlplus_weekly.latest.gz
Starting downloading weekly update
2011-12-14 10:44:24 URL: 
ftp://ftp.pdbj.org/mine/weekly/pdbmlplus_weekly.latest.gz [49297910] -> 
"pdbmlplus_weekly.latest.gz" [1]
Downloaded pdbmlplus_weekly.latest.gz
Starting weekly update with [gunzip < pdbmlplus_weekly.latest.gz | psql 
pdbmlplus pdbj] and logging to: psqlUpdate.latest.log
Getting overall number of entry count
Currently found 77620 number of entries in pdbmlplus
Ending weeklyUpdatePdbjMine
CING started at : Wed Dec 14 10:42:02 2011
CING stopped at : Wed Dec 14 11:21:18 2011
CING took       : 2355.758 s

Original issue reported on code.google.com by jurge...@gmail.com on 14 Dec 2011 at 2:00

[GoogleCodeExporter]

Found missing entries:

Found RCSB PDB entries count: 77878
Creating temporary tables; disable this step for speedier testing in 
NrgCingRdb.__init__()
Found PDBj entries count: 77620
Found PDBj missing entries count: 258 [2l5r, 2l67, ...
Found RCSB-PDB missing entries count: 0 []

These are all entries from 20111111 and 20111118. So need to restore those 
week's.

Original comment by jurge...@gmail.com on 15 Dec 2011 at 9:32

• Changed state: Started

[GoogleCodeExporter]

Original comment by jurge...@gmail.com on 15 Dec 2011 at 9:32

• Added labels: Priority-High

[GoogleCodeExporter]

Downloading the complete archive is taking an overnight effort.

Original comment by jurge...@gmail.com on 15 Dec 2011 at 11:16

[GoogleCodeExporter]

Update still gave only 77620 entries. The changes must have been rolled back.

Original comment by jurge...@gmail.com on 16 Dec 2011 at 1:23

[GoogleCodeExporter]

Trying again.

Original comment by jurge...@gmail.com on 16 Dec 2011 at 2:18

[GoogleCodeExporter]

Fixed by first dropping the pdbj schema completely before doing restore.

cat pdbmlplus_split.* | pg_restore -U pdbj -d pdbmlplus

Log with error msgs below.

iLnmr/tmpPdbj/ echo "SELECT count(*) FROM brief_summary s" | psql pdbmlplus pdbj
 count 
-------
 77878
(1 row)

iLnmr/tmpPdbj/ date
Sat Dec 17 16:01:07 CET 2011

pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 35; 1255 20242023 FUNCTION 
docid_all() akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.docid_all(OUT docids integer[]) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 39; 1255 20242026 FUNCTION 
docid_of_citation_author_i(text, boolean, integer[]) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.docid_of_citation_author_i(name text, prim boolean, integer[], OUT docids integer[]) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 37; 1255 20242025 FUNCTION 
docid_of_deposition_date_i(text, text, integer[]) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.docid_of_deposition_date_i(text, text, integer[], OUT docids integer[]) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 36; 1255 20242024 FUNCTION 
docid_of_release_date_i(text, text, integer[]) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.docid_of_release_date_i(text, text, integer[], OUT docids integer[]) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 41; 1255 20242028 FUNCTION 
entry_xpath(text, text, text) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.entry_xpath(_pdbid text, _xpath text, _prefix text) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 38; 1255 20242027 FUNCTION 
release_date_of_pdbid(text) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: ALTER FUNCTION pdbj.release_date_of_pdbid(_pdbid text) OWNER TO akinjo;

pg_restore: [archiver (db)] Error from TOC entry 119083; 0 0 ACL 
docid_of_citation_author_i(text, boolean, integer[]) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: REVOKE ALL ON FUNCTION docid_of_citation_author_i(name text, prim boolean, integer[], OUT docids integer[]) FROM PUBLIC;
REV...
pg_restore: [archiver (db)] Error from TOC entry 119103; 0 0 ACL 
entry_xpath(text, text, text) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: REVOKE ALL ON FUNCTION entry_xpath(_pdbid text, _xpath text, _prefix text) FROM PUBLIC;
REVOKE ALL ON FUNCTION entry_xpath(_...
pg_restore: [archiver (db)] Error from TOC entry 119133; 0 0 ACL 
release_date_of_pdbid(text) akinjo
pg_restore: [archiver (db)] could not execute query: ERROR:  role "akinjo" does 
not exist
    Command was: REVOKE ALL ON FUNCTION release_date_of_pdbid(_pdbid text) FROM PUBLIC;
REVOKE ALL ON FUNCTION release_date_of_pdbid(_pdbid t...
WARNING: errors ignored on restore: 9

Original comment by jurge...@gmail.com on 17 Dec 2011 at 3:03

• Changed state: Fixed