Importing dsc data

[cybersiddhu]

31

[cybersiddhu]

• Strain: Imported 5977 strain entries. Missed 148 entries
• Properties: Imported 14854 strain property entries. Missed 412 entries
• Inventory: Imported 2052.44444444444 strain inventory entries. Missed 427.777777777778 entries
• Genotype:
• Plasmid: Failed imports and and plasmid entry not found strain_plasmid_import.txt Have to create plasmid entries before running this import.
• Publication:

DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  duplicate key value violates unique constraint "stock_pub_c1"
DETAIL:  Key (stock_id, pub_id)=(59487, 9181) already exists. [for Statement "INSERT INTO stock_pub ( pub_id, stock_id) VALUES ( ?, ? ) RETURNING stock_pub_id" with ParamValues: 1='9181', 2='59487'] at /usr/local/lib/perl5/site_perl/5.20.3/Modware/Import/Stock/StrainImporter.pm line 240

strain_publication_import.txt

• Phenotype:

  Need to create a publication entry titled 'Dicty Stock Center Phenotyping 2003-2008' Also all html entries had to be converted to mediawiki format. In notes remove any string like this [strain ID: 158, parent: DH1] or [strain #1975, parent: torA-]

[cybersiddhu]

[05-20-2016 09:44:53] INFO > StrainImporter.pm:23 - Importing data from /dsc/strain_strain.tsv [05-20-2016 09:45:08] INFO > StrainImporter.pm:70 - Imported 6125 strain entries. Missed 0 entries [05-20-2016 09:45:08] INFO > StrainImporter.pm:81 - Importing data from /dsc/strain_props.tsv [05-20-2016 09:45:35] INFO > StrainImporter.pm:122 - Imported 15266 strain property entries. Missed 0 entries

[cybersiddhu]

The import process seems to be failing because of embedded HTML in the data. It's causing the csv parser to throw up. Those HTML has to be taken care of and has to be converted to other markup like mediawiki.

[cybersiddhu]

Get a list of failing entries for the parental strains with no DBS ids. strain_no_parent.txt @pfey03

[cybersiddhu]

• [x] Order: Clean and load.
• [x] Strain : upsert.
  • [ ] Option for pruning all existing strains.
• [x] Properties : Clean and load.
• [x] Inventory: Clean and load. **
• [x] Publication: upsert.
• [x] Characteristics: find or create.
• [ ] Depositors.
• [x] Genotype: Clean and load.
• [x] Phenotype: Check , clean group and load.
• [ ] Parent: Clean and load.
• [ ] Load GWDI Strains - Clean and load
• [x] Strain gene relationships - upsert

[cybersiddhu]

• [x] Plasmid: Check and load
  • [ ] Option for pruning before load
• [ ] Inventory: Clean and load
• [x] Publication: Clean and load
• [ ] Sequences: Clean and load
• [ ] Plasmid map: Clean and load.
• [x] Plasmid gene relations: Clean and load.
• [ ] Plasmid depositors.
• [ ] Plasmid genbank accessions.