mattb112885 / clusterDbAnalysis

ITEP - Integrated Toolkit for Exploration of microbial Pan-genomes
26 stars 15 forks source link

sqlite3.OperationalError #72

Open spaver opened 8 years ago

spaver commented 8 years ago

I am running ITEP using the 64GB virtual box. The setup_step1 step completed successfully, but running setup_step2 yields the following error: Traceback (most recent call last): File "/home/itep/LATEST_ITEP/master/src/internal/db_loadPresenceAbsence.py", line 125, in cur.execute(cmd, tuple(sp)) sqlite3.OperationalError: table presenceabsence has 24 columns but 36 values were supplied

I think there are 24 columns for "runid", "clusterid", "annote", and 21 genomes. I do not know where 36 values would come from.

mattb112885 commented 8 years ago

Hello,

This is indeed very odd...

Could you add the following command to the db_loadPresenceAbsence.py right before the line that failed:

print tuple(sp)

and then run

$ db_loadPresenceAbsence.py

and let me know what gets printed? Maybe something fishy will come up there that will help me diagnose what happened.

As for the effects this has, you should be able to just run the db_loadPresenceAbsence.py (after we fix whatever the problem is) and not have to rerun the whole setup step 2 (loading this is the last step and the rest of the database gets loaded first). You can also still do any analysis you want with the blast results as long as you don't need the presence absence table.

spaver commented 8 years ago

Hi,

The following is the end of what gets printed: (u'all_I_2.0_c_0.4_m_maxbit', u'10728', u'hypothetical protein_wp_006043415.1_wh7805_rs12110', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|59931.88888.peg.2338', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'4873', u'1,4-alpha-glucan (glycogen) branching enzyme, gh-13-type (ec 2.4.1.18)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.2451', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166916.peg.241', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'12508', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166915.peg.561', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'5629', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.95', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.1270', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'7329', u'ribonuclease z (ec 3.1.26.11)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.530', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'462', u'sugar/maltose fermentation stimulation protein homolog', u'fig|6666666.166904.peg.1714', u'NONE', u'fig|110662.88888.peg.239', u'fig|221359.88888.peg.2361', u'fig|292564.88888.peg.1857', u'fig|6666666.166917.peg.309', u'fig|6666666.166913.peg.344', u'fig|6666666.166903.peg.2298', u'fig|6666666.166876.peg.2410', u'fig|6666666.166915.peg.2019', u'fig|6666666.166998.peg.110', u'fig|180281.88888.peg.1621', u'fig|32051.88888.peg.285', u'fig|6666666.166909.peg.669', u'fig|316278.88888.peg.2144', u'fig|69042.88888.peg.206', u'fig|6666666.166899.peg.2895', u'fig|6666666.166916.peg.1271', u'fig|6666666.166902.peg.1415', u'fig|59931.88888.peg.1366', u'fig|6666666.166900.peg.292') (u'all_I_2.0_c_0.4_m_maxbit', u'10133', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.786', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'2226', u'hypothetical protein', u'fig|6666666.166904.peg.281', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.2049', u'fig|6666666.166876.peg.89', u'NONE', u'fig|6666666.166998.peg.987', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|69042.88888.peg.2203', u'fig|6666666.166899.peg.1192', u'NONE', u'fig|6666666.166902.peg.1558', u'NONE', u'fig|6666666.166900.peg.2083') (u'all_I_2.0_c_0.4_m_maxbit', u'11897', u'hypothetical protein', u'fig|6666666.166904.peg.790', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'4966', u'arylsulfatase_wp_006041881.1_wh7805_rs04830', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|59931.88888.peg.935', u'fig|6666666.166900.peg.40') (u'all_I_2.0_c_0.4_m_maxbit', u'12589', u'fig00450972: hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166917.peg.1149', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6730', u'cytochrome p450', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.1627', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'7486', u'hypothetical protein_wp_041435226.1_syncc9605_rs11825', u'NONE', u'NONE', u'fig|110662.88888.peg.2278', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1631', u'gamma-glutamyltranspeptidase (ec 2.3.2.2)', u'fig|6666666.166904.peg.25', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.986', u'NONE', u'fig|6666666.166913.peg.2378', u'fig|6666666.166903.peg.1380', u'fig|6666666.166876.peg.2348', u'fig|6666666.166915.peg.2631', u'fig|6666666.166998.peg.2095', u'fig|180281.88888.peg.30', u'NONE', u'fig|6666666.166909.peg.2205', u'fig|316278.88888.peg.1527', u'fig|69042.88888.peg.2405', u'fig|6666666.166899.peg.2061', u'NONE', u'fig|6666666.166902.peg.2802', u'NONE', u'fig|6666666.166900.peg.2624') (u'all_I_2.0_c_0.4_m_maxbit', u'10210', u'hypothetical protein_wp_011935929.1_synrcc307_rs07220', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|316278.88888.peg.1401', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'2307', u'fig01154149: hypothetical protein', u'NONE', u'NONE', u'NONE', u'fig|221359.88888.peg.2163', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.1854', u'fig|180281.88888.peg.1966', u'fig|32051.88888.peg.91', u'NONE', u'NONE', u'fig|69042.88888.peg.2129', u'NONE', u'NONE', u'NONE', u'fig|59931.88888.peg.1599', u'fig|6666666.166900.peg.2017') (u'all_I_2.0_c_0.4_m_maxbit', u'11990', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166909.peg.2380', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'5111', u'alkaline phosphatase (ec 3.1.3.1)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.1576', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.2788', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'12730', u'putative exported protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166917.peg.2119', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6875', u'exodeoxyribonuclease v subunit gamma_wp_011935972.1_synrcc307_rs07415', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|316278.88888.peg.1437', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'7567', u'hypothetical protein_wp_050751566.1_wh5701_rs15885', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|69042.88888.peg.828', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1708', u'monooxygenase, putative', u'fig|6666666.166904.peg.2056', u'NONE', u'fig|110662.88888.peg.1072', u'NONE', u'fig|292564.88888.peg.1754', u'NONE', u'NONE', u'fig|6666666.166903.peg.64', u'fig|6666666.166876.peg.1394', u'NONE', u'fig|6666666.166998.peg.2077', u'fig|180281.88888.peg.852', u'NONE', u'NONE', u'NONE', u'fig|69042.88888.peg.354', u'fig|6666666.166899.peg.2992', u'fig|6666666.166916.peg.699', u'fig|6666666.166902.peg.1083', u'fig|59931.88888.peg.274', u'fig|6666666.166900.peg.582') (u'all_I_2.0_c_0.4_m_maxbit', u'9331', u'hypothetical protein_wp_038023483.1_rs9916_rs07930', u'NONE', u'NONE', u'NONE', u'fig|221359.88888.peg.1537', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'2448', u'potassium efflux system kefa protein / small-conductance mechanosensitive channel', u'fig|6666666.166904.peg.2685', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.2345', u'NONE', u'NONE', u'fig|6666666.166903.peg.746', u'fig|6666666.166876.peg.2622', u'NONE', u'NONE', u'NONE', u'fig|32051.88888.peg.1585', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.2845', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'12071', u'hypothetical protein', u'NONE', u'fig|6666666.166910.peg.1046', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'4164', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.248', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|69042.88888.peg.1571', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166900.peg.2490') (u'all_I_2.0_c_0.4_m_maxbit', u'6952', u'possible phage integrase family', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166900.peg.746') (u'all_I_2.0_c_0.4_m_maxbit', u'8732', u'bifunctional protein: zinc-containing alcohol dehydrogenase; quinone oxidoreductase ( nadph:quinone reductase) (ec 1.1.1.-); similar to arginate lyase', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.1097', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1853', u'hypothetical protein', u'fig|6666666.166904.peg.2028', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.2659', u'fig|6666666.166917.peg.383', u'fig|6666666.166913.peg.1365', u'fig|6666666.166903.peg.1468', u'fig|6666666.166876.peg.3115', u'fig|6666666.166915.peg.2225', u'NONE', u'NONE', u'NONE', u'fig|6666666.166909.peg.559', u'NONE', u'fig|69042.88888.peg.32', u'fig|6666666.166899.peg.1633', u'NONE', u'fig|6666666.166902.peg.2287', u'NONE', u'fig|6666666.166900.peg.926') (u'all_I_2.0_c_0.4_m_maxbit', u'9408', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.2028', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'2529', u'abc transporter atp-binding protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.69', u'NONE', u'fig|6666666.166915.peg.228', u'NONE', u'fig|180281.88888.peg.1552', u'NONE', u'fig|6666666.166909.peg.66', u'NONE', u'NONE', u'fig|6666666.166899.peg.112', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'12212', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.1222', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'4309', u'aromatic-l-amino-acid decarboxylase (ec 4.1.1.28)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.833', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.2068', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'7097', u'twitching motility protein pilt', u'NONE', u'fig|6666666.166910.peg.429', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'8813', u'fig01150229: hypothetical protein', u'fig|6666666.166904.peg.23', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1930', u'amino acid permease', u'fig|6666666.166904.peg.1705', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.282', u'NONE', u'fig|6666666.166913.peg.893', u'fig|6666666.166903.peg.105', u'fig|6666666.166876.peg.904', u'fig|6666666.166915.peg.2503', u'fig|6666666.166998.peg.623', u'NONE', u'NONE', u'fig|6666666.166909.peg.1126', u'NONE', u'NONE', u'fig|6666666.166899.peg.2471', u'NONE', u'fig|6666666.166902.peg.907', u'NONE', u'fig|6666666.166900.peg.1587') (u'all_I_2.0_c_0.4_m_maxbit', u'9553', u'hypothetical protein_wp_038023103.1_rs9916_rs02365', u'NONE', u'NONE', u'NONE', u'fig|221359.88888.peg.462', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'3710', u'cell division protein ftsl', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.1261', u'fig|6666666.166876.peg.1344', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.1100', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'11269', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166900.peg.1363') (u'all_I_2.0_c_0.4_m_maxbit', u'4386', u'hypothetical protein_wp_011933579.1_synwh7803_rs08275', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|32051.88888.peg.1608', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|59931.88888.peg.912', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6166', u'hypothetical protein', u'NONE', u'fig|6666666.166910.peg.1245', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.706', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'8954', u'conserved domain protein_wp_006911159.1_cpcc7001_rs13925', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|180281.88888.peg.473', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1051', u'lsu ribosomal protein l33p @ lsu ribosomal protein l33p, zinc-dependent', u'NONE', u'NONE', u'fig|110662.88888.peg.1293', u'fig|221359.88888.peg.779', u'fig|292564.88888.peg.479', u'NONE', u'fig|6666666.166913.peg.2490', u'fig|6666666.166903.peg.2775', u'fig|6666666.166876.peg.774', u'fig|6666666.166915.peg.24', u'fig|6666666.166998.peg.1022', u'fig|180281.88888.peg.505', u'fig|32051.88888.peg.1234', u'fig|6666666.166909.peg.1372', u'fig|316278.88888.peg.1124', u'fig|69042.88888.peg.2240', u'fig|6666666.166899.peg.1282', u'fig|6666666.166916.peg.589', u'fig|6666666.166902.peg.2810', u'fig|59931.88888.peg.431', u'fig|6666666.166900.peg.361') (u'all_I_2.0_c_0.4_m_maxbit', u'9646', u'hypothetical protein_wp_043325229.1_cyagr_rs00515', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.100', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'3791', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.391', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.1755', u'NONE', u'fig|6666666.166900.peg.1043') (u'all_I_2.0_c_0.4_m_maxbit', u'11410', u'fig00562554: hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166900.peg.2900') (u'all_I_2.0_c_0.4_m_maxbit', u'4531', u'glycosyltransferase', u'fig|6666666.166904.peg.1793', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|69042.88888.peg.2137', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6247', u'alpha-ketoglutarate decarboxylase_wp_011364259.1_syncc9605_rs06445', u'NONE', u'NONE', u'fig|110662.88888.peg.1240', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'9035', u'type i restriction-modification system, restriction subunit r (ec 3.1.21.3)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.270', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1128', u'ribosomal rna large subunit methyltransferase n (ec 2.1.1.-)', u'fig|6666666.166904.peg.1478', u'NONE', u'fig|110662.88888.peg.1987', u'fig|221359.88888.peg.1593', u'fig|292564.88888.peg.2726', u'fig|6666666.166917.peg.2520', u'NONE', u'fig|6666666.166903.peg.656', u'fig|6666666.166876.peg.153', u'fig|6666666.166915.peg.1715', u'fig|6666666.166998.peg.3185', u'fig|180281.88888.peg.2384', u'fig|32051.88888.peg.1973', u'fig|6666666.166909.peg.2424', u'fig|316278.88888.peg.1802', u'fig|69042.88888.peg.2695', u'fig|6666666.166899.peg.892', u'NONE', u'fig|6666666.166902.peg.1930', u'fig|59931.88888.peg.2200', u'fig|6666666.166900.peg.257') (u'all_I_2.0_c_0.4_m_maxbit', u'10815', u'phosphoglycolate phosphatase (ec 3.1.3.18)', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.1312', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'3932', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.1531', u'fig|6666666.166903.peg.385', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.455', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'11491', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.1039', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'5632', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.728', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.1300', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6388', u'hypothetical protein_wp_011364169.1_syncc9605_rs05965', u'NONE', u'NONE', u'fig|110662.88888.peg.1146', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'533', u'cytochrome c oxidase polypeptide ii (ec 1.9.3.1)', u'fig|6666666.166904.peg.2664', u'NONE', u'fig|110662.88888.peg.591', u'fig|221359.88888.peg.1382', u'fig|292564.88888.peg.2895', u'fig|6666666.166917.peg.1455', u'fig|6666666.166913.peg.2196', u'fig|6666666.166903.peg.2726', u'fig|6666666.166876.peg.2644', u'fig|6666666.166915.peg.2078', u'fig|6666666.166998.peg.88', u'fig|180281.88888.peg.1158', u'fig|32051.88888.peg.1790', u'fig|6666666.166909.peg.1070', u'fig|316278.88888.peg.614', u'fig|69042.88888.peg.1329', u'fig|6666666.166899.peg.2271', u'fig|6666666.166916.peg.803', u'fig|6666666.166902.peg.1158', u'fig|59931.88888.peg.2417', u'fig|6666666.166900.peg.311') (u'all_I_2.0_c_0.4_m_maxbit', u'9176', u'hypothetical protein_wp_007098041.1_rs9916_rs04355', u'NONE', u'NONE', u'NONE', u'fig|221359.88888.peg.853', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1273', u'dna primase (ec 2.7.7.-)', u'NONE', u'NONE', u'fig|110662.88888.peg.1128', u'fig|221359.88888.peg.553', u'fig|292564.88888.peg.385', u'fig|6666666.166917.peg.1404', u'fig|6666666.166913.peg.311', u'fig|6666666.166903.peg.1836', u'NONE', u'fig|6666666.166915.peg.1268', u'fig|6666666.166998.peg.2890', u'fig|180281.88888.peg.617', u'fig|32051.88888.peg.1023', u'fig|6666666.166909.peg.2414', u'fig|316278.88888.peg.1216', u'fig|69042.88888.peg.1444', u'fig|6666666.166899.peg.1070', u'NONE', u'fig|6666666.166902.peg.580', u'fig|59931.88888.peg.349', u'fig|6666666.166900.peg.1539') (u'all_I_2.0_c_0.4_m_maxbit', u'10892', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.2160', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'4013', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.673', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.2798', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.2687', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'11632', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.2595', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'5777', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.1796', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.2694', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6469', u'hypothetical protein_wp_006041527.1_wh7805_rs13505', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|59931.88888.peg.618', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'610', u'solanesyl diphosphate synthase (ec 2.5.1.11)', u'fig|6666666.166904.peg.2615', u'NONE', u'fig|110662.88888.peg.1097', u'fig|221359.88888.peg.525', u'fig|292564.88888.peg.353', u'fig|6666666.166917.peg.1438', u'fig|6666666.166913.peg.517', u'fig|6666666.166903.peg.2026', u'fig|6666666.166876.peg.1158', u'fig|6666666.166915.peg.555', u'fig|6666666.166998.peg.2507', u'fig|180281.88888.peg.645', u'fig|32051.88888.peg.988', u'NONE', u'fig|316278.88888.peg.1061', u'fig|69042.88888.peg.1711', u'fig|6666666.166899.peg.774', u'fig|6666666.166916.peg.1168', u'fig|6666666.166902.peg.1675', u'fig|59931.88888.peg.321', u'fig|6666666.166900.peg.2051') (u'all_I_2.0_c_0.4_m_maxbit', u'8233', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.2891', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1366', u'unnamed protein product', u'fig|6666666.166904.peg.1973', u'fig|6666666.166910.peg.996', u'fig|110662.88888.peg.266', u'fig|221359.88888.peg.2388', u'fig|292564.88888.peg.1800', u'fig|6666666.166917.peg.744', u'fig|6666666.166913.peg.1011', u'NONE', u'fig|6666666.166876.peg.1470', u'fig|6666666.166915.peg.2852', u'fig|6666666.166998.peg.2837', u'fig|180281.88888.peg.1589', u'fig|32051.88888.peg.312', u'fig|6666666.166909.peg.641', u'NONE', u'fig|69042.88888.peg.179', u'NONE', u'NONE', u'fig|6666666.166902.peg.2511', u'fig|59931.88888.peg.1339', u'fig|6666666.166900.peg.234') (u'all_I_2.0_c_0.4_m_maxbit', u'11037', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.660', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'3130', u'hypothetical protein', u'fig|6666666.166904.peg.1832', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.2161', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.2466', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.163', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'11713', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.794', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'5870', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.2003', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166899.peg.751', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'6610', u'hypothetical protein_wp_041434760.1_syncc9605_rs06905', u'NONE', u'NONE', u'fig|110662.88888.peg.1331', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'755', u'putative holliday junction resolvase yqgf', u'fig|6666666.166904.peg.2770', u'NONE', u'fig|110662.88888.peg.1800', u'fig|221359.88888.peg.661', u'fig|292564.88888.peg.663', u'fig|6666666.166917.peg.2219', u'fig|6666666.166913.peg.1670', u'fig|6666666.166903.peg.272', u'fig|6666666.166876.peg.475', u'fig|6666666.166915.peg.1899', u'fig|6666666.166998.peg.211', u'fig|180281.88888.peg.220', u'fig|32051.88888.peg.1106', u'fig|6666666.166909.peg.355', u'fig|316278.88888.peg.935', u'fig|69042.88888.peg.1630', u'fig|6666666.166899.peg.1346', u'NONE', u'fig|6666666.166902.peg.2758', u'fig|59931.88888.peg.551', u'fig|6666666.166900.peg.648') (u'all_I_2.0_c_0.4_m_maxbit', u'8326', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166913.peg.1199', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'1447', u'ferredoxin, 2fe-2s', u'fig|6666666.166904.peg.1452', u'NONE', u'fig|110662.88888.peg.1903', u'fig|221359.88888.peg.1088', u'fig|292564.88888.peg.1036', u'fig|6666666.166917.peg.375', u'fig|6666666.166913.peg.2260', u'fig|6666666.166903.peg.1357', u'fig|6666666.166876.peg.181', u'fig|6666666.166915.peg.1086', u'NONE', u'fig|180281.88888.peg.2636', u'fig|32051.88888.peg.1555', u'fig|6666666.166909.peg.1145', u'NONE', u'fig|69042.88888.peg.2482', u'fig|6666666.166899.peg.2722', u'NONE', u'fig|6666666.166902.peg.1885', u'fig|59931.88888.peg.865', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'11114', u'hypothetical protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166903.peg.904', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'3211', u'hypothetical protein', u'fig|6666666.166904.peg.1790', u'NONE', u'NONE', u'NONE', u'fig|292564.88888.peg.1021', u'NONE', u'NONE', u'NONE', u'fig|6666666.166876.peg.1711', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166902.peg.3129', u'NONE', u'NONE') (u'all_I_2.0_c_0.4_m_maxbit', u'12894', u'adp-ribose 1"-phosphate phophatase related protein', u'', u'', u'', u'atgatccgcgacaccgctgcctccgtgttcaacacctctgcggaggtggtggtcaacaccgtgaactgcgagggtgtgatgggggccggcctggcgcttgagttcgcccttcgccatcccgatctggaggccgactatcagctccgctgccaggccggcgcggtgcagatcggccggccctatctctttccggtagccggctgcccctatcgggaagtactgaatttccccaccaaacagcactggcgatttccgtcccgcctgggctggatcgagcaggccttgtctttcatcgcctcccactacagccggtcgagcccagcgatcacctccctggccctgccgcggctcggctgcgacaagggcgggctgaactgggccgatgtgcgccccctgatcgagcgccacctcgctgatctcccaggcctcaccgtctacctctgcgccgacagcgccccggcggagggcaccgaggcggtaatgctgacggccttcgccagggatcagcaggccggtgagctgcctgccttcctcaagggcaaggcccgtcaggccctgctcaagtcatccccgccgccgcgcttccgtcagctggcagccgtcgccggcgtgggcaagcagagctatgcccgcctcttccagcactactaccgctgcggtgatgccgcccagctcagtttgctgggcgtagagacagcctga', u'mirdtaasvfntsaevvvntvncegvmgaglalefalrhpdleadyqlrcqagavqigrpylfpvagcpyrevlnfptkqhwrfpsrlgwieqalsfiashysrsspaitslalprlgcdkgglnwadvrplierhladlpgltvylcadsapaegteavmltafardqqagelpaflkgkarqallkssppprfrqlaavagvgkqsyarlfqhyyrcgdaaqlsllgveta\ncontig-120_32', u'fig|6666666.166998.peg.1624', u'peg', u'contig-120_32_35032_34412', u'35032', u'34412', u'-', u'adp-ribose 1"-phosphate phophatase related protein', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'fig|6666666.166998.peg.1623', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE', u'NONE') Traceback (most recent call last): File "/home/itep/LATEST_ITEP/master/src/internal/db_loadPresenceAbsence.py", line 126, in cur.execute(cmd, tuple(sp)) sqlite3.OperationalError: table presenceabsence has 24 columns but 36 values were supplied

There was so much output that I was not able to capture the first part.

On Jan 25, 2016, at 4:18 PM, mattb112885 notifications@github.com wrote:

Hello,

This is indeed very odd...

Could you add the following command to the db_loadPresenceAbsence.py right before the line that failed:

print tuple(sp)

and then run

$ db_loadPresenceAbsence.py

and let me know what gets printed? Maybe something fishy will come up there that will help me diagnose what happened.

As for the effects this has, you should be able to just run the db_loadPresenceAbsence.py (after we fix whatever the problem is) and not have to rerun the whole setup step 2 (loading this is the last step and the rest of the database gets loaded first). You can also still do any analysis you want with the blast results as long as you don't need the presence absence table.

— Reply to this email directly or view it on GitHub https://github.com/mattb112885/clusterDbAnalysis/issues/72#issuecomment-174694543.

mattb112885 commented 8 years ago

It looks like something went wrong with parsing a file (most likely the genbank file) here, because a gene sequence and protein sequence got pulled into one of the annotations.

I'd recommend looking for this gene in your Genbank files and seeing if anything looks particularly strange for that gene...

Regardless, I have pushed up a band-aid for this (makes it not crash if there are tabs in the annotation) so please try to do a git pull and see if it works for you (pull down the latest, then just run db_loadPresenceAbsence.py). As mentioned above the root cause is probably something more insidious -- maybe it wouldn't happen with an updated version of Biopython?

If you don't mind missing info on that one gene, the rest of the results look fine up to that point so my guess is this was a very rare occurrence in your dataset and any other analysis that doesn't need to include that gene should be OK.