Data Analysis Team tasks for the rest of the semester

[kdahlquist]

For the rest of the semester, the Data Analysis Team will focus on finishing out the collection and analysis of the modeling and graph statistics results, compiling them on the DahlquistLab repository, and writing a group report.

I am going to summarize here what we talked about at the meeting on March 30 and either update or assign new issues as appropriate.

@kdahlquist will organize the DahlquistLab repository so that we have clear directories for storing both "raw" and "processed" data, as well as a PowerPoint with graphs and a Word document for discussion of the data #336.

• Both the PowerPoint and Word document should provide hyperlinks to the actual data files on each slide/discussion paragraph
• The README.md file in each directory will serve as a table of contents for the directory, listing each file and its contents

Data we have collected (or are in the process of collecting) are:

• [x] Model runs for the six database-derived networks (db1-6)
  • [x] @bklein7 compiled the weight values into a spreadsheet, computed some descriptive statistics, and made the spreadsheet generalizable so that we could cut and paste data into it and automatically have the statistics computed for us. #315
    • [x] the heatmap on @bklein7's poster was created in SPSS(?) If so, we need instructions on how to reproduce it.
  • [x] We still need to compile the production rates and b values into a single spreadsheet
  • [x] expression plots have been compiled into a PowerPoint and zipped .jpg's for each run are in the repository
• [x] Model runs for the 30 random networks based on db5 #327
  • [ ] @Nwilli31 has compiled the weight values using @bklein7's spreadsheet for the three best (rand15, rand16, rand24) and three worst (rand7, rand12, rand31) random networks, but we need to compile the data for all 30 of the random networks; @Nwilli31 will not complete the rest so that she can focus on finishing her thesis.
  • [x] We still need to compile the production rates and b values into a single spreadsheet
  • [x] compile expression plots into a PowerPoint; or at least making sure zipped .jpg's are in the repository
• [x] We have the ANOVA p values for each gene in each deletion strain. Let's make sure that we have this compiled in a single spreadsheet. I know I've seen it in a PowerPoint, but let's make sure we have the spreadsheet.
• [x] @Nwilli31 compiled the LSE/minLSE ratios for both db1-6 and rand1-31 and made a nice bar chart. Let's make sure we have the original spreadsheet for this as well.
• [x] @Nwilli31 has computed the MSE/minMSE ratios for db5 #326; she needs to set up a spreadsheet with the formulas so that we can compute these by "plug-and-play":
  • [x] the rest of db1, db2, db3, db4, db6
  • [x] 3 best/3 worst rand
• [x] Degree distributions
  • [x] unweighted bar chart has been automated by a script by @bklein7 in R
    • [x] db1-6 have been done
    • [x] rand1-30 still need to be done
  • [x] weighted bar charts have been created in SPSS by @bklein7; we need step-by-step directions
    • [x] db1-6 have been done
    • [x] rand1-30 still need to be done; prioritize rand7, 12, 15, 16, 24, 31 (3 best and 3 worst)
  • [x] cumulative plots have been created in SPSS by @bklein7; we need step-by-step directions
    • [x] db1-6 have been done
    • [x] rand1-30 still need to be done; prioritize rand7, 12, 15, 16, 24, 31 (3 best and 3 worst)
  • [ ] @bengfitzpatrick notes that there should be a theoretical treatment of the degree distribution for random networks and that we shouldn't need to derive it empirically; we need to investigate this.
• [ ] Gephi graph statistics (db1-6 assigned to @maggie-oneil; rand1-31 assigned to @khorstmann, #325, rand1-20 completed so far); each statistic needs to be compiled in one worksheet of an Excel workbook so that we can compare across networks.
  • [ ] db1-6 #290
    • [ ] unweighted
      • [ ] in-degree, out-degree, total degree
      • [ ] betweenness centrality
      • [ ] closeness centrality
      • [ ] eigen centrality
      • [ ] eccentricity
    • [x] weighted
      • [x] in-degree, out-degree, total degree #328
      • [x] betweenness centrality
      • [x] closeness centrality
      • [x] eigen centrality
      • [x] eccentricity
  • [ ] rand1-31 #325
    • [ ] unweighted
      • [ ] in-degree, out-degree, total degree
      • [ ] betweenness centrality
      • [ ] closeness centrality
      • [ ] eigen centrality
      • [ ] eccentricity
    • [x] weighted
      • [x] in-degree, out-degree, total degree #329
      • [x] betweenness centrality
      • [x] closeness centrality
      • [x] eigen centrality
      • [x] eccentricity
  • [ ] we need to compute one example of each statistic by hand and compare to the Gephi results to verify that we know how Gephi is doing the computations #344 (assigned to @maggie-oneil)
    • [ ] unweighted
      • [ ] betweenness centrality
      • [x] closeness centrality
      • [ ] eigen centrality
      • [ ] eccentricity (@maggie-oneil notes that this one may be difficult)
    • [ ] weighted
      • [ ] betweenness centrality
      • [x] closeness centrality
      • [ ] eigen centrality
      • [ ] eccentricity (@maggie-oneil notes that this one may be difficult)
  • [x] we need to write a protocol that gives step-by-step instructions for how to generate the graph statistics with Gephi (issue #335)
• [x] @bklein7 has run a multiple regression in SPSS on db1-6 comparing MSE, P, d, b, ANOVA p value, in-degree, out-degree, betweenness centrality, closeness centrality, eigen centrality, eccentricity #330; double-check to confirm that Gephi stats were from unweighted or weighted networks
  • [x] we need step-by-step documentation on how this was performed
  • [x] re-run the analysis on db1-6 including the following variables:
    • [x] MSE/minMSE ratios
    • [x] weighted in-degree
    • [x] weighted betweenness centrality
    • [x] weighted closeness centrality
    • [x] weighted eigen centrality
    • [x] weighted eccentricity
    • [x] actual expression (ANOVA doesn't say whether gene was up-regulated or down-regulated; ANOVA p value could be large either because no change in expression or variance is high)
  • [ ] run the multiple regression analysis (original + variables above on three best/worst random networks rand7, 12, 15, 16, 24, 31

I will update the individual issues annotated above with the remaining tasks per issue. I will update this comment with new issue numbers as appropriate. I will make another comment on this issue giving the task assignments by person.

[kdahlquist]

So, tasks by person are:

@bklein7

• [x] #315
• [x] #330

@khorstmann

• [x] #325
• [x] #329
• [x] #335

@maggie-oneil

• [ ] #344
• [x] #290
• [x] #328
• [x] #335

@Nwilli31

• [x] #326
• [x] #327

[bklein7]

The optimized production rates and optimized threshold b values for db1-db6 have been complied here: https://github.com/kdahlquist/DahlquistLab/blob/master/data/Spring2017/15-gene_networks_analysis/db1-through-db6_compiled-optimized_parameters_BK20170405.xlsx.

The ANOVA B&H p-values for each gene in db1-db6 (useful for multiple regression analyses) have been compiled here: https://github.com/kdahlquist/DahlquistLab/blob/master/data/Spring2017/15-gene_networks_analysis/db1-through-db6_compiled-ANOVA-values_BK20170405.xlsx.

[khorstmann]

Weighted Random Networks 1-31 have been completed and uploaded into repository https://github.com/kdahlquist/DahlquistLab/tree/master/data/Spring2017/Gephi_output/Gephi_db5-derived-random-network-1-through-31_output

[khorstmann]

And Maggie wrote the step-by-step protocol for running Gephi: Gephi Upload Protocol.pdf

[maggie-oneil]

Weighted db1-6 gephi stats have been completed and uploaded to the repository https://github.com/kdahlquist/DahlquistLab/tree/master/data/Spring2017/Gephi_output/Gephi_db1-through-db6_output

[bklein7]

While working with the excel file containing the compiled, weighted Gephi statistics from db1-db6 (found here), I noticed that the betweenness centrality and eigen centrality tabs contain the same data. Looking back at the "raw" Gephi data, it looks like the betweenness centrality data is present in both tabs. So it looks like the eigen centrality data should be updated here.

[bklein7]

A new multiple regression analysis using the updated protocol specified above has been completed for db1-db6. The results have been compiled here: https://github.com/kdahlquist/DahlquistLab/blob/master/data/Spring2017/15-gene_networks_analysis/db1-db6_Multiple-Regression_BK20170329.pptx.

[maggie-oneil]

Corrected the eigencentrality sheet. New compiled and corrected file can be found here https://github.com/kdahlquist/DahlquistLab/blob/master/data/Spring2017/Gephi_output/Gephi_db1-through-db6_output/Gephi_node_stats_weighted_all_6_db_MO_02232017%20(1).xlsx

[bklein7]

Notes from the April 20th Meeting:

@bengfitzpatrick reviewed the updated multiple regression results and discussed the implications of the findings therein. Here are the main takeaways:

• The negative coefficient relating the average log-fold expression changes at 15 minutes for each node to lower MSE:min MSE ratios makes sense, as the model is best able to fit a node's expression when it is upregulated (production is high). Conversely, when degradation is the dominant part of a node's dynamics, it is harder to model. Thus, this relationship appears to be meaningful.
• Nodes that have high eigen centrality have sufficient excitation to the system to allow the node to be better fit by the model.

@maggie-oneil computed that the closeness centrality of HMO1 from db5 by hand and verified that it corresponds to the Gephi output. Also, it was verified that this is an unweighted statistic. She will check other nodes for completeness.

[kdahlquist]

Verified with @khorstmann that all files are in the repository; she is checked out of the lab for Spring 2017!

[kdahlquist]

@Nwilli31 is checked out.

[bengfitzpatrick]

running late from meeting in UH. Will be there soon.

[kdahlquist]

@maggie-oneil has been checked out for all her files.

[kdahlquist]

@Nwilli31, it seems that something got messed up and some of the changes we made to the repository didn't get saved. I need to meet with you one more time so that I can double-check some files, sorry.

[bklein7]

My work on issues #315 and #330 for the semester has been completed. See the comments on those issues for further details.

[kdahlquist]

Assigning this to myself as review requested to do a final pass through all the files.

[kdahlquist]

Haven't managed to do a pass through, but closing anyway.