L-curve analysis with varied network sizes, optimization parameters, data

[kdahlquist]

@kdahlquist is going to run L-curve analyses in the Seaver 120 computer lab as follows:

• db5, all data, estimate all parameters (no missing values)
• db5, all data, estimate all parameters (missing values; for rest below, allow missing values)
• rand24 (good), all data, estimate all parameters
• rand7 (bad), all data, estimate all parameters
• db5, all data, estimate P, fix b
• db5, all data, fix P, estimate b
• db5, all data, fix P, fix b
• db5, wt data only, estimate all parameters
• db5-delGLN3 (14 nodes, 27 edges), wt data only, estimate all parameters
• db5-delGLN3-delZAP1 (13 nodes, 26 edges), wt data only, estimate all parameters
• db5-delGLN3-delZAP1-delGCR2 (12 nodes, 25 edges), wt data only, estimate all parameters
• db5-delGLN3-delZAP1-delGCR2-delACE2 (11 nodes, 24 edges), wt data only, estimate all parameters
• db5-delGLN3-delZAP1-delGCR2-delACE2-delSWI5 (10 nodes, 22 edges), wt data only, estimate all parameters
• db5-delGLN3-delZAP1-delGCR2-delACE2-delSWI5-delASH1 (9 nodes, 19 edges), wt data only, estimate all parameters

Note that because of the problem of getting slightly different results on different computers, the results are not 100% comparable, but there isn't anything we can do about that right now.

[kdahlquist]

• Running 15-genes_28-edges_db5_Sigmoid_estimation_no-missing-values_L-curve.xlsx on SEA120-14, starting on 6/5/17 at 4:34 PM.
• Running 15-genes_28-edges_db5_Sigmoid_estimation_missing-values_L-curve.xlsx on SEA120-15, starting on 6/5/17 at 4:42 PM.
  • Checked both at 3:30 PM on 6/6 and they were on the last of the 16 alpha values. Will check again tomorrow to see if finished.
  • UPDATE: both finished and zipped results can be found on the DahlquistLab repo: https://github.com/kdahlquist/DahlquistLab/tree/master/data/Summer2017/L-curve

[kdahlquist]

The good news is that the L-curves I ran on the first two files took about 28 hours to run (instead of a week!), so I won't need to monopolize all the computers, I can just run my tests sequentially. Link to the results files has been appended to the previous post. I plan to run 2 analyses per day until I exhaust the permutations I want to try.

[kdahlquist]

Runs have been completed. Data and notes can be found in the Dahlquist Lab repository here:

https://github.com/kdahlquist/DahlquistLab/tree/master/data/Summer2017/L-curve

The compiled data with the run ID, alpha, LSE, and penalty values can be found here:

https://github.com/kdahlquist/DahlquistLab/blob/master/data/Summer2017/L-curve/compiled_L-curve_data_2017-06-21_updated.xlsx

[bklein7]

An updated version of the R script for generating L-curves has been uploaded to the Dahlquist Lab repository: https://github.com/kdahlquist/DahlquistLab/blob/master/R_scripts/L-Curves.Rd. Although this code is still a work in progress, the following features have been successfully added:

• The user can now manually adjust the x and y axes.
• The user can opt not to plot alpha value labels.
• The user can plot up to seven L-curves on the same graph (this limitation stems from the fact that only 7 distinct colors have been hard-coded into the script).

(Updated originally posted on September 5th to issue #354.)

[bklein7]

A newly updated version of the R script for generating L-curves has been uploaded to the Dahlquist Lab repository: https://github.com/kdahlquist/DahlquistLab/blob/master/R_scripts/L-Curves.R. This version includes the following updates:

• File type changed to .R
• Changes were made to the color options hard-coded into the script when plotting multiple L-curves on the same graph. Specifically, yellow was replaced by grey, and the line type was set to change to dashed after exhausting the first 7 colors (allowing for 14 unique L-curves on a single graph).
• Individual data points now appear as small, solid circles.

Using this iteration of the script, new L-curves were produced for all analysis sets previously generated in this issue (#351). All L-curves were plotted on a fixed scale. The PowerPoint file containing these results can be accessed here: https://github.com/kdahlquist/DahlquistLab/blob/master/documents/L-Curve-Analysis_Varied-Optimization-Data-Size_BK20170909.pptx.

[kdahlquist]

Loose ends from today's meeting:

• plot the three "middle" random networks and db5 on same graph
• plot the three smallest error and middle random networks with db5 on same graph
• create a mega plot with all 38 L-curves, repeating colors is OK.

[bklein7]

The latest version of the R script for generating L-curves can be found here: https://github.com/kdahlquist/DahlquistLab/blob/master/R_scripts/L-Curves.R. This update added hard-coded color/pattern combinations for 42 distinct lines when plotting multiple L-curves on the same graph.

An updated L-curve analysis PowerPoint presentation can be found here: https://github.com/kdahlquist/DahlquistLab/blob/master/documents/L-Curve-Analysis_Varied-Optimization-Data-Size_BK20170909.pptx. This version addresses the following issues from last week's meeting:

• The three "middle" random networks and db5 are plotted on the same graph.
• The three "middle" and "best" random networks are plotted alongside db5 on the same graph.
• A "mega plot" containing all 38 L-curves (with distinct colors + line types) was created.
• A plot tracking the optimization of the production rates for db5 (fixb) was added.

Finally, I did some research on AIC curves that we can discuss during today's meeting (see #360).

[bklein7]

The "L-curves" R script has been commented out and can now be considered complete until further features are proposed: https://github.com/kdahlquist/DahlquistLab/blob/master/R_scripts/L-Curves.R.

[bklein7]

The L-curve powerpoint presentation was updated to include proper labeling of the step-down experiment graph (LSE vs. parameter #): https://github.com/kdahlquist/DahlquistLab/blob/master/documents/L-Curve-Analysis_Varied-Optimization-Data-Size_BK20170909.pptx.

[kdahlquist]

This has been completed.