Begin converting the codebase to utilize our new data structure

[trixr4kdz]

From issue #302, @bengfitzpatrick mentioned that the heart of the code is in the least squares error function. The loops in that code should be adjusted to now acknowledge the new data structure with possible missing cells.

[trixr4kdz]

Before I forget, this loop specifically is what needs to be addressed when using the new structure.

• Looking at line 82, you take the log2 of the solution to the general_network_dynamics function (which was outputted by ode45), transpose it (we're basically correcting the position of the values in the matrix since the ode solver gives a transposed matrix), subtract a corresponding (as in similar timepoint) column of the original user data, then square each entry in the matrix of the result. The result is then aggregated for each timepoint so we end up with a single column vector errorMat.
• Also, from what @bengfitzpatrick said, it might be helpful to note that the loop goes like this: iterate by strain, then by timepoint, then by replicates.

[azinge]

I've attached comments to my fork's lse, general_least_squares_error, and runForwardSimulation indicating where the majority of the changes concerning the new data structure will need to be made. The change seems relatively small in scope, so tests will be necessary to ensure that no unwarranted changes accompany our adapting of the new data structure.

[trixr4kdz]

Some questions/comments for the meeting:

• From what I understand from line 87, we are taking values by columns from the expression data and performing some operations on it. E.g., if the original expression data structure was

-0.3763   -0.3763   -0.3763   -0.7067   -0.7067   -0.7067   -0.9872   -0.9872   -0.9872 
-0.2277   -0.2277   -0.2277   -0.4080   -0.4080   -0.4080   -0.5464   -0.5464   -0.5464
 0.2693    0.2693    0.2693    0.4159    0.4159    0.4159    0.4975    0.4975    0.4975  
-0.2909   -0.2909   -0.2909   -0.5798   -0.5798   -0.5798   -0.8558   -0.8558   -0.8558

then this part of the code d(:,log2FC(qq).t(iT).indx(iF)) would take just the first column.

But since we wouldn't have the same dimensions for each smaller matrix (of a particular timepoint) in the new structure, how should we be aggregating the errorMatStrain column vector? I'm probably thinking about this wrong, but I'm finding it a bit tricky to make that similar column vector to the original when looping through the new structure since the nested matrices have varying sizes now. Also, there might be a problem if there is a missing value at the first/last position of a matrix if all the other matrices don't have missing values, e.g., if 1 matrix only has values in indices [1, 2] but the rest of the matrices for that timepoint have values in [1,2,3].

• I remember that we are supposed to save the raw data into the GRNstruct with the new compressed structure, would we call it rawExpressionData and the cleaned up data will be expressionData?

[im-deepfriedwater]

Discussion about our new loop structure took place.

[kdahlquist]

Complete, ready for release.