Test files Audit

[kdahlquist]

We need to perform an audit of the "test_files" directory.

• [x] we need to know which files are being used for actual tests in the code
• [x] which files are there for illustrative purposes (for users)
• [x] which files don't serve a purpose and should be removed.

For any files we keep, we need to check that

• [x] they conform to the "new" (which isn't so new anymore) way of creating input workbooks
• [x] there are no errors in the files (like the decimal point problem with the time designation on certain inputs

Assigning this a 0.5 priority because other issues have priority right now. However, when we tackle this, we want to be thorough and detail-oriented so that we don't have to do it again moving forward. Paying off some more technical debt.

[im-deepfriedwater]

Within GRNmap/test_files/matlab_codes/sampleTests there are matlab test files that are useful as examples for beginning to write tests, however we are beyond that level as we already have a whole test suite to look back on for examples. These are subject for removal.

[im-deepfriedwater]

GRNmap\test_files\matlab_codes\calculationTests\newLSETests\GeneralLSETest.m

Is a remnant from last semester as we tried to work on it in tandem. It is no longer needed as a correct and completed version exists at GRNmap\test_files\matlab_codes\calculationTests\GeneralLSETest.m

[im-deepfriedwater]

Directories left for inspection for keeping

• [x] GRNmap\test_files\matlab_codes\data_samples
• [x] GRNmap\test_files\matlab_codes\deleted_strains_test
• [x] GRNmap\test_files\estimation_tests
• [x] GRNmap\test_files\forward_tests
• [x] GRNmap\test_files\initialize_arrays_test
• [x] GRNmap\test_files\lse_tests
• [x] GRNmap\test_files\matlab_codes
• [x] GRNmap\test_files\matrix_dimensions_test
• [x] GRNmap\test_files\MSE_tests
• [x] GRNmap\test_files\optimization_diagnostic_test
• [x] GRNmap\test_files\pertubation_tests
• [x] GRNmap\test_files\plots
• [x] GRNmap\test_files\review_requested
• [x] GRNmap\test_files\sixteen_tests

[im-deepfriedwater]

@kdahlquist is there a heuristic I could use for checking which test files are no longer needed?

[im-deepfriedwater]

folders of test functions in matlab_codes I am looking through

• [x] calculationTests
• [x] dataStructureTests
• [x] excelTests
• [x] sampleTests
• [x] testStructs
• [x] utils

[im-deepfriedwater]

Test files subject for removal/discussion:

• [x] GRNmap/test_files/perturbation_tests/math_post_L-curve_corrected/4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_no-graph_test1_LCurve_4_output.xlsx

  • It seems we only need the input sheets in this folder and having an output file in here for one of the input files seems extraenous

• [x] GRNmap/test_files/lse_tests/LSETest.m

  • This test is incomplete. It doesn't seem that another test is written that tests what LSETest.m originally was meant to test for.
  • [x] GRNmap/test_files/lse_tests/4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_no-graph.xlsx
    • Is called only by LSETest.m thus if LSETest.m is removed this input sheet is applicable for removal as well. We can remove the entire folder as lse_tests only contains the single unused input sheet.

• [x] GRNmap/test_files/MSE_tests/dHAP4_15_gene_network_deletion_added_input_KD_20160126_output_with_check.xls.xlsx

  • Is another output file that seems extraneous.

• [x] GRNmap/initialize_arrays_test/

  • GRNmap/test_files/matlab_codes/dataStructureTests/InitializeArraysTest.m tests a MM input sheet in here.
  • However, there is a sigmoid input sheet that does not get tested. The test script does not seem to have capability to test sigmoid input sheets.

• GRNmap/test_files/perturbation_tests has:

  • [x] math_post_L-curve_corrected
  • [x] to_be_reformatted
  • [x] with_manual_calculations

• [x] GRNmap/test_files/plots has graph plots that seem to be archival.

[kdahlquist]

We can talk about this list of files at the meeting.

[dondi]

• sixteen_tests should have both inputs and outputs for archival purposes (even though we still need to run the outputs over when bugs are fixed)
• other test files that are seen to be unused can be pruned
• output files that are not used can be pruned (except for 16 tests and data samples)

[dondi]

Bullet-by-bullet decision on audit

• GRNmap/test_files/perturbation_tests/math_post_L-curve_corrected/4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_no-graph_test1_LCurve_4_output.xlsx looks prunable
• GRNmap/test_files/lse_tests/LSETest.m needs to be finished
• GRNmap/test_files/MSE_tests/dHAP4_15_gene_network_deletion_added_input_KD_20160126_output_with_check.xls.xlsx prune
• GRNmap/test_files/initialize_arrays_test/ sigmoid variant can be removed; rename test file so that it actually states its purpose as an array initialization test
• GRNmap/test_files/plots prune

[dondi]

Also add a README.md at the test_files folder to document the contents for future GRNmappers.

[im-deepfriedwater]

Removed the files as detailed by the discussion of the checklist I posted. Next, is to create an issue to finish the LSETest.m and also finishing checking the rest of the test files I couldn't get to from the previous work sessions.

UPDATE: Issue for LSETest.m created at #376

[im-deepfriedwater]

I've also added a readme.md within the test_files folder. There was already instructions on how to run the test suite within matlab_codes that show up as a readme.txt. I will most likely go in and reformat it as a markdown and leave it in the /test_files/matlab_codes folder and make note of its availability from the readme.md in test_files.

[im-deepfriedwater]

2nd Wave of folders/files subject for removal or are of interest:

• [ ] test_files/review_requested/101216
  • This remains from when we were told to create output files for @bengfitzpatrick to compare.
• [ ] test_files/forward_tests
  • This is exclusively being used for allReadInputSheetTest.m which we discussed earlier.
• [ ] test_files/estimation_tests
  • Same issue as the folder above.
• [ ] test_files/data_samples
  • This is used for illustrative purposes and is denoted as so in the first readme.md in the root directory.
• [ ] test_files/perturbation_tests/with_manual_calculations
  • This is not used anywhere at all and is subject for removal. However, there might be value in that these were manual calculations? Not sure, and would like some insight to the origin of these files.
• [ ] test_files/matlab_codes/sampleTests
  • there are matlab test files that are useful as examples for beginning to write tests for general matlab learnings, however we are beyond that level as we already have a whole test suite to look back on for examples. Additionally, it is more common during work sessions to seek examples online rather than look through these for help. These are subject for removal.
• [ ] test_files\matlab_codes\calculationTests\newLSETests\GeneralLSETest.m
  • Is a remnant from last semester as we tried to work on it in tandem. It is no longer needed as a correct and completed version exists at GRNmap\test_files\matlab_codes\calculationTests\GeneralLSETest.m

[im-deepfriedwater]

With the 2nd wave the first part of the test file audit will have been completed. The next step is to double check that the input workbooks we keep are valid and conform to our new format as denoted in the initial issue.

[kdahlquist]

Round 2 test file audit notes:

• test_files/review_requested/101216
  • 🗑 Delete; we can always revert if we remember why we needed them
• test_files/forward_tests
  • Since other files are being read and checked, this sounds superfluous so can be deleted.
• test_files/estimation_tests
  • Ditto.
• test_files/data_samples
  • Move data_samples to the top level in order to separate these from files that are actually used by the test suite.
• test_files/perturbation_tests/with_manual_calculations
  • See if we can track down the data in this sheet to some unit test fixture. That will simply confirm that the sheet has served its purpose. Either way this can be deleted.
• test_files/matlab_codes/sampleTests
  • Green light to remove these files except for the readme; we need to track down what this readme references (likely the 4-gene 6-edge minimal cases) then this file can be moved to that location.
• test_files\matlab_codes\calculationTests\newLSETests\GeneralLSETest.m
  • 🗑 🚚

[im-deepfriedwater]

I've removed the appropriate files, what's next is to take care of these left over issues.

• test_files/perturbation_tests/with_manual_calculations
  • See if we can track down the data in this sheet to some unit test fixture. That will simply confirm that the sheet has served its purpose. Either way this can be deleted.
• test_files/matlab_codes/sampleTests
  • Green light to remove these files except for the readme; we need to track down what this readme references (likely the 4-gene 6-edge minimal cases) then this file can be moved to that location.

[im-deepfriedwater]

Couldn't find an issue related to the data sheet in test_files/perturbation_tests/with_manual_calculations that explained its origins. Removing as of now.

[im-deepfriedwater]

Spent quite a bit of time crawling through commit histories. The test_files/perturbation_tests/with_manual_calculations/readme.txt lists files that I cannot find anywhere and have not seen it while going about 2 years back within the commit history. The readme.txt does reference graphs from the plots folder that we denoted was subject for removal.

Overall, I believe the readme should be removed as after reading through and not finding other files, it does not seem to have a purpose in the repository.

[im-deepfriedwater]

@dondi For the README.md that goes within test_files am I listing one by one the purposes of each folder?

For example I'd write,

initialize_arrays_test folder provides 3 test input sheets used by test_files/matlab_codes/dataStructureTests/InitializeArraysTest.m

deleted_strains_test folder provides test input sheets used by test_files/matlab_codes/excelTests/DeletedStrainTest.m

etc

[kdahlquist]

@jtorre39 can you point me to where the readme is so that I can take a look at it one last time? Thanks.

[im-deepfriedwater]

@kdahlquist Yes, it is located at GRNmap/test_files/matlab_codes/sampleTests/readme.txt

[dondi]

Test files audit now moves on to verifying that the unculled files comply with the latest input and output sheet formats (https://github.com/kdahlquist/GRNmap/wiki/How-to-format-the-input-file-for-GRNmap-v1.4-and-above, https://github.com/kdahlquist/GRNmap/wiki/How-to-interpret-the-output-file-for-GRNmap).

Brandon has already audited the 16-tests input sheets to that end; just the output sheets remain for those. @kdahlquist will post this information later.

[kdahlquist]

I'm pasting the text of the readme file GRNmap/test_files/matlab_codes/sampleTests/readme.txt here:

The scenario is a four gene network. Two genes are purely self regulated; two others feedback to each other. The data was created by executing two forward simulations, one with all four and one with a single gene deleted. This pair gives us the two data sheets (wt and dcin5).

Input_4_gene_inverse.xls is the input sheet for the estimation run. Input_4_gene_inverse_estimation_output_archived.xls is the output sheet. Name has been changed so that when you run the code, you won't overwrite this file. Input_4_gene_inverse_estimation_output_archived.mat is the output matlab binary file. Name has been changed so that when you run the code, you won't overwrite this file. figure_1, figure_2, and figure_3 are saved output. once again I have added the phrase _archived to the name. figure_4 is generated in the output, but there is no saved .jpg file (bug?).

also included is the file Input_4_gene_testing.xlsx, which was used to generate the forward (model-generated) testing data. In this file, you can see "the right answers" to the estimation problem, in the production_rates and network_weights sheets. fix_b is set to one, so the b's are not estimated in this example.

I still need to look at this and figure out the place where this information needs to reside. In the meantime, since the text is here, you can get rid of the file itself.

[im-deepfriedwater]

Got it!

[im-deepfriedwater]

Directories left for validating input sheet formats:

• [x] GRNmap\test_files\matlab_codes\deleted_strains_test
• [x] GRNmap\test_files\initialize_arrays_test
• [x] GRNmap\test_files\matrix_dimensions_test
• [x] GRNmap\test_files\MSE_tests
• [x] GRNmap\test_files\optimization_diagnostic_test
• [x] GRNmap\test_files\pertubation_tests
• [x] GRNmap\test_files\sixteen_tests

[im-deepfriedwater]

Many input sheets have been having an extra row or two. These rows were called sheet and deletion in the optimization_parameters sheet. They were not mentioned in the input sheet format guide for 1.4 and above. Additionally, they do not exist in the sixteen_tests sheets so they have been removed from the input sheets I've gone over.

[im-deepfriedwater]

Is there a reason why certain data cells are highlighted within GRNmap\test_files\MSE_tests\dHAP4_15_gene_network_deletion_added_input_KD_20160126.xlsx?

[im-deepfriedwater]

Within optimization_diagnostic_test/optimization_diagnostic_under_100_iterations_test.xlsx in the network_weights sheet the data cells are formatted to numbers instead of general. Should they be changed to general?

[im-deepfriedwater]

It might be worth to note in the wiki if data cells should be formatted to general or numbers for additional clarification.

[dondi]

• The sheet row can be deleted
• deletion has been renamed to strain so if a strain row already exists, deletion can be removed
• On the General vs. Number-formatted cells, write a test that reads and runs two workbooks that are identical except for the formats. Report back on whether the results differ.

[dondi]

On the issue of highlighted cells, these are remnants of pre-missing-data workarounds where missing cells were populated with average values then highlighted so that users would know which cells were missing.

Nothing needs to be done with the files themselves; however a note in the new README explaining this will help retain this information for the future.

[im-deepfriedwater]

Took some time but I checked each of the input sheets! Anything with an extra row has been corrected. I didn't catch any other oddity than the ones mentioned above.

[kdahlquist]

That is great! Thanks for doing this!

[kdahlquist]

I closed a duplicate issue #177. In that issue, @juansc and @trixr4kdz created a wiki page with documentation of the unit tests. We should revisit this wiki and update.

We should look into writing a script that would automatically generate testing documentation as has been discussed for the GRNsight team.

[dondi]

This issue can finally be closed. Will write the testing documentation task(s) up as a new issue.