How to run statistics?

[MichaelCurrie]

What’s the command to run statistics? I want to be able to step through the code. In your answer, feel free to make use of the example_data/30m_wait folder to test stats like you had mentioned to me back on 29 July 2014.

I've started a document to detail my current understanding of how it is run, so feel free to add to that if that is easier.

Thanks very much Jim

[JimHokanson]

The command to run statistics eh? In my old lab we had something called the "Jim button" that we joked would run experiments for us and give us the answer. The button actually setup a lot of the experiment for you so that you didn't have to constantly be pressing buttons in the right order to start and stop things and create new trials. Soon everyone wanted a "Jim button."

Forget about the command, your answer is that there is not a significant difference at the p = 0.05 level.

Anyway ...

All the code you need should be in the file: seg_worm.testing.stats.t001_oldVsNewStats

Unfortunately, I never actually made the file so that you could run it in one sweep. Feel free to give it a try. Alternatively I'll modify the code so that it can actually run in one sweep, but that is going to have to wait until tomorrow night :/

The critical missing part is in: hgenerateOldStats (also in the file, as denoted by my convention of h) see Approach 3

I forget why but I couldn't find this code anywhere in their files that actually did all of the steps together. Perhaps they were just pressing a series of buttons, which of course I can't actually track. There is also some flexibility in how you add things (new controls and new non-controls and in what order).Some of these approaches just don't work with their code. This took a lot of playing around with to get something that was somewhat plausible. I don't think I ever actually compared these stats to the published stats, just my implementation versus their implementation.

Best of luck.

Jim

[MichaelCurrie]

Haha! I can see why such a button would be most useful!!

[MichaelCurrie]

OK, so actually my first problem comes with

base_path = 'C:\Backup\Dropbox\worm_data\segworm_data\features\gene_NA\allele_NA';
hist_path = 'C:\Backup\Dropbox\worm_data\segworm_data\histograms';

I don't appear to have the segworm_data folder either on the vestigial DropBox folder references, or on the new example_data Google Drive folder.

Can you perhaps move the segworm_data folder from (hopefully) your computer into the example_data Google Drive folder and then I'll change the path here? Thanks!

Also you had mentioned that I'll need 10 experiment and control files, which I have saved in the "30m_wait" folder in the example_data folder. (not sure why it's named "30m_wait" but that's another issue :))

Also, to your question if it's better to make it run in one sweep, yes that would be ideal (as we just discussed in person). Thanks!

[JimHokanson]

I changed the code. It still needs some work to summarize the results but it should be somewhat straight forward now to run ...

[MichaelCurrie]

OK this works now! Incredible that you got the stats code sped up from 7+minutes to about 5 seconds.