Merge MCMC parameter estimation toolbox

[murrayrm]

Vipul has now finished his PhD and so I'm assuming we can merge this code into the master repository on BuildACell.

@vipulsinghal02 Is this ready to go? If not, can you post here what needs to be added/fixed?

[murrayrm]

There's a bunch of junk in this repository (crash files, etc) that should be removed before we merge.

@vipulsinghal02 Can you fix these when you have a chance?

[vipulsinghal02]

Ok to the merging into the main branch. Will remove the crash logs and other junk and add the relevant regular expressions to the gitignore file. Should probably also include all the png files / any data files etc.

On the broader question of what needs to be tied up before the merge can be done, I think I have to go through the code base and separate out the core toolbox + a set of minimal and well documented examples, and just merge that, and have all the examples and use cases still under development as a separate branch (there are a bunch of examples I was working on to try out calibration-corrections at various parameter sharing configurations / levels of modeling.. all partially done so not worth including in the main branch.)

Timeline wise: have a few interviews with labs at ASTAR over the next week or so, but after that I can take some time to look at sorting through this, writing out a documentation file / user guide etc. Should take a ~4 weeks working on this part time.

[murrayrm]

Sounds like a plan. I'd like to get something minimal merged quickly, so that others can access it. Is there a way to create a branch on your end with just the set of things that are already working and then we can do a pull request off of that.

[vipulsinghal02]

Looks like we will have to be a bit careful here, since along with the mcmc_simbio code, there are also a bunch of changes in the main toolbox (a lot of code cleanup, added the ATP regeneration mechanism, etc). I would suggest making a new branch in the BuildACell/txtlsim repo and calling it something like "from_vipul" or "mcmc_simbio_and_other_changes". Then I can do a pull request from my repo to that branch, and we can look at the changes in that branch, and only merge what we want / see if everything is OK to merge. What do you think?

More generally, I think we should discuss how to our collaborative workflow should be structured. I like the fork-pull request methodology, with branches for every feature in my fork. However, there are a couple of things to clarify if we try this:

1. If, over a few months, I open a bunch of branches, make the relevant changes to each branch, and remerge into my fork's master branch, my version of the toolbox will have advanced quite a bit. So now after a few months, if we try to do a pull request, there is presumably a lot of discussion to be had: one discussion for each branch / feature that was added. This is the situation we are in right now.
2. An alternative to this is that after every branch that gets completed on my end, and merged into my master, I do a pull request to the BuildACell/txtlsim repo. Can do this, but then we might decide that some feature that I add into my fork is not really needed or wanted in the BuildACell/txtlsim repo. So now if I merge my branch in, from that point on, my repo can no longer be pulled into the BuildACell repo, right? I am guessing that for the foreseeable future, a situation where I make a feature that does not get incorporated into the BuildACell version is probably unlikely, so this is not really a problem. Actually, with the MCMC toolbox, a version of this situation has already arisen: we want to merge the mcmc_toolbox, but not necessarily any of the changes I made to the rest of txtlsim toolbox.

[vipulsinghal02]

Another thing we can try is: I check out the BuildACell/txtlsim repo again, and add (a minimal version of) the mcmc_simbio toolbox to this new working tree, and we just do a pull request off that. That should be clean, but I think that means that we can no longer pull from my current fork, since the mcmc_toolbox files might be in conflict... Not sure, we could try, and this could be a learning experience for me. :-)

If we do try this, then I would just treat this new forked repo of mine as my main repo, and carefully open individual branches for each small change, and merge them one my one, and all the while we could be pulling each new merge into BuildACell, with discussion / explanation to document everything... Could be clean and nice, and can be done at a slower pace, since the mcmc_simbio is already in the BuildACell at this point, but it will take more effort (i think..)

[vipulsinghal02]

Another update (sorry for the spam!). Here is a list of the files that got modified/added/deleted in my fork wrt the BuildACell/txtlsim repo that are not part of the mcmc_simbio toolbox (and would therefore need discussion if they change anything significant (and a few of them do.. like txtl_transcription, txtl_translation, the ATP regeneration etc)). It is really not as bad as I had feared, so I think this should be doable quickly. (Tue onwards I have some time to do this)

Will add the Crash logs, png files, html files etc to the .gitignore and delete / untrack, and apart from the txtl_transcription, txtl_translation, the ATP regen, new parameters in the config files, and a couple of other minor things (all of which I can say a few words about), there aren't any other changes that can affect the core functioning of the BuildACell/txtlsim repo.

M .gitignore A Crash_logs_14_10_17_22_01_18/matlab_crash_dump.17071-1 A Crash_logs_14_10_17_22_01_18/matlab_crash_dump.2495-1 A Crash_logs_14_10_17_22_01_18/matlab_crash_dump.38232-1 A Crash_logs_14_10_17_22_01_18/matlab_crash_dump.6005-1 A Crash_logs_14_10_17_22_01_18/matlab_crash_dump.628-1 A auxiliary/globalize_params.m A auxiliary/listmodelparts.m M auxiliary/plotCustomSpecies2.m M components/txtl_prom_p70.m M config/E30VNPRL_config.csv M config/E30_config.csv M config/E32_config.csv M config/E9_config.csv A config/Emcmc2017_config.csv A config/Emcmc2018_config.csv M core/txtl_addreaction.m M core/txtl_enzyme_resource_degradation.m M core/txtl_mrna_degradation.m D core/txtl_plot_gui.fig M core/txtl_reaction_config.m M core/txtl_runsim.m M core/txtl_transcription.m M core/txtl_translation.m A cov_extract2.jpg D doc/extractTODOlist D doc/txtl_template.m M examples/geneexpr.m A examples/geneexpr_test_regen_mode.m M html/txtl_tutorial.html D html/txtl_tutorial.png D html/txtl_tutorial_01.png D html/txtl_tutorial_02.png D modules/paramest/paramestCostFcn.m D modules/paramest/parameter_estimation_patternsearch.m D sysID/MCMC_example.m D tests/RNAdegradation_test.m D tests/atp_test.m D tests/geneexpr_DNAsweep.m A txtl_de_init.m M txtl_init.m M txtl_tutorial.m A txtl_tutorial_1.m

[vipulsinghal02]

@murrayrm

Just created a new pull request from my minimal_mcmc_simbio branch to a branch of the same name in the BuildACell\txtlsim repo. I think we should do this first, and then only merge into master once we are sure everything is fine. For now people can simply checkout the minimal_mcmc_simbio branch to explore.

Vipul #43