Open jonc125 opened 5 years ago
~old to-do list was here~
Tests:
To discuss: how much we want a spec 'language' for fitting (perhaps just a config file essentially, but supporting comments unlike JSON!), and how much just allow pints code. Cf sandboxing discussion in #61.
I think I am in favour of just pints code (I thought that was what we concluded in last face to face meeting). Please feel free to edit this list of pros and cons:
Pros:
Cons:
*Maybe for really simple cases (least squares fit) learning Pints is harder than learning a simple spec command like Aidan's prototype, but it's one of those cases where we probably want to make people understand what they are doing behind the scenes - so maybe no bad thing to force them to look at the pints code.
We'd need to keep careful track of the version of Pints that was used quite explicitly at the top of the 'spec' (code), since the interface is still in flux. Whereas a 'spec parser' could be updated just once to keep track of latest pints release API. So if we allow pints code we'd need to make sure we can get hold of old versions to run the fitting. (when we discussed I think you both said numpy API was very stable so not as much of an issue there).
We'd need a 'local executable' for people to play with to write their pints code (but this would be useful anyway), then some way of ensuring versions of everything were compatible on server
Would we need some way of saying what annotated model parameters are available in python - some new WebLab wrapping Python interface to the data and the simulations for humans to use which hasn't existed before?
Sandboxing needed for Joe Bloggs to upload fitting specs.
I've added some thoughts on what a fitting spec needs to capture in the issue description - please edit!
Let's have another chat about this via skype then? I think I'm also in favour of fitting scripts. It gets complex really fast
@jonc125 I've updated the top post above with a tentative to-do. Now wondering about the approach for running fits:
Something like this? Neither?
Either approach should work, depending on how much you want separate libraries. The fitting runner would need to know a fair bit about FC, so it might make more sense to combine them.
There will be some improvements to fc-runner in any case to use the new weblab-fc protocol parser as a library to extract the protocol interface (now needing to include outputs as well - so this bit would overlap with a fitting use case) and send it to the Web Lab front-end when new protocols are uploaded. Currently it uses a rather hacky bit of code partially parsing the protocol! It will also then need to use cellmlmanip instead of pycml to determine model/protocol compatibility - again overlapping with processing needed for fitting. So these common features would sit either in cellmlmanip or weblab-fc to be used by other components.
Or should we define some common language? Some kind of manifest file that everyone can read that tells you what an entity is, what it needs, etc.? Would open the door to future additions!
(where by "language" I mean one or two standardised fields in an xml or json file)
Well, we could, but for models & protocols this information is already defined internally to those documents (or associated RDF files for models in due course). So you'd still need some software to extract it from there to a new common format.
For models it's going to end up separately though, presumably in a COMBINE archive. So it'd make sense for every component (model, protocol, data, fitting spec) to be a COMBINE archive too?
They already are COMBINE archives. Doesn't change the fact that for e.g. protocols the canonical list of outputs is in the protocol file itself, and would need to be copied into another file in the archive.
Yeah. I suppose parsing just the "most interfacy" bits of the model interface section isn't hard though, so might be good to have this outside of FC. I really like the idea of having it modular (even if just in principle), so that we could theoretically support other FC implementations (e.g. other domains, other simulation types)
(Updated) to-do list
74 Design a Python/PINTS based "fitting spec" that will work for a sine wave fit, i.e. some kind of interface through which a python script can:
tau
vsV
ErrorMeasure
orLogPDF
in PINTSController
, passing in the requiredmethod
controller.sampler().set_special_setting(123)
Things that need to be captured by a fitting spec are:
See also #74.