JGCRI / cassandra

Human-earth system multi-scale model coupling framework
Other
6 stars 3 forks source link

Generic component for models implemented as python modules #39

Closed rplzzz closed 5 years ago

rplzzz commented 5 years ago

Right now we have to create a whole new component for each model we want to support, but for models implemented as python modules these components are going to be functionally identical. Namely, they are going to declare a few capabilities (typically in init, though this could also be done in finalize_parsing). Then the run_component() method will look something like this:

import somemodule

rslt = somemodule.main(self.params)

self.addresults('somecapability', rslt)

It should be possible to create a generic component that takes the name of the module to import as one of its parameters and automatically calls the main function and adds the results as capabilities. This could save prospective users some effort writing a bunch of boilerplate for each model they want to run.

If we do that, we may have to come up with a way for users to whitelist their component modules so that we don't reintroduce the problem where a rogue configuration file could run arbitrary code on a user's machine.

calebbraun commented 5 years ago

I am not sure this generalization will actually simplify things. There is no standard for how to run models implemented as python modules, so it won't be as simple as calling somemodule.main(). For example, Tethys runs upon initializing a Tethys object. Xanthos is first initialized and then the execute method is called. Just accounting for how to run these two would require adding more parameters to our configuration.

Furthermore, this raises the question of where to implement any logic connecting one of these components to another. A likely-common case comes up with Xanthos, where we want to use temperature and precipitation data from another component if it exists, and if not load the data from somewhere else.

Considering writing boilerplate code for these components is relatively simple compared to what may be a complex configuration set up, I would recommend passing on this issue. Components running models written in R, however, are a bit more complex and might warrant re-visiting this idea.

rplzzz commented 5 years ago

Both of those things sound reasonable to me.