Confusion on UQP ->primitives -> elements

[dww100]

Looking at the submoduel organization: I thought we had moved from uqp/analysis/ to primitives/analysis/ etc. But seems we haven't. Are we now just going to rename uqp to elements and bring collate inside that?

[raar1]

That's because I haven't merged the primitives/elements stuff in yet. Only what was needed for travis to work (including pep8 changes) has been moved into master.

There are fundamental issues I've discovered while trying to make a BaseElement class that we'll need to discuss. In particular this relates to the logging of element application.

Essentially, sampling elements require a campaign object to act on, therefore they always are (or should be) logged by that campaign object. However, analysis elements are far more ambiguous in that regard. They generally apply to pandas dataframes (or csv files etc) that were the output of a particular campaign's runs (usually after collation) but they don't clearly modify the campaign object itself. The campaign is merely the logical place, for now, to log analysis steps (and the output files they may or may not produce) but this logging is conceptually different from that done for the sampling step.

You can see that this would not be obvious any more in the event that we create an algorithm with more than one campaign object, and which performs analysis over the results of several campaigns. Doing the analysis with multiple dataframes in python would be trivial and transparent in such a case, but the logging would not be. And if it doesn't always make sense to log analysis in a campaign object then I'm not sure the mechanic makes sense in individual cases either - apart from being currently convenient.

Do we have an external, overarching log for analysis? I don't like this either.

But in any case it's taking me longer to reach a conclusion on how best to treat VVUQ elements programmatically, since they don't even share common logging characteristics.

For now I will simply merge what I have done with regards to refactoring the uqps -> elements. We'll discuss the rest later.

[dww100]

Would some form of queue object (https://docs.python.org/3/library/queue.html) work? In the first instance a Campaign would instigate it's own but have a method to switch to an external one (logging proposed switch).

Might need unique IDs associated with Campaigns.

[raar1]

OK, so in the end I found a way to do the logging (at least, a much better way). The campaign object now handles a lot more of the burden.

There is now a BaseElement, BaseSamplingElement and BaseAnalysis architecture. Elements have names, versions and categories ("sampling", "analysis", "comparison" etc).

The biggest change I have made is to (partially) sort out the mess with sampling elements. All sampling elements are now generators that create runs. You then use the campaign object's add_runs() method to draw (however many) runs from those samplers. Some samplers can be infinite, and some are finite - the new code deals with that.

My reason for this change is that talks we've had in VECMA meetings over the last few weeks have suggested we may sometimes want to limit how many runs are generated in the dictionary at a time (for example) and with these generators you just add as you go along. They also make it easier to decorate the output runs with the output of other generators, such as we may wish to do to solve the category variables issue.

The refactored code is sitting in the primtives_to_elements branch, and if it seems ok to you then I want to merge with dev. I would then consider that work done, with respect to the mid-December release.

[dww100]

I'm not quite sure what you mean when you say "make it easier to decorate the output runs with the output of other generators" - pseudocode of what you mean will probably help.

[dww100]

Reviewed this now and really like it.

Does the collation need updating to fit in now?

[raar1]

Collation has now been converted to a VVUQ element.

I've opened a pull request (#50) to merge this all into dev now. Tests pass on travis.