harmonize construction behavior

[gidden]

There are a number of functions which "construct" dataframes, e.g., concat() and append(). How these work should be harmonized to a common data model.

At the moment, concat() requires that all input be pyam.IamDataFrames. append() on the other hand, can take any castable object such as pd.DataFrame or even file names which are read in at execution time.

The pros and cons deal with user experience vs. explicit interfaces using native datatypes. It is not clear which outweighs the other, so further comments from devs and users is appreciated.

So let us assume you have data in two files a.csv and b.csv that you want to combine. The options would look as follows:

User Experience

pyam.IamDataFrame('a.csv').append('b.csv')

pyam.concat(['a.csv', 'b.csv'])

Explicit Interfaces

pyam.read_file('a.csv').append(pyam.read_file('b.csv'))

pyam.concat([pyam.read_file('a.csv'), pyam.read_file('b.csv')])

The trickiness here is that our constructor currently supports two primary castable types: pd.DataFrames and files. If we choose "Explicit Interfaces", that would suggest that we no longer support reading files directly in pyam.IamDataFrame.__init__ .

Thoughts? @znicholls @danielhuppmann others?

[danielhuppmann]

strongly in favour of the "User Experience" option, see https://github.com/IAMconsortium/pyam/pull/190#issuecomment-462260875

note that the IPCC notebooks at https://github.com/iiasa/ipcc_sr15_scenario_analysis all use pyam.IamDataFrame('<file>'), so that would introduce an incompatibility going forward...

[znicholls]

I'd be in favour of the user experience too and refactoring the back end so they all get sent to the same format_data (or similar) pipeline which handles all the different possibilities

[danielhuppmann]

I think that having merged #199, we have implicitly decided on the way forward.