Merging case_runner crop fails

[AdrianSosic]

Hi @jcmgray,

I noticed that crop.reap() fails when the there are several crops created by a Harvester that uses a case_runner instance. The reasons seems to be that case_runners return pandas DataFrames instead of xarray DataSets (unlike combo_runners) and hence the merge function is called on the resulting DataFrame object, which fails accordingly:

TypeError: merge() got an unexpected keyword argument 'compat'

My current workaround is to merge all cases manually so that I can avoid creating multiple crops in the first place.

[jcmgray]

Thanks for the issue @AdrianSosic - sounds like df.merge should not be called there. If you have a minimal working example then I can look into fixing it?

[AdrianSosic]

Sure =)

import pandas as pd
import xyzpy

def dummy_function(**args):
    return 0

cases1 = pd.DataFrame(
    [
        (1, 2),
        (3, 4),
    ],
    columns = ('a', 'b')
)

cases2 = pd.DataFrame(
    [
        (5, 6),
        (7, 8),
    ],
    columns = ('a', 'b')
)

runner = xyzpy.Runner(dummy_function, var_names=['Dummy_Value'])
harvester = xyzpy.Harvester(runner, engine='joblib')

crop1 = harvester.Crop()
crop1.sow_cases(fn_args=cases1.columns, cases=cases1.values.tolist())
crop1.grow_missing()
crop1.reap()

crop2 = harvester.Crop()
crop2.sow_cases(fn_args=cases2.columns, cases=cases2.values.tolist())
crop2.grow_missing()
crop2.reap()

[jcmgray]

I think sow_cases was introduced initially only as a method for sow_samples which indeed is meant to return a DataFrame but anyhow should be fixed by https://github.com/jcmgray/xyzpy/commit/c8f872df77015dcb830a2b02cbc60e017b9e8e8b, let me know if you run into any problems!

[AdrianSosic]

Works like a charm! At least I haven't seen any problems since then, so I think this issue can be closed. Thanks a lot =)