AutoResearch / autora

Automated Research Assistant
https://autoresearch.github.io/autora/
MIT License
39 stars 6 forks source link

Experimentalist Consistency #585

Open chadcwilliams opened 10 months ago

chadcwilliams commented 10 months ago

There is still some variations between the experimentalists, especially with what they return. Most of them (maybe all except the assumption experimentalist, which needs updating) will return a pandas DF when given conditions as a pandas DF. But the behaviour varies when providing conditions as a numpy array. For example, when getting an np.array as input, the novelty sampler outputs a pandas DF, but the falsification sampler outputs a np.array. In the tutorial we explicitly state that experimentalists MUST output a pd.DF, and I think we should adapt the experimentalists to do this no matter the input. You'll notice in the first 2 basic tutorials that there are discrepancies in the code when dealing with different experimentalists because of this and it is confusing.

chadcwilliams commented 10 months ago

Let's highlight this problem in the tutorials. i.e., rely on pandas arrays, especially as outputs but in general.