Open janvanrijn opened 6 years ago
I added two unit tests:
the first one is deactivated, as the python api is indeed not ready for this yet (although it's not a bad thing, it doesn't seem to miss any obvious fields) the second one runs nicely (had to add something to server for that)
I just wanted to have a look at this issue because getting this right is crucial to the success of OpenML. However, I can't find your tests. Could you please post a link to the tests you added so that I can extend them?
Great catch.
They are actually in the PR that will be deleted.
@unittest.skip('Function _create_description_xml does not write server fields yet.')
def test_run_description_deserialize_serialize(self):
openml.config.server = self.production_server
run_id = list(openml.evaluations.list_evaluations(function='predictive_accuracy',
flow=[7707], size=1).keys())[0]
run_xml_orig = openml._api_calls._perform_api_call('run/%d' %run_id)
run_obj_orig = openml.runs.functions._create_run_from_xml(run_xml_orig)
run_xml_prime = run_obj_orig._create_description_xml()
# TODO: _create_description_xml does not add run id, uploader, etc
self.assertEqual(run_xml_orig, run_xml_prime)
def test_run_description_deserialize_serialize(self):
model = DecisionTreeClassifier(max_depth=1)
task = openml.tasks.get_task(119)
run_orig = openml.runs.run_model_on_task(task, model)
run_orig = run_orig.publish()
run_new = openml.runs.get_run(run_orig.run_id)
# evaluations might not be aligned (original run has some locally generated measures,
# downloaded run might have or have not obtained server evals)
run_orig.evaluations = None
run_new.evaluations = None
run_orig.fold_evaluations = None
run_new.fold_evaluations = None
run_orig.sample_evaluations = None
run_new.sample_evaluations = None
self.assertEqual(run_orig._create_description_xml(), run_new._create_description_xml())
We should merge these two functions to the development branch.
Make sure that the tests ensure that functions _create_description_xml() and _create_run_from_xml() are each others exact complements.
e.g.,