search

openml / openml-r

R package to interface with OpenML
http://openml.github.io/openml-r/
Other
95 stars 37 forks source link

simple implementation to upload tasks #431

Closed bakopyan closed 5 years ago

bakopyan commented 5 years ago

added a simple function for uploading tasks. please review and let me know what you think!

lintr-bot commented 5 years ago

R/writeOMLTaskXML.R:5:1: style: Trailing whitespace is superfluous.

​  
^~

R/writeOMLTaskXML.R:11:1: style: Trailing whitespace is superfluous.

​  
^~

R/writeOMLTaskXML.R:14:1: style: Trailing whitespace is superfluous.

​  
^~

R/writeOMLTaskXML.R:23:1: style: Trailing whitespace is superfluous.

​  
^~

R/writeOMLTaskXML.R:24:5: style: Place a space before left parenthesis, except in a function call.

​  if(!is.null(evaluation_measures)) {
    ^

R/writeOMLTaskXML.R:28:1: style: Trailing whitespace is superfluous.

​  
^~

tests/testthat/test_server_uploadOMLTask.R:9:1: style: Trailing whitespace is superfluous.

​    
^~~~
pfistfl commented 5 years ago

I think the estimation_procedure might be outdated @giuseppec ? If so we should not expose it here.

PhilippPro commented 5 years ago

I tried it out and it doesnt work. Can you give a simple example @bgrt?

bakopyan commented 5 years ago

@PhilippPro, sorry now it should work:

ds = getOMLDataSet(41158L)
task.id = uploadOMLTask("Supervised Classification", ds$desc$id, ds$target.features, 2L)

this should work, like in the test. you don't need the getOMLDataSet(), it's just for automatically get the target.features from already existing dataset on openML.

PhilippPro commented 5 years ago

@bgrt Your example also works for me. The example that didnt work for me (regression):

ds = getOMLDataSet(41539)
task.id = uploadOMLTask("Supervised Regression", ds$desc$id, ds$target.features, 7L)

Also the 2L (here 7L) at the end is kind of complicated and obscure. Can you put a reasonable default for that and make this optional?

bakopyan commented 5 years ago

@PhilippPro

there was a problem with estimation.procedure... openML returns a server error if the provided estimation.procedure doesn't correspond to the given task.type.

now I'm extracting the est.name by estimation.procedure from listOMLEstimationProcedures() and check if the est.name is valid for given task.type and then extract the corresponding est.id.

let me know if it works for you :)

PhilippPro commented 5 years ago

No, it does not. Complete Code:

devtools::install_github("bgrt/openml-r")
.rs.restartR()
library(OpenML)
ds = getOMLDataSet(41539)
task.id = uploadOMLTask("Supervised Regression", ds$desc$id, ds$target.features, 7L)
#Downloading from 'https://www.openml.org/api/v1/json/tasktype/list' to '<mem>'.
#Downloading from 'https://www.openml.org/api/v1/json/estimationprocedure/list' to '<mem>'.
#Uploading task to server.
#Uploading to 'https://www.openml.org/api/v1/task'.
#Fehler in doHTTRCall(method, url = url, query = list(api_key = conf$apikey),  : 
#  ERROR (code = 622) in server response: Input value does not match allowed values in foreign column.
#  problematic input: target_feature

You could include this in the test file also.

bakopyan commented 5 years ago

hmm... this seems bizarre. I get the same error (with same arguments from your example) by using the REST API provided by openML, with generated xml from writeOMLTaskXML().

I tried to upload a task on a gina dataset by wilful providing incorrect target_feature and also get the same error.

the error occurs every time, the target_feature is not class or Class. Seems to me like there're some restrictions in openML REST API for naming target_feature. Would it be better to move the issue to the openML issues?

PhilippPro commented 5 years ago

If you are sure that this is the problem, yes...

pfistfl commented 5 years ago

Using the website form form does work though? @bgrt Additionally, there is a closed issue which might help: https://github.com/openml/OpenML/issues/917 If everything still fails: Wanna create an issue in the main OpenML repo?