Some datasets are not working when i run auto sklearn classifier via flask.

[shabir1]

Some datasets are not working when i run auto sklearn classifier via flask

When I train a dataset on auto sklearn classifier it works fine and is completed within the given time but when I train the same dataset using flask (using API and make auto sklearn classifier as service) it hangs, training is not completed at all. Example dataset: EEG Eye State Data Set I think it is due to libsvm_svc it hangs

[eddiebergman]

Hi @shabir1,

We are generally unfamiliar with flask and it's the first we've heard of this issue, could you provide more information, code, or anything that could help us reproduce this?

• Where can we get this dataset
• What system are you running this on?
• What version of auto-sklearn are you using
• Why do you suspect it's due to libsvm_svc? Do you have logs you could share?
• What parameters are you using with autosklearn?
• Have you tried disabling svm models?
• What's different about your environment than the one you run autosklearn on?
• There's been issues with /tmp before, do you have access to this folder on the system running auto-sklearn?

[shabir1]

@eddiebergman
Where can we get this dataset? Dataset link: https://datahub.io/machine-learning/eeg-eye-state/r/eeg-eye-state.csv or https://archive.ics.uci.edu/ml/datasets/EEG+Eye+State What system are you running this on? Linux What version of auto-sklearn are you using 0.13 Why do you suspect it's due to libsvm_svc? Do you have logs you could share? I exclude libsvm_svc using exclude parameter in contractor then it worked. What parameters are you using with autosklearn?

autosklearn.classification.AutoSklearnClassifier( 
            time_left_for_this_task=3600,
            memory_limit=3072,
            ensemble_size=10,
            ensemble_nbest=10,
            max_models_on_disc=10
)

What's different about your environment than the one you run autosklearn on? I use auto sklearn as a service and create the end point for train the model using flask. There's been issues with /tmp before, do you have access to this folder on the system running auto-sklearn? Yes

[eddiebergman]

• The Dataset seems very normal, it's unlikely to be dataset specific.
• I have no idea why libsvm_svc hangs only when being used by flask. My only guess is that whatever machine is running libsvm_svc needs some underlying numeric library updated. This is indicated by the fact it runs fine locally and also in our automated tests. sklearn handles this, calling lower level C libraries for the numerical calculations of these algorithms and is not something we can manage. If this is of critical importance to you to solve, my best suggestion is to isolate that algorithm on wherever you are running it. This would mean having an endpoint lead to the sklearn version of SVC and see if it still hangs. If it does, then it's a sklearn problem with the machine you are running the service on. If it doesn't hang, then it's an issue on our side with our code.

Could I ask if it's a cloud provided machine or a local server? If it is a cloud one, would you have any more specifics you could share such as the instance name, size etc...