Model with random features shouldnt be used for stacking

[pplonski]

Right now a model with random feature inserted (part of feature selection procedure) can be used for stacking - it shouldnt.

[rushiv0609]

Hi @pplonski , I am new to contributing to open source projects. I want to contribute and solve this issue, any help from where I can start solving it would be great.

[pplonski]

Hey @rushiv0609 ! Great to hear that you would like to contribute! :) BTW, how have you found this project?

The issue:

• during the features selection procedure there is one model trained with random feature inserted. (docs about features selection)
• the model with random feature shouldn't be used in the next steps, for example during hill_climbing, stacking, and ensembling

In all three steps, the model with random feature should be excluded. Give me a few minutes, I will find in the code where it should be.

[pplonski]

@rushiv0609 some tips on how to start:

• to exclude in stacking step, please check the _perform_model_stacking() method in base_automl.py
• to exclude in hill climbing step, please check the get_hill_climbing_params in tuner/mljar_tuner.py file
• to exclude in the ensemble, please check get_oof_matrix() in ensemble.py

To check if model has inserted random feature:

if model.params["preprocessing"].get("add_random_feature", False):
    print("there is random feature")

If you will need further help, please let me know. Happy to help!

[rushiv0609]

Thanks a lot @pplonski . I was just browsing through good first issues tags and came across this project.

[rushiv0609]

When I run AutoML.fit(), I get the following runtime error, I cannot find anywhere how to resolve this

* Step default_algorithms will try to check up to 3 models
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x000002577A5FE3C8>>
Traceback (most recent call last):
  File "D:\Python36\lib\tkinter\__init__.py", line 3507, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x000002577C962630>>
Traceback (most recent call last):
  File "D:\Python36\lib\tkinter\__init__.py", line 3507, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x000002577C9626A0>>
Traceback (most recent call last):
  File "D:\Python36\lib\tkinter\__init__.py", line 3507, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
2020-09-26 07:24:15,387 concurrent.futures ERROR exception calling callback for <Future at 0x2570013f0f0 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback:

[pplonski]

It is fixed in 0.10.3