search

ray-project / tune-sklearn

A drop-in replacement for Scikit-Learn’s GridSearchCV / RandomizedSearchCV -- but with cutting edge hyperparameter tuning techniques.
https://docs.ray.io/en/master/tune/api_docs/sklearn.html
Apache License 2.0
465 stars 52 forks source link

Can't suppress warning messages through standard python methods #255

Closed RNarayan73 closed 1 year ago

RNarayan73 commented 1 year ago

Hello,

I am getting warning messages below even with verbose=0 or 1:

2022-11-07 12:04:56,052 INFO tensorboardx.py:267 -- Removed the following hyperparameter values when logging to tensorboard: {'enc__target': BayesianTargetEncoder(columns=['Symbol', 'CandleType', 'h1CandleType1', 'h2CandleType1'], prior_weight=3, suffix=''), 'enctimecyclicity': CyclicalFeatures(drop_original=True)} (_Trainable pid=39836) C:\Anaconda3\envs\skl_py310\lib\site-packages\category_encoders\target_encoder.py:92: FutureWarning: Default parameter min_samples_leaf will change in version 2.6.See https://github.com/scikit-learn-contrib/category_encoders/issues/327 (_Trainable pid=39836) warnings.warn("Default parameter min_samples_leaf will change in version 2.6." (_Trainable pid=39836) C:\Anaconda3\envs\skl_py310\lib\site-packages\category_encoders\target_encoder.py:97: FutureWarning: Default parameter smoothing will change in version 2.6.See https://github.com/scikit-learn-contrib/category_encoders/issues/327

which I have tried to suppress using standard python approaches like:

from warnings import filterwarnings
filterwarnings(action='ignore', category=FutureWarning)   # works for optuna but not ray
filterwarnings(action='ignore', module='target_encoder.py', lineno=92)
filterwarnings(action='ignore', module='target_encoder.py', lineno=97)
filterwarnings(action='ignore', module='tensorboardx.py', lineno=267)

Any suggestions on how to resolve this?

Regards Narayan

Yard1 commented 1 year ago

The issue here is that the changes to the environment (filterwarnings in your case) are not propagated to the Trainable Ray Actors, which run in separate Python processes. Ray will capture all stdout and stderr from those and pipe it to the driver process.

You can initialise Ray before using tune-sklearn with ray.init(log_to_driver=False) argument, which will prevent any output from the Actors from showing up. This is probably not what you want if using Ray for other purposes, but if you are only using it for tune-sklearn, it should be sufficient.

RNarayan73 commented 1 year ago

Thanks, Does this mean that the verbose argument is irrelevant? I want warning messages to appear to know if something is wrong, but having acknowledged them, I would like to shut specific ones down. Sounds like that can't be done with tune-sklearn as it stand? Narayan

Yard1 commented 1 year ago

The verbose argument concerns the verbosity of Tune's output. Unfortunately, I don't think there is an easy way of supressing warnings in Actors with tune-sklearn. Would you be interesting in contributing that feature? I am happy to offer guidance.

RNarayan73 commented 1 year ago

@Yard1 thanks for your reply. I am not very familiar with the architecture of ray itself and am only using it due to tune-sklearn's api that allows me to use it instead of or alongside alternative hyperparameter search algorithms in my current project. Appreciate you offer of guidance and when I do delve deeper into ray down the line, I'll be glad to take up your offer.