aimclub / FEDOT

Automated modeling and machine learning framework FEDOT
https://fedot.readthedocs.io
BSD 3-Clause "New" or "Revised" License
632 stars 86 forks source link

"ValueError: I/O operation on closed file" in Github test runner #765

Closed gkirgizov closed 2 years ago

gkirgizov commented 2 years ago

Several PRs on github are failing now because of this error.

Seems related to these problems in pytest

UPD: Problem is because pytest tampers with stdout/stderr. Anything that uses them, can cause this problem (logging, print, tqdm in our case)

UPD2: Now the only problem is tqdm that must be disabled completely in certain tests

UPD3: more related things

gkirgizov commented 2 years ago

One of the failing tests shows this:

 test/unit/remote/test_remote_composer.py:56: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fedot/api/main.py:157: in fit
    self.api_composer.obtain_model(**self.params.api_params)
fedot/api/api_utils/api_composer.py:62: in obtain_model
    return self.compose_fedot_model(api_params_dict, composer_params_dict, tuner_params_dict)
fedot/api/api_utils/api_composer.py:205: in compose_fedot_model
    best_pipelines = gp_composer.compose_pipeline(data=train_data)
fedot/core/composer/gp_composer/gp_composer.py:98: in compose_pipeline
    opt_result = self.optimiser.optimise(objective_function)
fedot/core/optimisers/gp_comp/gp_optimiser.py:189: in optimise
    with self.timer, tqdm(total=self.requirements.num_of_generations,
/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/tqdm/std.py:1107: in __init__
    self.sp = self.status_printer(self.fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
file = <tqdm.utils.DisableOnWriteError object at 0x7fc84aeee5b0>
    @staticmethod
    def status_printer(file):
        """
        Manage the printing and in-place updating of a line of characters.
        Note that if the string is longer than a line, then in-place
        updating may not work (it will print a new line at each refresh).
        """
        fp = file
        fp_flush = getattr(fp, 'flush', lambda: None)  # pragma: no cover
        if fp in (sys.stderr, sys.stdout):
            getattr(sys.stderr, 'flush', lambda: None)()
>           getattr(sys.stdout, 'flush', lambda: None)()
E           ValueError: I/O operation on closed file.

So the problem is really related to closed stdout