intel-analytics / analytics-zoo

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
https://analytics-zoo.readthedocs.io/
Apache License 2.0
16 stars 3 forks source link

jenkins: chronos tests failed on mac #215

Closed pinggao187 closed 3 years ago

pinggao187 commented 3 years ago

`____ TestAutoTrainer.test_fit_lstmfeature ____

self =

def test_fit_lstm_feature(self):
    input_feature_dim = 11  # This param will not be used
    output_feature_dim = 2  # 2 targets are generated in get_tsdataset

    search_space = {
        'hidden_dim': hp.grid_search([32, 64]),
        'layer_num': hp.randint(1, 3),
        'lr': hp.choice([0.001, 0.003, 0.01]),
        'dropout': hp.uniform(0.1, 0.2)
    }
    auto_trainer = AutoTSTrainer(model='lstm',
                                 search_space=search_space,
                                 past_seq_len=hp.randint(4, 6),
                                 future_seq_len=1,
                                 input_feature_num=input_feature_dim,
                                 output_target_num=output_feature_dim,
                                 selected_features="auto",
                                 metric="mse",
                                 loss=torch.nn.MSELoss(),
                                 logs_dir="/tmp/auto_trainer",
                                 cpus_per_trial=2,
                                 name="auto_trainer")
    auto_trainer.fit(data=get_tsdataset().gen_dt_feature(),
                     epochs=1,
                     batch_size=hp.choice([32, 64]),
                     validation_data=get_tsdataset().gen_dt_feature(),
                   n_sampling=1

)

../test/zoo/chronos/autots/test_experimental.py:95:


../zoo/chronos/autots/experimental.py:165: in fit scheduler_params=scheduler_params ../zoo/chronos/autots/model/auto_lstm.py:141: in fit scheduler_params=scheduler_params, ../zoo/orca/automl/auto_estimator.py:185: in fit self.searcher.run() ../zoo/automl/search/ray_tune_search_engine.py:181: in run self.logger.run(tf_config, tf_metric) ../zoo/automl/logger/tensorboardxlogger.py:115: in run self._write_hparams(new_config[key], newmetric[key], name=key.replace("/", "")) ../zoo/automl/logger/tensorboardxlogger.py:120: in _write_hparams exp, ssi, sei = hparams(hparam_dict, metric_dict) /Users/arda/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboardX/summary.py:102: in hparams v = make_np(v)[0] /Users/arda/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboardX/x2num.py:20: in make_np return check_nan(np.array(x)) /Users/arda/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboardX/x2num.py:12: in check_nan tmp = np.sum(array)

<__array_function__ internals>:6: in sum ??? /Users/arda/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2229: in sum initial=initial, where=where) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ obj = array(['WEEKDAY(datetime)', 'IS_AWAKE(datetime)'], dtype=', method = 'sum', axis = None, dtype = None, out = None kwargs = {'initial': , 'keepdims': , 'where': } passkwargs = {} def _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs): passkwargs = {k: v for k, v in kwargs.items() if v is not np._NoValue} if type(obj) is not mu.ndarray: try: reduction = getattr(obj, method) except AttributeError: pass else: # This branch is needed for reductions like any which don't # support a dtype. if dtype is not None: return reduction(axis=axis, dtype=dtype, out=out, **passkwargs) else: return reduction(axis=axis, out=out, **passkwargs) > return ufunc.reduce(obj, axis, dtype, out, **passkwargs) E TypeError: cannot perform reduce with flexible type /Users/arda/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/fromnumeric.py:90: TypeError` jenkins link: http://10.239.176.111:18888/job/ZOO-NB-UnitTests-2.4-PYTHON-MAC/387/consoleFull PR: [AutoTSTrainer with tuning of model hyperparams and data preprocessing intel-analytics/analytics-zoo#4105](https://github.com/intel-analytics/analytics-zoo/pull/4105)
pinggao187 commented 3 years ago

@TheaperDeng pls have a look

TheaperDeng commented 3 years ago

pls upgrade tensorboardx to 2.1 and rerun. thx

pinggao187 commented 3 years ago

re-run http://10.239.176.111:18888/job/ZOO-NB-UnitTests-2.4-PYTHON-MAC/389/