search

timeseriesAI / tsai

Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
https://timeseriesai.github.io/tsai/
Apache License 2.0
5.1k stars 639 forks source link

Feature Importance & Step Importance Not working #647

Closed liyy2 closed 1 year ago

liyy2 commented 1 year ago

Hi here is my system versioning,

python          : 3.7.12
tsai            : 0.3.4
fastai          : 2.7.10
fastcore        : 1.5.27
torch           : 1.13.1+cu117
device          : 1 gpu (['Tesla P100-PCIE-16GB'])
cpu cores       : 28
threads per cpu : 1
RAM             : 251.23 GB
GPU memory      : [16.0] GB
os              : Linux-3.10.0-1160.76.1.el7.x86_64-x86_64-with-redhat-7.9-Maipo
python          : 3.7.12
tsai            : 0.3.4
fastai          : 2.7.10
fastcore        : 1.5.27
torch           : 1.13.1+cu117
device          : 1 gpu (['Tesla P100-PCIE-16GB'])
cpu cores       : 28
threads per cpu : 1
RAM             : 251.23 GB
GPU memory      : [16.0] GB

when I am running the following lines of code:

    X = np.random.randn(300, 2, 50)
    Y = np.zeros(300)
    Y[200:] = 1
    X_train , X_test, y_train, y_test = train_test_split(X, Y,  test_size = 0.2, random_state = 333)
    X, y, splits = combine_split_data([X_train, X_test], [y_train, y_test])
    tfms = [None, TSClassification()]
    dls = get_ts_dls(X, y, splits=splits, tfms=tfms) 
    learn = ts_learner(dls, XceptionTimePlus, 
            loss_func = LabelSmoothingCrossEntropyFlat(),
            # metrics = accuracy,)
            metrics= RocAucBinary())
    learn.fit_one_cycle(2, 1e-4)
    learn.step_importance()

it throws an error:

Traceback (most recent call last):                                                                                                      
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/tsai/analysis.py", line 367, in step_importance
    try: value = metric(output[0][:, 1], output[1]).item()
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 580, in roc_auc_score
    sample_weight=sample_weight,
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/sklearn/metrics/_base.py", line 72, in _average_binary_score
    raise ValueError("{0} format is not supported".format(y_type))
ValueError: continuous format is not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/gpfs/ysm/home/yl2428/YL_ABCD/ABCD/ABCD/train/trainer.py", line 177, in <module>
    learn.step_importance()
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/tsai/analysis.py", line 368, in step_importance
    except: value = metric(output[0], output[1]).item()
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 580, in roc_auc_score
    sample_weight=sample_weight,
  File "/gpfs/ysm/project/gerstein/yl2428/conda_envs/ABCD/lib/python3.7/site-packages/sklearn/metrics/_base.py", line 72, in _average_binary_score
    raise ValueError("{0} format is not supported".format(y_type))
ValueError: continuous-multioutput format is not supported