Closed dennisbader closed 1 year ago
Yes, please, will be very helpful, it's one of the advantages of TFT.
@jacobblb As a quick and dirty workaround to have some insights into the variable selection weights until the feature is available one can:
tft.predict(n=Your horizon)
darts.models.forecasting.tft_module.py
print(encoder_sparse_weights.mean(axis=1))
- for past variables selection (including the past of target)
print(decoder_sparse_weights.mean(axis=1))
- for future variables selectionthen do someting like:
def plot_cov_selection(filename, title = "Variable importance"):
variable_selection = np.loadtxt(filename)
cov_percentage = variable_selection.mean(axis=0).round(3) * 100
# your covariates names, for example created with datetime_attribute_timeseries
cov_names = covs.columns
# instert target value for past covariates
if len(cov_names) != len(cov_percentage):
cov_names = ["Past of Target"] + [cov_names.values[i] for i in range(len(cov_names))]
print(cov_names)
plt.bar(cov_names, cov_percentage)
plt.title(title)
plt.xlabel("Variable", fontsize=12 )
plt.xticks(rotation=45, ha='right', )
plt.ylabel("Variable importance in %")
plt.show()
plot_cov_selection('value_selection_encoder.txt', title = "Variable importance") plot_cov_selection('value_selection_decoder.txt', title = "Variable importance")
To get something like
![variable_importance_past](https://user-images.githubusercontent.com/67061560/193576135-06d0e766-2f83-4706-9f0b-01af774b7e5f.png)
![a2c4b711-196c-4053-b413-87663fc23da3](https://user-images.githubusercontent.com/67061560/193576649-8066c70b-0c89-439a-bbc6-f6ffe90d259b.png)
@doaa-altarawy Similar for the attention weights:
In tft_model.py in forward pass:
print(attn_out_weights.squeeze().sum(axis=1).numpy().tolist(), file=open("attn_out_weights.txt", "a"))
Read file
import ast
import numpy as np
with open('attn_out_weights.txt', 'r') as f:
attention_matrix = np.array([ast.literal_eval(line) for line in f])
My shape is (439, 24, 192)
Average over the number of forecasts
attention_matrix = attention_matrix.mean(axis=0)[:, :168]
and I also just take the 168 past attention values, you could maybe also roll over them to include them at the beginning.
Plot mean attention - similar like in initial TFT paper
plt.plot( attention)
plt.xticks(range(0, attention_matrix.shape[1], attention_matrix.shape[0]) )
plt.xlabel("Time steps in past")
plt.ylabel("Attention")
plt.show()
attention_matrix_avarege = attention_matrix #.mean(axis=0)
plt.figure(figsize=(20,10))
plt.imshow(attention_matrix_avarege, cmap='hot', interpolation='nearest')
plt.xticks(range(0, attention_matrix_avarege.shape[1], attention_matrix_avarege.shape[0]) ) plt.xlabel("Time steps in past") plt.ylabel("Horizon") plt.show()
![e67118ec-ae0e-4668-b7d1-64440d90dd3d](https://user-images.githubusercontent.com/67061560/193628851-d51ca852-ba49-4549-9f8c-6fb8451f04d3.png)
7. Now you can inspect how much attention your model pays to your input values (Values) for each forecast timestamp (Query) to get a kind of seasonal analysis.
@MagMueller this looks very nice!
We are about to introduce a submodule darts.explainability
which will contain some explainability features in 0.22.0. Once this is out, would you be willing to help us work on an explainability module for the TFT model? It could be doing things very much along the lines of what you showed here (e.g., producing similar plots, or returning TimeSeries of attention weights over historical time steps).
@hrzn Thank you! Yes I would love to help you with the TFT.
Just let me know what you need and how you work together etc... See you:)
@hrzn Thank you! Yes I would love to help you with the TFT.
Just let me know what you need and how you work together etc... See you:)
@MagMueller Here's perhaps how it could look. You could create a class TFTExplainer
in the darts.explainability
submodule (we currently have one other explainer called ShapExplainer
in there, which currently only works for RegressionModel
s).
This new class would receive an already-trained TFTModel at creation, and could offer e.g. methods to get/plot variable selection weights. The class should probably inherit from ForecastingModelExplainer
, which means that it should also implement the explain()
function, whose purpose is to "explain" a particular (set of) forecasts, and return an ExplainabilityResult
, which contains TimeSeries
representing "explanation values" for given forecasting horizons and components. There might be a way to use the attention weights as the "explanation values" there.
If needed, we can also change/adapt this ForecastingModelExplainer
super-class, as currently it's only used in one implementation (ShapExplainer
) and may not be 100% appropriate for TFT. We can also add new getter methods to the TFTModel
class itself to get its inner parameters cleanly.
So to recap, I think it would be nice if users would be able to run code looking as follows
from darts.models import TFTModel
from darts.explainability import TFTExplainer
my_model = TFTModel(...)
my_mode.fit(...)
explainer = TFTExplainer(my_model, ...)
# look at (or get) variable selection weights
explainer.plot_vsw(...)
# get ExplainabilityResult containing "explanation" TimeSeries, similar to your plot showing Attention over time
expl_result = explainer.explain(my_series, my_past_covs, ...)
explanation = expl_result.get_explanation(component="my_target_dimension_of_interest", horizon=10)
# (not 100% sure we need to keep horizon here ^)
I'm pinging @dennisbader who is most knowledgeable about our TFTModel and @dumjax who implemented the ShapExplainer
in case they have inputs to this discussion.
We can discuss the design here on this issue, but also don't hesitate to ping me in private by sending a DM on the Darts gitter (or Discord).
Thanks!
@hrzn Sorry, I'm really overloaded right now and haven't taken the time yet. What is the Discord's name?
Hi, since I am working with a TFT model right now and I am interested in understanding the connections the model has learned, I have created a draft for the class based on @MagMueller's code and @hrzn suggestions about where to put the code and how to structure it.
There are still a few open points that are not clear to me like how to get the variable names or how to better structure the code to make it more robus for the different cases TFT can handle.
Here is the draft PR: https://github.com/Cattes/darts/pull/1/files
Thanks @Cattes ! I added a couple of high level comments. Do you think it would be possible to (as much as possible) comply with a usage like this:
from darts.models import TFTModel
from darts.explainability import TFTExplainer
my_model = TFTModel(...)
my_mode.fit(...)
explainer = TFTExplainer(my_model, ...)
# look at (or get) variable selection weights
explainer.plot_vsw(...)
# get ExplainabilityResult containing "explanation" TimeSeries, similar to your plot showing Attention over time
expl_result = explainer.explain(my_series, my_past_covs, ...)
explanation = expl_result.get_explanation(component="my_target_dimension_of_interest", horizon=10)
# (not 100% sure we need to keep horizon here ^)
That means we need plot_vsw()
and explain()
(returning TimeSeries similar to what @MagMueller implemented).
If you feel you have something looking solid enough, you can already open a Draft PR (even if it's not perfect) in Darts repo's master branch. Thanks!
Ping @dumjax @dennisbader
Hi @hrzn
I have added a PR with an implementation of the TFTExplainer
with the usage as you suggested: https://github.com/unit8co/darts/pull/1392
I have added an "Explainability" section to the 13-TFT-examples.ipynb
notebook to show how to use it: Link to the notebook in my PR
I am looking forward to comments on how to improve it :+1:
Solved by #1392.
We should think about adding interpretable outputs for TFTModel as done in the original paper.
Mainly: