WenjieDu / SAITS

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

https://doi.org/10.1016/j.eswa.2023.119619

MIT License

292 stars 48 forks source link

Loss_MIT wrong? #32

Closed lcnnnn closed 6 months ago

lcnnnn commented 6 months ago

I saw that the loss of MIT computaion in core.py was 'MIT_loss = self.customized_loss_func( X_tilde_3, inputs["X_ori"], inputs["indicating_mask"] )' ,which computed the MAE between M~3 and X_ori and differed to the paper.

WenjieDu commented 6 months ago

Hi there,

Thank you so much for your attention to SAITS! If you find SAITS is helpful to your work, please star⭐️ this repository. Your star is your recognition, which can let others notice SAITS. It matters and is definitely a kind of contribution.

I have received your message and will respond ASAP. Thank you again for your patience! 😃

Best,
Wenjie

WenjieDu commented 6 months ago

The imputation values for X are from X_tilde_3, hence it is fine.

lcnnnn commented 6 months ago

Thank you for your reply , I know that the imputation values for X are from X_tilde_3, just as the code shows:

combine X_tilde_1 and X_tilde_2

    X_tilde_3 = (1 - combining_weights) * X_tilde_2 + combining_weights * X_tilde_1
    # replace non-missing part with original data
    X_c = masks * X + (1 - masks) * X_tilde_3

but the equation in paper is L_MIT=MAE(X_c,X,I)

WenjieDu commented 6 months ago

Yes, but as I said, the code logic here is right. The parts for loss calculation in L_MIT=MAE(X_c,X,I) and MIT_loss = self.customized_loss_func(X_tilde_3, inputs["X_ori"], inputs["indicating_mask"]) both determined by indicating_mask, i.e., the term I, and they are the same. You can try to replace X_tilde_3 with X_c, the MIT_loss result will not change.

lcnnnn commented 6 months ago

Thank you for your sincerely reply! I will try it later

宁成林 @.***

---Original--- From: "Wenjie @.> Date: Fri, Jan 19, 2024 18:00 PM To: @.>; Cc: @.**@.>; Subject: Re: [WenjieDu/SAITS] Loss_MIT wrong? (Issue #32)

Yes, but as I said, the code logic here is right. The parts for loss calculation in L_MIT=MAE(X_c,X,I) and MIT_loss = self.customized_loss_func(X_tilde_3, inputs["X_ori"], inputs["indicating_mask"]) both determined by indicating_mask, i.e., the term I, and they are the same. You can try to replace X_tilde_3 with X_c, the MIT_loss result will not change.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

WenjieDu commented 6 months ago

My pleasure. If you're working on time series imputation, our recent survey work https://github.com/WenjieDu/Awesome_Imputation may be helpful to you. The benchmark code has been open source, and the preprint paper will be released soon.

lcnnnn commented 6 months ago

好的！非常感谢！

宁成林 @.***

---Original--- From: "Wenjie @.> Date: Fri, Jan 19, 2024 19:01 PM To: @.>; Cc: @.**@.>; Subject: Re: [WenjieDu/SAITS] Loss_MIT wrong? (Issue #32)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

WenjieDu commented 6 months ago

I am closing this issue because the proposed questions have been solved. If you have further concerns, reopening it will be welcome.

BTW, you can also join our PyPOTS community to discuss research related to partially-observed time series data.