search

ant-research / EasyTemporalPointProcess

EasyTPP: Towards Open Benchmarking Temporal Point Processes
https://ant-research.github.io/EasyTemporalPointProcess/
Apache License 2.0
239 stars 25 forks source link

Predicting a single event sequence #33

Closed HaochenWang1243 closed 2 months ago

HaochenWang1243 commented 2 months ago

Hi. Thank you for your work. I have a question regarding the prediction task. In most TPP papers, I see the experiments are conducted on datasets containing many sequences (of different lengths). Sometimes, a single long sequence is used, such as Financial Transaction Dataset mentioned in the RMTPP paper. I want to check my understanding on how "one-step-ahead prediction" of these two types of data is done since I'm not too sure if my understanding of the code is correct (even though this is such a high-level thing...):

  1. For datasets of multiple sequences, is the model trained, validated and tested on separate sequences? If so, is validation and testing done by, say, evaluating how well the model predicts the time and type of event 2 from 1, 3 from 2,1, and 4 from 3,2,1 and so on, where the real events history from event 1 to event k-1 is given to predict event k?
  2. For datasets of a single sequence, I think the train, validation and test data are consecutive segments of that one sequence. As a result, when evaluating the model on validation or test set, the RNN hidden state generated from its preceding part of the sequence has to be used as initial state of the model. From what I see in the NHP implementation, https://github.com/ant-research/EasyTemporalPointProcess/blob/53b7b7f33fcdbb5fef53d43c4cada3bb9df39eb6/easy_tpp/model/torch_model/torch_nhp.py#L166 the model seem to start from the initial state whatsoever. So I want to ask how the model handles prediction of a single sequence.

I feel this question may be more of a general one than something specific to EasyTPP, but this has confused me for a long time, and I do want to clarify and use EasyTPP given how easy-to-use it is. Thank you for your help.

iLampard commented 2 months ago

For datasets of multiple sequences, is the model trained, validated and tested on separate sequences?

Yes.

If so, is validation and testing done by, say, evaluating how well the model predicts the time and type of event 2 from 1, 3 from 2,1, and 4 from 3,2,1 and so on, where the real events history from event 1 to event k-1 is given to predict event k?

Yes

For datasets of a single sequence, I think the train, validation and test data are consecutive segments of that one sequence. As a result, when evaluating the model on validation or test set, the RNN hidden state generated from its preceding part of the sequence has to be used as initial state of the model.

For a single sequence, normally we truncate it into many shorter sequences, split them into train, valid and test set and feed them batch-wise (therefore init the state at each batch) into the model. . Otherwise, the sequence might be too long for the models, especially for attention-based models.