search

patrick-kidger / torchcde

Differentiable controlled differential equation solvers for PyTorch with GPU support and memory-efficient adjoint backpropagation.
Apache License 2.0
422 stars 44 forks source link

About how to use on seq2seq works #39

Open caisikai opened 2 years ago

caisikai commented 2 years ago

Hi, patrick! First, thank you very much for share your work for us! It is very useful! And I have a question. Is it work on the regression tasks, such as seq2seq. I found all examples are classification and specifically cdeint only return terminal value( initial value also but not used ) to linear map to category. So if I want to use it to seq2seq, such as fill nan, it is somehow diffcult. How or where can I make some modifications to adapte it on seq2seq task. Thank you very much!

patrick-kidger commented 2 years ago

Seq2seq is possible, e.g. just use the final value of a CDE as the initial condition of an ODE. You can return multiple values from cdeint by passing a vector of integration times.

pruthamodak commented 2 years ago

Hi @patrick-kidger , First of all, thank you so much for sharing your work with us. I have a doubt regarding the CDE and ODE blocks. Could you please give an example of how exactly the architecture of the CDE and ODE blocks will look like in the Seq2Seq regression task if possible? Thank you very much!

Saltsmart commented 2 years ago

Hi @patrick-kidger , First of all, thank you so much for sharing your work with us. I have a doubt regarding the CDE and ODE blocks. Could you please give an example of how exactly the architecture of the CDE and ODE blocks will look like in the Seq2Seq regression task if possible? Thank you very much!

@pruthamodak I have created a repo TANODE for studying how to use ODE-based models in timeseries forecasting. It shows ODE-RNN (see paper) models in Recurrent Arch, Seq2Seq Arch and Variational AutoEncoders. CDEs are in TODO list.

You can change Args["arch"] in run_models.py to "Recurrent", "Seq2Seq" or "VAE", and Args["using"] should be "ODE_RNN". Please forgive me for not writing README until I finish my research!

chooron commented 2 years ago

Hi @patrick-kidger , First of all, thank you so much for sharing your work with us. I have a doubt regarding the CDE and ODE blocks. Could you please give an example of how exactly the architecture of the CDE and ODE blocks will look like in the Seq2Seq regression task if possible? Thank you very much!

@pruthamodak I have created a repo TANODE for studying how to use ODE-based models in timeseries forecasting. It shows ODE-RNN (see paper) models in Recurrent Arch, Seq2Seq Arch and Variational AutoEncoders. CDEs are in TODO list.

You can change Args["arch"] in run_models.py to "Recurrent", "Seq2Seq" or "VAE", and Args["using"] should be "ODE_RNN". Please forgive me for not writing README until I finish my research!

Hello! The repo TANODE is not found in the github.

Saltsmart commented 2 years ago

Hi @patrick-kidger , First of all, thank you so much for sharing your work with us. I have a doubt regarding the CDE and ODE blocks. Could you please give an example of how exactly the architecture of the CDE and ODE blocks will look like in the Seq2Seq regression task if possible? Thank you very much!

@pruthamodak I have created a repo TANODE for studying how to use ODE-based models in timeseries forecasting. It shows ODE-RNN (see paper) models in Recurrent Arch, Seq2Seq Arch and Variational AutoEncoders. CDEs are in TODO list. You can change Args["arch"] in run_models.py to "Recurrent", "Seq2Seq" or "VAE", and Args["using"] should be "ODE_RNN". Please forgive me for not writing README until I finish my research!

Hello! The repo TANODE is not found in the github.

Sorry for changing this repo private (our research team is testing some unstable features)

Here is a public branch: Forecasting with Neural ODEs. I add a README file.