HEmile / storchastic

Stochastic Automatic Differentiation library for PyTorch.
GNU General Public License v3.0
180 stars 5 forks source link

Implement variable-sized sequence sampling #71

Open HEmile opened 4 years ago

HEmile commented 4 years ago

Implement variable-sized sequence sampling, both with and without replacement. Sampling from eg LSTMs creates a sequence of random samples, but the amount of random variables sampled varies depending on eg when the EOS token is sampled.

Using the AncestralPlate, we should be able to keep track of exactly what sequences are sampled. When a sequence reaches EOS, we can remove it from the samples in the future to not do double computation. Should be possible both with and without replacement: When doing without replacement, we should only sample the k-1.