yang-song / score_sde_pytorch

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)
https://arxiv.org/abs/2011.13456
Apache License 2.0
1.58k stars 295 forks source link

Added sliced score matching; does not learn (DO NOT MERGE) #20

Closed KyleM4t1qbit closed 1 year ago

KyleM4t1qbit commented 2 years ago

Hello,

I'm wondering if you have any idea why this does not work? I have updated get_sde_loss_fn to do sliced score matching as you've done in your sibling repo (https://github.com/ermongroup/sliced_score_matching). The eventual goal is to implement numerical sampling for an SDE without a closed form perturbation kernel.

The best I can obtain are patterns that look like this: download

This is CIFAR data set. I know 2000 iterations is substantially shorter than is required to get "good" results, but when I use get_sde_loss_fn as it exists in your code, I get acceptable results.

image

Do you have any idea where I could be going wrong? Thanks in advance