search

dpeerlab / Harmony

Harmony framework for connecting scRNA-seq data from discrete time points
GNU General Public License v2.0
44 stars 12 forks source link

Is Harmony deterministic? #7

Closed concatenize closed 4 years ago

concatenize commented 4 years ago

I have run Harmony several times with the same input and gotten different results. Is Harmony expected to be deterministic? If so, I can try to work up a minimal example of what I am seeing. If not, then that's fine; I just want to know what to expect. Thanks!

ManuSetty commented 4 years ago

The augmented affinity matrix itself is deterministic and therefore should not change with different runs. The force directed layouts however do have a random initialization and can change from run to run. This can be fixed by setting the random seed.

concatenize commented 4 years ago

OK, I will look into this further and see if I can get a minimal example of the problem. What seed should I set? np.random.seed() from numpy or random.seed() from random or something else?

ManuSetty commented 4 years ago

random.seed should work.

concatenize commented 4 years ago

OK, I just tried it out on a 500x500 example. I find that random.seed(0) does not work, but np.random.seed(0) does work, at least for this example. I am attaching example data, code, and environment details. I am using python 3.6.8.

harmony_determinism.zip

Can you check and see if you get the same result? Thanks for your help!

concatenize commented 4 years ago

Sorry, there is a misplaced line in my example that explains the discrepancy: in fact, the working code block is seeded with random, not numpy. This fits better with my past experience as well, because I was sure I had tried the numpy seed and it failed (as it should have).

This should completely resolve my issue. Thanks again!

ManuSetty commented 4 years ago

Thanks for verifying!

concatenize commented 4 years ago

Would you please take a look at this example? I have set the seeds but the result comes out different each time. Maybe I am just missing something simple.

irreproducible_example.zip

ManuSetty commented 4 years ago

Hello

Thanks for sending the example. I was able to reproduce the layouts by setting np.random.seed. Please see the attached notebook and let me know if still continue to have issues.

irreproducible_example-np-random.zip

concatenize commented 4 years ago

Yes, that works for me too. Thank you!

Eric