search

tskit-dev / tsdate

Infer the age of ancestral nodes in a tree sequence.
MIT License
19 stars 10 forks source link

Add variational method accuracy tests to `test_accuracy.py` #265

Open hyanwong opened 1 year ago

hyanwong commented 1 year ago

Once we have a reasonable variational method working, we should add a test or two to test_accuracy.py so that we pick up any regressions. Do you think you could do this @nspope ?

hyanwong commented 5 months ago

I think we are in a good place to add a few of these tests, right @nspope ?

nspope commented 5 months ago

Sure -- but what are we testing, exactly? If it's statistical accuracy then I don't think unit tests are the right place for that -- we want a separate set of benchmarks that uses data of a reasonable size.

hyanwong commented 5 months ago

The idea was simply to check if changes accidentally created any major regressions (I don't think they should). I agree that larger-scale accuracy tests should be in the unit tests. I think that's what #379 is about

hyanwong commented 5 months ago

I think a few very simple tests with tree sequences with (say) 1, 2, and 5 trees of 2 and 20 samples would probably be fine. But maybe you think it's not worth it?

nspope commented 5 months ago

Not worth it, IMO. I'd rather have a benchmark suite that could be run whenever there's a change to the low-level stuff.