Add a forward-only task to sbibm

[psteinb]

this PR paves the way to allow users to add simulations without a reference posterior. This way, (I hope) it becomes easier for users to test drive and benchmark sbi for their use case even.

Closes #19

[psteinb]

I am not yet 100% happy with the naming scheme. I also observed that no tasks appear to have unit tests.

[jan-matthis]

Thanks for starting off a PR! Perhaps it would be best to have a general unit tests that checks all tasks in the same way and consider more specific ones on a per-task basis?

[psteinb]

• [x] added a very general test suite for two_moons (a starting point for more perhaps)
• [x] finished norefposterior implementation for the time being
• [x] add unit tests to illustrate usage limitations

[psteinb]

The code is pretty much done at this point. the simulate step for norefposterior turns out to be quite slow. I am not a pyro expert and hence need to check at some later point in time, what the root cause of this might be.

@jan-matthis please review.

[psteinb]

alright, this PR is ready to go from my side. Studying the other tasks, I see that my application tends to be somewhat special as I simulate an entire dense representation. So I am happy to receive any feedback on that.

I'll check now, how to obtain benchmark results on this task.

[jan-matthis]

Great, I will review it tomorrow

[jan-matthis]

Could you rebase on main for the next round of reviews?

[psteinb]

huh, I thought I did so last night. Let me check.

[psteinb]

Done, I hope.

[psteinb]

Sorry folks, I hand-cuffed myself by using git rebase in a wrong fashion. I undid my rebase from earlier now (using cherry-pick) and repeated the exercise.

[psteinb]

I hope I have addressed all comments now. Sorry for the mess with the accidental rebase.

[jan-matthis]

No worries at all -- unfortunately, I think something is not quite right yet: Currently the PR still contains changes are on the main branch already

[psteinb]

Not sure what you mean. github says this PR has no conflicts. I just did a manual git merge upstream main ... what am I missing?

[jan-matthis]

I think you resolved the issue with commit b876c91 above. All looks good now!

I assume this is ready to be merged then -- or should we rather do #29 first? 🎉

[psteinb]

Let's wait for #29 to be merged first. Then I will close this PR, split the code according to #29 and resubmit a new PR. One merit of this is also, that this very long conversation gets squashed into a handful of commits. Feel free to disagree.

Quite frankly: I'd love to get on with stuff and start running experiments.

[jan-matthis]

29 is merged -- so only this one is left -- almost done

[jan-matthis]

Let's wait for #29 to be merged first. Then I will close this PR, split the code according to #29 and resubmit a new PR. One merit of this is also, that this very long conversation gets squashed into a handful of commits. Feel free to disagree.

Sounds good to me!

[psteinb]

closed in favor of #34 Thanks to @jan-matthis and @janfb for all your valuable feedback