`Error: no such option: --report` When Following Baseline Tutorial

[FloatingSunfish]

I'm trying to follow the tutorial on how to use Cosmic Ray and there seems to be something wrong with the Baselining section.

When I run the provided command cosmic-ray baseline --report tutorial.toml tutorial.sqlite, I get the following error:

Usage: cosmic-ray baseline [OPTIONS] CONFIG_FILE SESSION_FILE
Try 'cosmic-ray baseline --help' for help.

Error: no such option: --report

The --report flag can no longer be used with the baseline command.

I can't find a section in the documentation about what [OPTIONS] I can pass because --help doesn't provide that info.

Any advice?

[abingham]

The tutorial seems to be a bit out of date. I'll do some work to update it soon. For now, don't pass --report, and use tutorial_baseline.sqlite instead of tutorial.sqlite in the baseline command. Essentially, baselining will overwrite/replace everything in a session file, so you don't want to destroy what's in tutorial.sqlite (i.e. what was created by the init call).

I can't find a section in the documentation about what [OPTIONS] I can pass because --help doesn't provide that info.

The command cosmic-ray baseline --help should show you everything about that command.

[abingham]

The tutorial should now be updated to reflect reality. Can you look it over and, if it looks ok, close it?

Also, thanks for reporting this. It's a huge help when people find and report these kinds of things.

[FloatingSunfish]

Noted, and thanks for the quick reply, @abingham. 🙂

Quick question:

>Essentially, baselining will overwrite/replace everything in a session file, so you don't want to destroy what's in tutorial.sqlite (i.e. what was created by the init call).

Might I ask why I wouldn't want to overwrite something that was generated by init which is pretty much a blank slate?

[FloatingSunfish]

@abingham I just checked out the updated tutorial and everything looks great. 🙂

I'll be happy to close this issue, but before I do, I want to hear your answer to my question above.

Thanks!

[abingham]

Baselining as currently implemented might qualify as a hack. A session is essentially just a list of WorkItems, each of which might have some results attached to it. Each WorkItem has zero or more mutations that it performs. Under normal operation, each has one or more, but baselining creates a WorkItem with zero mutations. This lets us test the entire CR system with unmutated code; this is all a baseline really is.

So the rationale for baseline overwriting the session is that we just want to create a session with a single workitem (which has zero mutations), exec that session, and see that the test suite pass. We don't want to add that mutation to your real session since it's not really part of your mutation testing run.

Might I ask why I wouldn't want to overwrite something that was generated by init which is pretty much a blank slate?

To really answer your question: the session created by init is substantially more than a blank slate. It contains a WorkItem for each set of mutations CR will make, though none of these WorkItems yet has any results. It's really a manifest of all the work left to be done, so it contains a lot of information.

[FloatingSunfish]

Noted, thanks for the answer, @abingham. 🙂

Before I close this issue, might I suggest adding a new paragraph after verifying that the result of cosmic-ray --verbosity=INFO baseline tutorial.toml tutorial_baseline.sqlite is as expected and telling the reader that they can now delete tutorial_baseline.sqlite because we only used it to make sure that all tests pass without any mutations.

[abingham]

This is a good idea. In fact, you've got me thinking that baselining should just create its own temporary session and delete it after it's done. We could add options for keeping the session (e.g. in case the user needs to see what's going wrong), but by default they don't need to see the session.

[FloatingSunfish]

That sounds great, @abingham! It would certainly help CR be more elegant for its users. 🙂

Looking forward to seeing this update in the future! Hope you add it to your backlog.

Closing this issue now. 😉

[abingham]

I've updated baseline to use a temporary session by default.

[FloatingSunfish]

@abingham Looks great, and I see the docs have been updated as well! 🙂 I'll let you know if I encounter any more issues regarding this scenario.

Many thanks for the quick fixes and replies! 😊

[FloatingSunfish]

@abingham It looks like the baseline command still expects a session parameter, as the command in the docs can't run without one.

Perhaps the version I got from pip is still outdated? Or are the changes only present in the source version for now?

[abingham]

Version 8.3.1 in PyPI should have the latest changes. Is that the version you have installed?

[FloatingSunfish]

@abingham Ah, my mistake. I got a bit confused and didn't get the version from pip yet. 😅 I was using the outdated one from source.

I can now run the new command just fine. 🙂

Thanks!