DrRacket coverage overlay

[jackfirth]

I'd really like it if pressing Run in DrRacket colored covered and uncovered lines after running tests. Is there any project in the cover ecosystem working on this? I see the "2.0 (IDE Integration)" milestone but nothing else.

[florence]

The TL;DR is yes, I would like to use cover to get multifile and crossphase coverage into drracket, someday.

(note: Multifile coverage was a thing at somepoint http://blog.racket-lang.org/2011/05/multi-file-code-coverage-viewing-tool.html )

Long version:

There is a lot of work that needs to happen first.

1. Cover would need to swap out the single file test coverage in DrRacket. This is what the (unstable) racket API in cover is for, but it doesn't quite match up with what DrRacket wants. Cover has a fairly complex harness for running code, and how to fit this harness in DrRacket is unclear. Outstanding issues here are things like: Who controls what in the coverage namespace? How should DrRacket handle starting and stoping Cover's listener thread(s)? What is the correct API for reporting coverage to DrRacket? Can we do this without a performance degradation in the single file coverage case? How do we separate different instances of coverage within one DrRacket instance?

2. We would need to pick a decent UI for describing what files to run and what files to render coverage for. I don't think "just run this collection" is the right interface, but a large file-picker dialog would probably be unwieldy.

3. We would need to decide if we want to bring cover into the main distribution? (Since doing this would cause DrRacket to depend on cover, we would have to).

P.S. That milestone stuff is out of date... Since you've opened this issue I'll use it to track any progress made on this front!

[jackfirth]

Great to hear there's interest! I should probably narrow down my use case to give you a better idea of what I'm after then. What I specifically want is this workflow:

• Open a module foo.rkt.
• Run the module, which runs the test submodule.
• Have DrRacket automatically show a green background for expressions that are covered and a red background for expressions that are uncovered directly in the definitions window (like how Check Syntax colors different identifiers).

That's it. I'm fine with needing to install packages to get this, and I'm fine with needing to use a different button / keyboard shortcut than the standard Run button. I would prefer not to mess with preference settings or dialogs or other configuration to get this behavior after installing packages, but it wouldn't be a deal breaker.

Multi-file coverage is definitely useful, but much of the time it's more than what I need. There's a lot of different use cases involved in editor coverage information so I think they should be scoped out into different issues. Does this use case require less work or do roughly the same changes need to happen?

[florence]

Oh. Since file coverage is already in DrRacket, albeit in a weird place: "Choose Language" (bottom left) -> "Show Details" -> "Syntactic test suite coverage".

[jackfirth]

Huh, I never knew that existed. Cover uses errortrace as well correct? Other than being single-file vs. multi-file, cover shouldn't behave differently from the "syntactic test suite coverage" option?

My first impression is that it's very annoying to use and I'd like to file some bugs somewhere to request changes to things like the approach to coloring, where to set the preference, etc. but I have no idea where this code is hosted, let alone if it's maintained.

A single tool that supported the single-file case easily while allowing you to configure it for multi-file behavior would be ideal.

[florence]

Cover uses errortrace as well correct?

Yes.

cover shouldn't behave differently...

Yes: in the single file case they should generate the same information.

In the multifile case cover will attempt to generate correct coverage of code above phase 1. Unfortunately this isn't feasible in the single file case (see #133). There also are some cosmetic coloring differences in coloring comments and submodules, but thats shouldn't matter for your use case. (see cover's -e flag).

but I have no idea where this code is hosted, let alone if it's maintained.

The DrRacket repository is the right place to log bugs for that.

A single tool that supported the single-file case easily

This is what cover wants to be. But see the questions about how to actually integrate it into DrRacket.

[jackfirth]

Would it be better to open issues and implement changes for DrRacket's single file code coverage, or instead presume Cover will eventually replace that functionality completely?

[florence]

It would be better to open issues now I think. Worst case we can triage them as "fix later".