Coverlet.Core as NuGet package.

[p10tyr]

Is it possible to offer Coverlet.Core as a NuGet package please.

This will help in development of other extensions, tools and just general consumption of your amazing tool.

As you know, I would like to include automatic code coverage in my Extension but the only way I could to that is to clone your repo and then build the Core NuGet my self and consume that. This creates a problem that I will be missing out on updates.

Also as discussed in this PR https://github.com/tonerdo/coverlet/pull/105 it would be better if this functionality consumed Coverlet.Core -- instead of adding that functionality INTO Coverlet.Core -- It is the same as if I wanted to add VSIX into Coverlet.Core.

Please consider this. Thanks again! 🥇

[tonerdo]

@ppumkin what about the alternative of just wrapping coverlet.console instead? Opening up Coverlet.Core will require changing things around a little bit as all the important stuff is either internal/private

[p10tyr]

I did try this to use the Coverlet.Console package but I get the error

Package coverlet.console 1.2.0 has a package type DotnetTool that is not supported by project PrestoCoverage

This is why I downloaded your branch and pushed it into my AzureDevops and built the Coverelt.Core package hosted in my private feed. I just need to do a subset of things you are doing in the Console

So in the extension I can just use this

Coverlet.Core.Coverage coverage = new Coverlet.Core.Coverage(fullPath, new string[0], new string[0], new string[0], string.Empty);

coverage.PrepareModules();

var result = coverage.GetCoverageResult();

I am not sure what changes you would need to do to "open it up" ? It is fine jsut as it is now, I do not need anything else then to tell it where the dll is and what the coverage it.

I will automate the test execution using VS IDE API and then I can use Coverlet to get the Coverage directly from memory.

I think that building and publishing Coverlet.Core will just promote your package even more since you are allowing it to be used in other applications.

I am trying to understand why you would not want to publish the package so the options that I can see now are: (in order of preference)

• *1 - You publish Coverlet.Core onto NuGet and allow people to consume it
• (99999) - I build and publish it to NuGet on your behalf so that who ever build my Extension can download the dependency
• (100000000) - I download the source project and include it inside my extension project so that it is built together

The only reason I ask for you to consider integrating that into your CI/CD pipeline is to avoid partitioning of your Package.

Thanks again.

[singhsarab]

Even I feel first class package for Coverlet.Core make a lot of sense and would make our life easier.

[p10tyr]

I have been getting requests for the NuGet package and will publish an unofficial one onto NuGet in the coming days to help contributors. As you can see people here are also waiting. If you have any reservations about this please let me know.

[tonerdo]

@ppumkin and what about when people have problems with your NuGet package? The whole reason coverlet.console and coverlet.msbuild remain the only public interfaces is because it allows me to evolve coverlet.core at will and without limitations.

As users start to build tools around your NuGet package, they will begin to expect certain things from the core library that should generally be only available via the wrappers and that's extra support overhead for myself and other core contributors.

[p10tyr]

Looking at the flip side it also allows users to create and maintain other cool things using your library. For example, the Extension I built solely exists because of your library and MsBuild and Console just cant be used to leverage this functionality. I had to download the source, make my own package and bundle it privately with the final build.

Looking at SOLID principles. Your code should be closed to modification but open to extension. Now the true meaning of that in terms of Design Patterns is different but for content makers it means being able to extend beyond your domain boundry.

I completely understand what you are saying about changing things. But if you change the API on the CORE then you have to update the MsBuild and Console the same way everybody else using the Core would have to go and update it.

Just building a package to allow access on the public API is no threat on how you decide to make all the magic happen inside it just allows people to consume and build more things from there. That is what I consider open to extension for content makers.

If you need to change the public API then you change it. Downstream implementors will either update it or not - You cannot really worry about that. The only thing that comes out of this is more people use your library.

-EDIT Another option, if you really don't want to create a package for Core is to create an encapsulating, de-coupled - NetStandard 2.0 Package that can be used for integrators and publish that as a NuGet?

This package could have an Interface and implement the things needed to work with core. It would enable Dependancy Injection and a minimal, non code, API Package?

[jnovick]

I think @ppumkin is right to publish it for others to use but I think it should be forked from this repo and given a name that indicates it is the forked version that way people who consume it don't get confused. If you, @tonerdo, don't like someone else publishing a forked version of your code, then publish it yourself. If you aren't willing to do that then people like myself will consume and enjoy the forked package.

[tonerdo]

@jnovick I don't think I like your tone. This project is open source and anyone is free to fork or publish whatever they want. I was simply stating my concerns on what having a coverlet.core NuGet package in the wild could look like for this project and its maintainers.

[jnovick]

@tonerdo And that is why I suggest it be a fork with a different name. I am sorry if my tone comes across as harsh. It is not intended that way. It is intended to be more a matter of fact than any emotional tone. This is one of the many reasons I prefer spoken language over written language. Emotions can't be expressed over writing. So if I offended you, I sincerely apologize. While I appreciate your concerns, I find your project to be too good not to try and help it grow by opening up the core project to public consumption. It is not meant to offend you. My strong feelings on being able to consume it are because of how much I love this project and want to be able to use it in more ways. I do hope you reconsider publishing a package, but if you do not then I will enjoy the version published by someone else.

[tonerdo]

@ppumkin @jnovick I'm inclined to put this under more consideration. Re-opening this issue and inviting the thoughts of @MarcoRossignoli @petli as core contributors and @ViktorHofer and @sharwell as users who've had to adapt Coverlet to edge cases.

Thoughts?

[sharwell]

I don't see any reason why you couldn't publish the package to NuGet. The API compatibility policy can be as relaxed or strict as you like.

[MarcoRossignoli]

Before express final thoughts I would like to have the "scenarios" list to understand risk-benefit, for risk I mean slow down the freedom to do whatever we want with internals, Coverlet changes very quickly, we added logging and maybe we will change the way we integrate with test framework(collectors), there are some issue with current "static model"(instrumentation/coverageresult in same process) https://github.com/tonerdo/coverlet/issues/364 and so on(signature/behaviour changes). I mean we could add some "core breaking changes" that will become an issue for "extensions maintainers"(is coverlet mature enought?). Well we could provide package "AS IS" with disclamer "we don't guarantee any stability on apis because we care more about console/msuild(main scenarios)" but personally I don't like this way, I would like to expose a stable contracts with stable behaviour/usage workflow and have correct visibility of members with clear creation pattern and so on. Maybe to be clean we should separate core from reports...and so on. Finally as I said I'm interested in scenarios because this change could(if we decide that users of core/msbuild/console has got same dignity) put some friction on "agile" evolution.

[tonerdo]

@MarcoRossignoli my thoughts exactly. I'm not against opening up the core package, I'm just concerned that it's a bit too early. I've also been thinking of creating a separate coverlet.reporters assembly so the core library is only concerned with instrumentation and coverage calculation.

I'm happy to put a disclaimer about breaking changes but I'll feel more inclined to keep compat when a package is being consumed publicly. Also, in a situation where the coverlet.core is split into multiple different assemblies, are we on the hook to release those as NuGet packages as well?

[p10tyr]

OK reading @MarcoRossignoli points it actually makes some sense to why you have not done it yet. OK.

In terms of segregation I have been reading the NuGet MS Docs and they suggest to package every useful thing into a separate package. So I get that the Core API is volatile so splitting out all the MsBuild, Console and Core things just creates unnecessary overhead.

So .. As for providing a Coverlet.Core package AS IS ...

Forked Coverlet.Core go crazy people but be warned things are going to change so please do not come crying back here, go cry in a pillow and then refactor your code.

[MarcoRossignoli]

Close this issue for the moment to cleanup a bit. BTW we're working on dependency injection for testability, this will help to reach a point where expose coverlet.core will be possible in a clean way.

[MarcoRossignoli]

Re-opened for re-discussion after https://github.com/tonerdo/coverlet/pull/577 Should we open core feature for stand-alone usage?

At the moment after https://github.com/tonerdo/coverlet/issues/387 and with merge of https://github.com/tonerdo/coverlet/pull/566 we've all pieces to start to expose stable contracts for corelib.

cc: @tonerdo @petli @shargon

[shargon]

Should we open core feature for stand-alone usage?

Of course, coverage is not only useful for testing, it could be used in other areas like fuzzing. So for me, as security auditor, it's required as stand-alone and allow it to configure it for persist the instrumentation.

[MarcoRossignoli]

So for me, as security auditor

Interesting...if possible, can you explain better how you use instrumentation for fuzzing/security audit?

[shargon]

We are developing a tool for fuzzing .net software using your library.

Coverage is very important in modern fuzzers, because you can know if you are hitting most of the parts of the software that you want to "fuzz".

Using coverage combined with fuzzing you can be more precise with your inputs, and you can know if you are doing a good job.

Also in a future is possible to do a "coverage guided fuzzing" that uses program instrumentation to trace the code coverage reached by each input fed to a fuzz target. Fuzzing engines use this information to make informed decisions about which inputs to mutate to maximize coverage.

What we need:

• Persistent instrumentation (done in our branch https://github.com/tonerdo/coverlet/compare/master...Red4Sec:master)
• Read coverage in real time (done with reflection reading HitsArray)

[MarcoRossignoli]

wow...very interesting way to use coverlet...what do you think @tonerdo?

[shargon]

You can see a demo with random entries here

https://www.youtube.com/watch?v=EKnMCkSQiGY&feature=youtu.be

[MarcoRossignoli]

This stuff is amazing dot We need to wait for @tonerdo and @petli thoughts I'm only a co-maintainer. My vote is that we should provide stand-alone package, we have 6 thumbs up and at least 2 possible advanced scenario: 1) Custom instrumentation for live scenario(i.e. past attempt https://github.com/tonerdo/coverlet/pull/174 cc: @szogun1987) 2) Fuzzing

Wait for community thoughts and maintainers...if we go on we'll discuss interface/usage on new feature issue. Thank's @shargon for the interest and for infos.

[MarcoRossignoli]

ping @tonerdo @petli do you agree to put this on roadmap?

[petli]

I agree it'll be good to have this.

(Though you hardly need my vote as a maintainer. I can help keeping an eye on the code and discussing the interfaces, but I'm not involved enough to set the path of the project.)

[MarcoRossignoli]

thank's for share you thoughts Peter!

[mdenhoedt]

This issue can be closed I think, there is a NuGet package available.

[MarcoRossignoli]

This issue can be closed I think, there is a NuGet package available.

Hi @mdenhoedt what do you mean?

[rachied]

I'm working on an interactive mutation testing playground and am also looking to use Coverlet's core to analyze and display coverage (not sure yet if this is possible in my scenario). Due to running in the browser on webassembly it unfortunately can't depend on MSBuild or call it from console, so an official Coverlet.Core package would be ideal.

[p10tyr]

Fork it and make your own. That's what I did in the end.