search

russross / blackfriday

Blackfriday: a markdown processor for Go
Other
5.42k stars 598 forks source link

v2: decide on import path and make a new release #587

Closed kolyshkin closed 3 years ago

kolyshkin commented 4 years ago

The latest release is from September 2018, maybe it's time to make a new one?

The primary reasons for a new release are #509 and #515; this will make the package not depend on any other packages. Including #586 would be nice, too.

dmitshur commented 4 years ago

This should be done.

In order to be able to publish new v2 versions, a blocking task is to resolve the current lack of clarity about what import path should be suggested and continued to be used for v2: github.com/russross/blackfriday/v2 or gopkg.in/russross/blackfriday.v2. The go.mod file says one thing, the README on master branch says something that isn't very consistent.

By making a newer release, some decision would be committed to, and ideally that decision should be a good one given the constraints this project is currently operating with. Some of the decisions made in the past predate a good understanding of modules and best practices regarding v2 versions, which has made this decision more difficult to make conclusively.

I suspect that in the current situation, the best course of action is to continue to use the github.com/russross/blackfriday/v2 import path, as defined by the module statement in go.mod file in v2 branch:

https://github.com/russross/blackfriday/blob/3b2b20c8cb0632a0442ea9694d170b8aea985549/go.mod#L1

I started talking to @rtfb about resolving this situation a number of months ago, but then unfortunately dropped the ball on it due to lack of bandwidth. But I wanted to post this comment to shed light on what I believe is a large contributing factor to no new v2 releases.

russross commented 4 years ago

I'm happy with either import path and agree that we should post a new release soon. I haven't been following developments in the module system very closely--is there one approach that is more forward looking/will fit in better with the community than the other?

SamWhited commented 4 years ago

There's also a third option which I always personally prefer: using a new vanity import path (eg. blackfriday.example.net/v2).

This requires that someone own a domain name and setup an HTTP server (or use a service like Netlify or GitHub Pages) which is more work for the maintainers, however, it gives you the benefit of owning your package import instead of relying on GitHub or gopkg.in never changing or going under or being changed to github.microsoft.com or similar. While this may not seem likely, other scenarios could be imagined where GitHub becomes impractical in the future, for example a price change, adding more extreme rate limiting that causes fetching packages to break more often, the author just decides they don't like GitHub and want to use , etc. Using a vanity import keeps things flexible in the future, and avoids unforeseen future issues at the cost of a tiny bit of infrastructure work. I would be willing to help set this up if you want.

kolyshkin commented 4 years ago

This requires that someone own a domain name and setup an HTTP server

... and in a couple years it will be down and we won't be able to reach a person who hosted it. Speaking from experience :-\

SamWhited commented 4 years ago

I would think relying on GitHub would be the bigger risk here. They could go under, or Microsoft could change their pricing and make it harder for open source projects to host on it, or some overzealous company could file a patent claim against something as trivial as tree traversal and Microsoft could take down the repo out of an abundance of caution, etc.

The package proxy smooths out this sort of thing either way though.

russross commented 4 years ago

I'd rather stick with github. It's pretty standard and predictable without requiring any maintenance. The Go project is hosted there as are most libraries in widespread use, so in the unlikely event that Microsoft pulls the rug out from under everyone, updating blackfriday will just be a small part of a much larger disruption.

laher commented 4 years ago

Assuming that we don't need to debate the module path any further ... I believe all this really needs is a README update to replace the 'gopkg.in' recommendation, and a tag on the v2 branch - v2.0.2/v2.1.0?

Or, are you considering a merge into master first?

If it's just the tag & readme, I can make a PR for the readme update.

FWIW I agree that github seems the pragmatic choice - I don't believe blackfriday has any need for gopkg.in any more now that modules have versioned paths - and also hasn't github already been chosen in go.mod, in both the v2 branch and the v2.0.1 tag?

mwat56 commented 4 years ago

@laher

Assuming that we don't need to debate the module path any further ... I believe all this really needs is a README update to replace the 'gopkg.in' recommendation, and a tag on the v2 branch - v2.0.2/v2.1.0?

… and copy the recent files to a ./v2/ subdirectory while leaving the v1.x.x files were they are. (see Go Modules: v2 and Beyond )

laher commented 4 years ago

@mwat56 it's worth raising that point, yup, especially this part:

if v2+ source code is on the repository's default branch (usually master), tools that are not version-aware — including the go command in GOPATH mode — may not distinguish between major versions.

I guess the question is, would it be OK to just break non-module-aware users that aren't pinning versions somehow? And, I guess the answer for such an established project like blackfriday, might be 'nope, not for some time to come'? Or not? pained thinking emoji

mwat56 commented 4 years ago

@laher

[...] I guess the question is, would it be OK to just break non-module-aware users that aren't pinning versions somehow?

Why break anything at all? Just leave the current files as v1.x.y and copy the newest version of each file into a new subdirectory ./v2/ and assign there a version tag v2.x.y

That way the non-version aware projects (are there still some of those after discussing versioning for years?) are using the v1.x.y and the version-aware projects can use the v2 branch if they want.

Sure, it will probably take an hour to do the copying and check-in into a new v2 branch. But in the end all those require/replace hacks won't be necessary anymore.

depado commented 4 years ago

Hey, any news on this? I think it can be as straightforward as @mwat56 suggested and it would finally fix all those replace tricks we've been doing to point to gopkg.in. This is a real pain point right now.

khuddlefish commented 4 years ago

I'd like to +1 this. Just tried to go get this using the path in the README and had all sorts of issues. go get github.com/russross/blackfriday/v2 worked for me. Can we change the recommendation to use that?

danneu commented 4 years ago

go get github.com/russross/blackfriday/v2 worked for me. Can we change the recommendation to use that?

Yeah, current readme install instruction leads to this output:

$ go get "gopkg.in/russross/blackfriday.v2"
go: gopkg.in/russross/blackfriday.v2 upgrade => v2.0.1
go get: gopkg.in/russross/blackfriday.v2@v2.0.1: parsing go.mod:
    module declares its path as: github.com/russross/blackfriday/v2
            but was required as: gopkg.in/russross/blackfriday.v2
russross commented 4 years ago

I haven't followed all of the module changes and import path stuff. If someone who is up-to-date on it will submit a PR I'll merge it and we can get this one resolved.

dmitshur commented 3 years ago

Picking up from https://github.com/russross/blackfriday/issues/587#issuecomment-603493129.

Current State

To recap, the current (unfortunate) state regarding blackfriday v2 and its import path is:

Unfortunately, some projects use the github.com/russross/blackfriday/v2 import path, while some other projects use the gopkg.in/russross/blackfriday.v2 one, so whatever decision we make involves causing some projects the need to change import path they use to be able to receive newer versions of blackfriday v2.

Some Data

To get some (imperfect) information about which of the two v2 import paths are being used by public projects in active development, I wrote a small program (source) that walks over the module index, considers only the latest version of each unique module, and checks whether it requires github.com/russross/blackfriday/v2 or gopkg.in/russross/blackfriday.v2. When considering the entire 2020 year so far:

total modules: 1111930
unique modules: 192022
modules that require github.com/russross/blackfriday (v1): 469
modules that require github.com/russross/blackfriday/v2: 202
modules that require gopkg.in/russross/blackfriday.v1 modules: 3
modules that require gopkg.in/russross/blackfriday.v2 modules: 81

When looking only at modules that have had new versions published within the last month:

total modules: 104163
unique modules: 34127
modules that require github.com/russross/blackfriday (v1): 117
modules that require github.com/russross/blackfriday/v2: 68
modules that require gopkg.in/russross/blackfriday.v1: 1
modules that require gopkg.in/russross/blackfriday.v2: 18

(Full list here.)

So it seems staying with github.com/russross/blackfriday/v2 would require fewer active projects to change their import path.

Trade-Offs Considered

The reason it has been so hard to make a decision and all the delays here is because neither solution is ideal—each one has some upsides and downsides—and the trade-offs also differ depending on whether you consider the short term, medium, or long-term outcomes.

From my earlier discussion with @rtfb, the discussion in the comments above, and the data from walking the module index in 2020, it seems better to choose github.com/russross/blackfriday/v2 as the supported import path for blackfriday v2 going forward. (The existing released version at the gopkg.in import path may still work, but it won't be supported or endorsed.)

If there are many reports of github.com/russross/blackfriday/v2 not being supported in GOPATH mode causing trouble, there will still be a way for us to address it: by switching from major branch model (v2 version is on v2 branch) to major directory model (v2 version is in a v2 directory on master branch). This can be done later.

Next Steps

To that end, I suggest we take this sequence of steps to resolve this issue:

depado commented 3 years ago

Thanks a lot for that summary.

As a side note, the divergence between the two import styles also forces some decisions on developers. As an example, if a transitive dependency uses the github.com/russruss/blackfriday/v2 import path, it's impossible to use the gopkg.in one. It's the case if you're using spf13/cobra, which depends on cpuguy83/go-md2man/v2 that imports blackfriday using the github style import. And the error when trying to compile isn't clear at all, you need to take a deep look at your dependency graph to understand what's going on.

I also agree that using the github.com version is better suited for long term, it's also more intuitive for newcomers.

dmitshur commented 3 years ago

@russross @rtfb Any concerns if I proceed with the plan suggested in "Next Steps" of the comment above soon?

russross commented 3 years ago

Sounds good to me.

On Thu, Oct 22, 2020 at 10:07 PM Dmitri Shuralyov notifications@github.com wrote:

@russross https://github.com/russross @rtfb https://github.com/rtfb Any objections if I proceed with the plan described above https://github.com/russross/blackfriday/issues/587#issuecomment-703393820 soon?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/russross/blackfriday/issues/587#issuecomment-714896277, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAP7FHB6CNRHZNGVN7JSALSMD6OJANCNFSM4LS5MTIA .

dmitshur commented 3 years ago

Moving on to step 2, I've published a pre-release version v2.1.0-pre.1 at the tip of v2 branch:

$ go list -m -versions github.com/russross/blackfriday/v2
github.com/russross/blackfriday/v2 v2.0.0 v2.0.1 v2.1.0-pre.1

If you're following this issue and are able to, please test v2.1.0-pre.1 and comment (or report an issue) if you find a serious problem, before we promote the pre-release version to a final v2.1.0 release. You should be able to install it in your project with:

$ go get github.com/russross/blackfriday/v2@v2.1.0-pre.1

Thanks.

dmitshur commented 3 years ago

There haven't been any reports of serious issues with the v2.1.0-pre.1 pre-release version (neither in this thread, nor in other issues), so I'm going to proceed with promoting it to v2.1.0. That will resolve this issue.

dmitshur commented 3 years ago

Version v2.1.0 is now released.

I've realized to resolve this fully, we also need to publish a new minor release for blackfriday v1, since its current latest release (v1.5.2) still has the old README. I'll follow a similar process:

Then we can close this issue.

dmitshur commented 3 years ago

There haven't been any reports of serious issues with the v1.6.0-pre.1 pre-release version (neither in this thread, nor in other issues), so I'm going to proceed with promoting it to v1.6.0.

Version v1.6.0 is now released.

I think this specific issue is finally resolved. Phew. 😓

laher commented 3 years ago

Great job @dmitshur - a big effort, well executed. The result makes it easy for people to use blackfriday v2 in this Go modules world, which most of us live in now. Many thanks