Doesn't build in projects that use `dep` Go dependency management tool due to issue with blackfriday v2.0.0 tag.

[markbates]

They released blackfriday v2, about 20 hours ago, and now this project doesn't build anymore. :(

vendor/github.com/shurcooL/github_flavored_markdown/main.go:82: undefined: blackfriday.EXTENSION_NO_INTRA_EMPHASIS
vendor/github.com/shurcooL/github_flavored_markdown/main.go:83: undefined: blackfriday.EXTENSION_TABLES
vendor/github.com/shurcooL/github_flavored_markdown/main.go:84: undefined: blackfriday.EXTENSION_FENCED_CODE
vendor/github.com/shurcooL/github_flavored_markdown/main.go:85: undefined: blackfriday.EXTENSION_AUTOLINK
vendor/github.com/shurcooL/github_flavored_markdown/main.go:86: undefined: blackfriday.EXTENSION_STRIKETHROUGH
vendor/github.com/shurcooL/github_flavored_markdown/main.go:87: undefined: blackfriday.EXTENSION_SPACE_HEADERS
vendor/github.com/shurcooL/github_flavored_markdown/main.go:88: undefined: blackfriday.EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK
vendor/github.com/shurcooL/github_flavored_markdown/main.go:88: const initializer blackfriday.EXTENSION_NO_INTRA_EMPHASIS | blackfriday.EXTENSION_TABLES | blackfriday.EXTENSION_FENCED_CODE | blackfriday.EXTENSION_AUTOLINK | blackfriday.EXTENSION_STRIKETHROUGH | blackfriday.EXTENSION_SPACE_HEADERS | blackfriday.EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK is not a constant
vendor/github.com/shurcooL/github_flavored_markdown/main.go:104: undefined: blackfriday.Html

[markbates]

I can easily reproduce this using dep.

Gopkg.toml ```Gopkg.toml [[constraint]] branch = "master" name = "github.com/bryanl/webbrowser" [[constraint]] branch = "development" name = "github.com/gobuffalo/buffalo" [[constraint]] branch = "master" name = "github.com/gobuffalo/makr" [[constraint]] branch = "master" name = "github.com/gobuffalo/packr" [[constraint]] branch = "master" name = "github.com/gobuffalo/plush" [[constraint]] branch = "master" name = "github.com/gopherguides/remark" [[constraint]] branch = "master" name = "github.com/markbates/inflect" [[constraint]] name = "github.com/pkg/errors" version = "0.8.0" [[constraint]] name = "github.com/sirupsen/logrus" version = "1.0.2" [[constraint]] branch = "master" name = "github.com/spf13/cobra" ```

Gopkg.lock: ```Gopkg.lock # This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'. [[projects]] name = "github.com/BurntSushi/toml" packages = ["."] revision = "b26d9c308763d68093482582cea63d69be07a0f0" version = "v0.3.0" [[projects]] branch = "master" name = "github.com/bryanl/webbrowser" packages = ["."] revision = "aa68d5a888ea07db1c2d2bed34f22e89cb3e3067" [[projects]] branch = "master" name = "github.com/daviddengcn/go-colortext" packages = ["."] revision = "805cee6e0d43c72ba1d4e3275965ff41e0da068a" [[projects]] branch = "master" name = "github.com/dustin/go-humanize" packages = ["."] revision = "259d2a102b871d17f30e3cd9881a642961a1e486" [[projects]] name = "github.com/fatih/color" packages = ["."] revision = "570b54cabe6b8eb0bc2dfce68d964677d63b5260" version = "v1.5.0" [[projects]] name = "github.com/fatih/structs" packages = ["."] revision = "a720dfa8df582c51dee1b36feabb906bde1588bd" version = "v1.0" [[projects]] name = "github.com/fsnotify/fsnotify" packages = ["."] revision = "629574ca2a5df945712d3079857300b5e4da0236" version = "v1.4.2" [[projects]] name = "github.com/go-sql-driver/mysql" packages = ["."] revision = "a0583e0143b1624142adab07e0e97fe106d99561" version = "v1.3" [[projects]] branch = "development" name = "github.com/gobuffalo/buffalo" packages = [".","binding","render","worker"] revision = "7276f85b179128018e5bef9d645644632a390fdc" [[projects]] branch = "master" name = "github.com/gobuffalo/envy" packages = ["."] revision = "a901aeb76c2647ce52acf7f3bac64c4def49a899" [[projects]] branch = "master" name = "github.com/gobuffalo/makr" packages = ["."] revision = "b741e6545d4605d7d8595423a00b89bdfed3572d" [[projects]] branch = "master" name = "github.com/gobuffalo/packr" packages = ["."] revision = "e9ca0e599847cf656d6c8d6c0680b561129cc3f3" [[projects]] branch = "master" name = "github.com/gobuffalo/plush" packages = [".","ast","lexer","parser","token"] revision = "8ed4c6e772069729c5ce37130912773c600f5d80" [[projects]] branch = "master" name = "github.com/gobuffalo/tags" packages = [".","form","form/bootstrap"] revision = "c6f054f107d94b330d38a824866fdcd94bd616eb" [[projects]] branch = "master" name = "github.com/gopherguides/remark" packages = ["."] revision = "ddce67243e3aa4a2a8bac66b9e3549f0679787e4" [[projects]] name = "github.com/gorilla/context" packages = ["."] revision = "1ea25387ff6f684839d82767c1733ff4d4d15d0a" version = "v1.1" [[projects]] name = "github.com/gorilla/mux" packages = ["."] revision = "bcd8bc72b08df0f70df986b97f95590779502d31" version = "v1.4.0" [[projects]] name = "github.com/gorilla/securecookie" packages = ["."] revision = "667fe4e3466a040b780561fe9b51a83a3753eefc" version = "v1.1" [[projects]] name = "github.com/gorilla/sessions" packages = ["."] revision = "ca9ada44574153444b00d3fd9c8559e4cc95f896" version = "v1.1" [[projects]] name = "github.com/gorilla/websocket" packages = ["."] revision = "ea4d1f681babbce9545c9c5f3d5194a789c89f5b" version = "v1.2.0" [[projects]] name = "github.com/inconshreveable/mousetrap" packages = ["."] revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75" version = "v1.0" [[projects]] branch = "master" name = "github.com/jmoiron/sqlx" packages = [".","reflectx"] revision = "d9bd385d68c068f1fabb5057e3dedcbcbb039d0f" [[projects]] branch = "master" name = "github.com/lib/pq" packages = [".","oid"] revision = "e42267488fe361b9dc034be7a6bffef5b195bceb" [[projects]] branch = "master" name = "github.com/markbates/going" packages = ["defaults","randx","wait"] revision = "0576708c56cea02331f864fe6e157ac7841923e4" [[projects]] branch = "master" name = "github.com/markbates/grift" packages = ["grift"] revision = "8e706d9b1370e074ed48f3e56b5d29c8b01dd635" [[projects]] branch = "master" name = "github.com/markbates/inflect" packages = ["."] revision = "6cacb66d100482ef7cc366289ccb156020e57e76" [[projects]] name = "github.com/markbates/pop" packages = [".","columns","fizz","fizz/translators","nulls"] revision = "f5ac4bbf44cc4accd00109c1c45f9e99a7bdfe1f" version = "v3.30.1" [[projects]] name = "github.com/markbates/refresh" packages = ["refresh","refresh/web"] revision = "31a58af168f3788ce36d49f66aca4b439f64c7d4" version = "v1.3.3" [[projects]] branch = "master" name = "github.com/markbates/validate" packages = [".","validators"] revision = "b4702c95219d84d5c88a133f20490459b47ae866" [[projects]] branch = "master" name = "github.com/mattn/anko" packages = ["ast","builtins","builtins/encoding/json","builtins/errors","builtins/flag","builtins/fmt","builtins/github.com/daviddengcn/go-colortext","builtins/io","builtins/io/ioutil","builtins/math","builtins/math/big","builtins/math/rand","builtins/net","builtins/net/http","builtins/net/url","builtins/os","builtins/os/exec","builtins/os/signal","builtins/path","builtins/path/filepath","builtins/regexp","builtins/runtime","builtins/sort","builtins/strings","builtins/time","parser","vm"] revision = "a3969c63681391ec413427cae813d6d59d6031f1" [[projects]] name = "github.com/mattn/go-colorable" packages = ["."] revision = "167de6bfdfba052fa6b2d3664c8f5272e23c9072" version = "v0.0.9" [[projects]] name = "github.com/mattn/go-isatty" packages = ["."] revision = "fc9e8d8ef48496124e79ae0df75490096eccf6fe" version = "v0.0.2" [[projects]] name = "github.com/mattn/go-sqlite3" packages = ["."] revision = "ca5e3819723d8eeaf170ad510e7da1d6d2e94a08" version = "v1.2.0" [[projects]] branch = "master" name = "github.com/microcosm-cc/bluemonday" packages = ["."] revision = "f0d1606e9e60cb2428f85ddcbfcccfeb6b507586" [[projects]] branch = "master" name = "github.com/mitchellh/go-homedir" packages = ["."] revision = "b8bc1bf767474819792c23f32d8286a45736f1c6" [[projects]] name = "github.com/monoculum/formam" packages = ["."] revision = "99ca9dcbaca6d91e9665f817d0da23af6184ced3" version = "v3.0" [[projects]] name = "github.com/pkg/errors" packages = ["."] revision = "645ef00459ed84a119197bfb8d8205042c6df63d" version = "v0.8.0" [[projects]] name = "github.com/russross/blackfriday" packages = ["."] revision = "cadec560ec52d93835bf2f15bd794700d3a2473b" version = "v2.0.0" [[projects]] name = "github.com/satori/go.uuid" packages = ["."] revision = "879c5887cd475cd7864858769793b2ceb0d44feb" version = "v1.1.0" [[projects]] branch = "master" name = "github.com/serenize/snaker" packages = ["."] revision = "1c7f65329f6524115af8ca01ab604c00cddf5558" [[projects]] branch = "master" name = "github.com/sergi/go-diff" packages = ["diffmatchpatch"] revision = "feef008d51ad2b3778f85d387ccf91735543008d" [[projects]] branch = "master" name = "github.com/shurcooL/github_flavored_markdown" packages = ["."] revision = "91d0d9cd2f34cdf3152a15e166ea599381ac1a5b" [[projects]] branch = "master" name = "github.com/shurcooL/go" packages = ["parserutil","printerutil","reflectfind","reflectsource"] revision = "b2ea7b593ffddfdfb43ab719c308c25cb2054db0" [[projects]] branch = "master" name = "github.com/shurcooL/go-goon" packages = ["."] revision = "a0792a0a8f8fa9a233843e67079b051b571e609c" [[projects]] branch = "master" name = "github.com/shurcooL/highlight_diff" packages = ["."] revision = "09bb4053de1b1d872a9f25dc21378fa71dca4e4e" [[projects]] branch = "master" name = "github.com/shurcooL/highlight_go" packages = ["."] revision = "78fb10f4a5f89e812a5e26ab723b954a51226086" [[projects]] branch = "master" name = "github.com/shurcooL/octiconssvg" packages = ["."] revision = "1aed2117d2aa8eed76a052936e468e5354750701" [[projects]] branch = "master" name = "github.com/shurcooL/sanitized_anchor_name" packages = ["."] revision = "541ff5ee47f1dddf6a5281af78307d921524bcb5" [[projects]] name = "github.com/sirupsen/logrus" packages = ["."] revision = "a3f95b5c423586578a4e099b11a46c2479628cac" version = "1.0.2" [[projects]] branch = "master" name = "github.com/sourcegraph/annotate" packages = ["."] revision = "f4cad6c6324d3f584e1743d8b3e0e017a5f3a636" [[projects]] branch = "master" name = "github.com/sourcegraph/syntaxhighlight" packages = ["."] revision = "bd320f5d308e1a3c4314c678d8227a0d72574ae7" [[projects]] branch = "master" name = "github.com/spf13/cobra" packages = ["."] revision = "b26b538f693051ac6518e65672de3144ce3fbedc" [[projects]] name = "github.com/spf13/pflag" packages = ["."] revision = "e57e3eeb33f795204c1ca35f56c44f83227c6e66" version = "v1.0.0" [[projects]] branch = "master" name = "golang.org/x/crypto" packages = ["bcrypt","blowfish","ssh/terminal"] revision = "b176d7def5d71bdd214203491f89843ed217f420" [[projects]] branch = "master" name = "golang.org/x/net" packages = ["context","html","html/atom"] revision = "1c05540f6879653db88113bc4a2b70aec4bd491f" [[projects]] branch = "master" name = "golang.org/x/sys" packages = ["unix","windows"] revision = "e42485b6e20ae7d2304ec72e535b103ed350cc02" [[projects]] branch = "v2" name = "gopkg.in/yaml.v2" packages = ["."] revision = "25c4ec802a7d637f88d584ab26798e94ad14c13b" [solve-meta] analyzer-name = "dep" analyzer-version = 1 inputs-digest = "6fead9f9ea413345c52e6934457a9ffe475018f4e0c732c315f9fc9d38e67a92" solver-name = "gps-cdcl" solver-version = 1 ```

[markbates]

I believe I've tracked down the problem, sort of.

Here is the release tag for blackfriday, v2 https://github.com/russross/blackfriday/releases/tag/v2.0.0

The dep tool is pulling that tag, because it like semver. This library, doesn't work with that tag. However, the tests for this library pass because the v2.0.0 tag of blackfriday is not against master, but rather against a branch.

Not really sure what the "correct" thing to do here is. It's kind of a big mess, mostly because of the way blackfriday has been tagged.

[dmitshur]

Thanks for filing. We should report this at blackfriday (done in https://github.com/russross/blackfriday/issues/383#issuecomment-321997075) and/or dep (already done at https://github.com/golang/dep/issues/999) and come up with a solution. The v2 version is supposed to be available at a different import path (gopkg.in/russross/blackfriday.v2) and shouldn't break existing packages that import original v1 blackfriday at its github.com/russross/blackfriday import path. /cc @rtfb @sdboyer

Note, I'm not closely familiar with the inner workings of dep yet. So I can't say if this is a problem dep should fix, or if dep is doing the right thing in this situation, and we should change blackfriday. I'm guessing the latter... but it needs discussion.

[markbates]

I've opened an issue with dep, here https://github.com/golang/dep/issues/999.

The problem, is in how dep does transitive dependencies. What it basically boils down to is, either this repo needs to update to v2 of blackfriday, or it needs to vendor it.

Because I was importing this repo, and it doesn't set a constraint on a version of blackfriday to use, I can't constrain it.

My "workaround" is I have to use a blank import to import blackfriday, before this package, and then I have set the constraint and vendor so dep doesn't pick up v2 of blackfriday.

It's a real mess, unfortunately.

[dmitshur]

What it basically boils down to is, either this repo needs to update to v2 of blackfriday, or it needs to vendor it.

I want to update to v2, but it's not a priority, so it won't be done very soon.

Until then, this package is using the latest "v1" version of blackfriday. Vendoring is not neccessary because it has little benefits—not enough to outweigh the downsides.

The problem is that the v2.0.0 tag makes dep think the latest version of github.com/russross/blackfriday package is v2, but that is not the intention of the blackfriday developers (see https://github.com/russross/blackfriday/tree/e40b8afb49fa3249ebf#versions). It's an unintentional coincidence. This is where the issue is. We need to figure out if it's better to fix it on blackfriday side (my guess) or dep side (less likely).

[markbates]

If they removed the v2.0.0 tag then this whole problem goes away. Dep sees that tag and assumes it's the "correct" version.

[dmitshur]

Another possible fix, if I understand things correctly, would be for my github_flavored_markdown package to include a Gopkg.toml file that requires v1 version of github.com/russross/blackfriday package.

That would fix the issue for you @markbates, right?

The problem with this solution is that it means every project that imports github.com/russross/blackfriday package and expecting v1 API would have to do this. This is not the kind of experience we the blackfriday developers intended when coming up with our v2 strategy. But maybe we should reconsider it in light of dep being a thing now, something it wasn't when we originally came up with the gopkg.in scheme for offering v2. /cc @rtfb

[markbates]

Yeah, right now everyone who uses this package and dep is going to have problems, unless they do a manual import of blackfriday and then lock the constraint down themselves.

Take this application:

package main

import _ "github.com/shurcooL/github_flavored_markdown"

func main() {
}

If you run dep init on it you get the following in the Gopkg.lock file:

[[projects]]
  name = "github.com/russross/blackfriday"
  packages = ["."]
  revision = "cadec560ec52d93835bf2f15bd794700d3a2473b"
  version = "v2.0.0"

Then, of course, it won't build:

# github.com/gobuffalo/depp/vendor/github.com/shurcooL/github_flavored_markdown
vendor/github.com/shurcooL/github_flavored_markdown/main.go:82: undefined: blackfriday.EXTENSION_NO_INTRA_EMPHASIS
vendor/github.com/shurcooL/github_flavored_markdown/main.go:83: undefined: blackfriday.EXTENSION_TABLES
vendor/github.com/shurcooL/github_flavored_markdown/main.go:84: undefined: blackfriday.EXTENSION_FENCED_CODE
vendor/github.com/shurcooL/github_flavored_markdown/main.go:85: undefined: blackfriday.EXTENSION_AUTOLINK
vendor/github.com/shurcooL/github_flavored_markdown/main.go:86: undefined: blackfriday.EXTENSION_STRIKETHROUGH
vendor/github.com/shurcooL/github_flavored_markdown/main.go:87: undefined: blackfriday.EXTENSION_SPACE_HEADERS
vendor/github.com/shurcooL/github_flavored_markdown/main.go:88: undefined: blackfriday.EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK
vendor/github.com/shurcooL/github_flavored_markdown/main.go:88: const initializer blackfriday.EXTENSION_NO_INTRA_EMPHASIS | blackfriday.EXTENSION_TABLES | blackfriday.EXTENSION_FENCED_CODE | blackfriday.EXTENSION_AUTOLINK | blackfriday.EXTENSION_STRIKETHROUGH | blackfriday.EXTENSION_SPACE_HEADERS | blackfriday.EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK is not a constant
vendor/github.com/shurcooL/github_flavored_markdown/main.go:104: undefined: blackfriday.Html

I believe, and I might be mistaken, if you add a Gopkg.toml like this:

required = ["github.com/russross/blackfriday"]

[[constraint]]
  branch = "master"
  name = "github.com/russross/blackfriday"

at the root of this repo, it should fix the problem.

You don't want to specify the v1 version, as that was cut a very long time ago and doesn't reflect the master branch of blackfriday.

[bpicode]

@markbates Drive-by comment:

Would it help if you define an override in Gopkg.toml?

[[override]]
  name = "github.com/russross/blackfriday"
  version = "1.4.0"

[markbates]

@bpicode I haven't tried, it might. The bigger problem is everyone using this project and dep will run into this problem, which is probably not the best experience for everyone.

[bpicode]

@markbates I agree that it is not a good experience. However the even bigger problem is that most projects do not have dependency management at all, leaving tools like dep to do ungrateful guesswork.

[dmitshur]

To clarify, this project has a specific form of dependency management in place (which is supported by me and working well, as can be seen here).

My intention is to make it work with dep as well, without any guessing required, after this specific issue is resolved.

[dmitshur]

I believe, and I might be mistaken, if you add a Gopkg.toml like this:

required = ["github.com/russross/blackfriday"]

[[constraint]]
  branch = "master"
  name = "github.com/russross/blackfriday"

at the root of this repo, it should fix the problem.

You don't want to specify the v1 version, as that was cut a very long time ago and doesn't reflect the master branch of blackfriday.

Makes sense, thanks.

@markbates, if this is time-sensitive for you, I can apply the suggested solution temporary to fix github_flavored_markdown for you until we find a long term solution in blackfriday. Otherwise, we'll wait until that.

[markbates]

@shurcooL I've patched Buffalo to fix this, so my "immediate" need is gone, but it definitely feels like a hack to fix it there.

I love this project, and just want to make sure no one else runs into this problem.

[sdboyer]

hi folks!

@bpicode is definitely right - we can't cover everything. we have to define some general rules, and one of those is preferring semver tags to branches. this is causing some growing pains now, but we expect that pain should largely subside once the community starts generating semver-tagged releases as a matter of course. the two cases it tends to hurt in are:

1. when there's just one or two really old semver releases
2. when the first semver release gets made, and it's incompatible with what was previously on the default branch (aka, this case)

@bpicode: Would it help if you define an override in Gopkg.toml?

an override would solve @markbates' problem, but we do try to discourage their use if possible. the ideal solution, from dep's perspective, is for this project to specify the a version constraint on github.com/russross/blackfriday.

I believe, and I might be mistaken, if you add a Gopkg.toml like this:

required = ["github.com/russross/blackfriday"]

[[constraint]]
  branch = "master"
  name = "github.com/russross/blackfriday"

at the root of this repo, it should fix the problem.

this would be an appropriate alternative to overrides for @markbates to exercise in his project. adding the required bit is equivalent to directly importing github.com/russross/blackfriday - dep now considers it a direct dependency, so he can constrain it himself in his Gopkg.toml

however, because this project DOES import github.com/russross/blackfriday, there'd be no need for @shurcooL to add it in a Gopkg.toml for this project. all that's needed is the [[constraint]] stanza.

there are two other alternatives, as well. if this project starts importing blackfriday at gopkg.in/russross/blackfriday.v1 instead of directly from github, then dep may do the right thing, as i believe that'll end up using master as the default branch, which i think will still be visited before the older tags blackfriday has.

blackfriday could also start using package import comments to enforce that its v2 is imported via gopkg.in - e.g.

package blackfriday // import "gopkg.in/russross/blackfriday.v2"

now, dep hasn't actually added the support for this yet (golang/dep#902), because people largely haven't reported needing it - but, i'd count this as such a report. anyway, what it will do is treat those package import comments as another satisfiability condition that must be met for a version to be acceptable.

the problem with this approach is that the v2.0.0 tag would need to be moved after introducing that comment, otherwise it'll still be the one that dep finds when no further constraints are specified.

[dmitshur]

Thanks a lot for the detailed and informative response @sdboyer! I think that should be sufficient information for us to find a good upstream solution in blackfriday.

[sdboyer]

always happy to help!

[rtfb]

Thanks everybody for reporting the issue and figuring out the details while I was away!

Having read all the material, I can see only one solution: retag v2 with v2.0 to avoid semantic versioning. When https://github.com/golang/dep/issues/902 is done, we can release v2.0.1 and get back to semver.

Or we could simply wait for https://github.com/golang/dep/issues/902 to get done. Or I can start hacking on https://github.com/golang/dep/issues/902 right away, but that would probably end up being a busier equivalent of waiting :-)

[sdboyer]

this sounds reasonable to me (as i noted over on the other issue) 👍

Or I can start hacking on golang/dep#902 right away, but that would probably end up being a busier equivalent of waiting :-)

🎉 🎉

honestly, i don't think it'd be that bad to do 😄 at least, not the additional parsing bits that #902 is focused on. incorporating a new satisfiability condition into the solver is trickier, but still probably pretty isolated - i'm happy to guide, or will do it myself if necessary.

[victorshinya]

Hello team, any update here? I am currently facing this issue with GFM and Dep. Any tips?

[victorshinya]

Hello team, any update here? I am currently facing this issue with GFM and Dep. Any tips?

I had to change dep to glide to work it

[dmitshur]

@victorshinya There are no updates on this as far as I know.

My suggested workaround would be to use Go modules that were added in Go 1.11. This package fully supports them as of PR #22.

[dmitshur]

Dep is deprecated by now, so there's nothing more to do here.