Double-hyphens in post URLs reduce to single-hyphens; regression introduced in v0.50 causes links to break

kaushalmodi commented 4 years ago

What version of Hugo are you using (`hugo version`)?

$ hugo version
Hugo Static Site Generator v0.70.0-7F47B99E linux/amd64 BuildDate: 2020-05-06T11:18:50Z

Does this issue reproduce with the latest release?

Yes

Expected behavior: If the content file is named content/foo--bar.md, the page URL is expected to be <site>/foo--bar/.

Observed behavior: If the content file is named content/foo--bar.md, the page URL becomes <site>/foo-bar/ (note that the double-hyphen becomes single-hyphen).

I believe this regression was caused ~~after the switch to Goldmark.~~ in https://github.com/gohugoio/hugo/commit/e421696d.

Recipe to reproduce this issue

git clone https://gitlab.com/hugo-mwe/double-hyphens-in-post-urls (Content file: https://gitlab.com/hugo-mwe/double-hyphens-in-post-urls/-/blob/master/content/post--with--double--hyphens.md )
cd double-hyphens-in-post-urls
hugo

Above demo site is also deployed at https://determined-leakey-137470.netlify.app/ .

Observed output

In public/index.html, we will see:

    <body lang="en">

            <p>Page link = /post-with-double-hyphens/</p>

Expected output

    <body lang="en">

            <p>Page link = /post--with--double--hyphens/</p>

Same issue reported by other users

https://discourse.gohugo.io/t/force-hyphens-in-url/23958

Seriousness of this regression

I have quite a few pages with double-hyphens.. Double hyphens are used as idea separator in URLs. Here are few examples:

posts/deeply-nested-org-todo-headings--h1.md
posts/deeply-nested-org-todo-headings--h2.md
posts/deeply-nested-org-todo-headings--h5.md
posts/deeply-nested-org-todo-headings--h6.md
posts/deeply-nested-org-todo-headings--default-h.md
posts/filling-not-preserved-for-chinese-characters--preserve-filling-off.md
posts/filling-not-preserved-for-chinese-characters--preserve-filling-on.md

I link to these pages for references, and now they break because now double-hyphens are auto-reduced to single-hyphens.

Can the number of hyphens specified by the user in the content file name be preserved?

bep commented 4 years ago

I believe this regression was caused after the switch to Goldmark.

No, I think this is how it has been behaving for several years.

kaushalmodi commented 4 years ago

Indeed, this change happened about 1.5 years back in https://github.com/gohugoio/hugo/releases/tag/v0.50 .

Sincere request to allow users to retain their hyphens where they need them.

I only landed up on this today as I was tracing through one of the old ox-hugo tests linked in old issues.

kaushalmodi commented 4 years ago

Making the best guess from the changelog.. ~~probably this caused it?: https://github.com/gohugoio/hugo/commit/fae48d7457de96969ec53349194dcbfa45adc269 /cc @moorereason~~

Update: It's actually this: https://github.com/gohugoio/hugo/commit/e421696d

Move the "double hyphen and space" logic into UnicodeSanitize

The last bullet may be slightly breaking for some that now does not get the "--" in some URLs, but we need to reduce the amount of URL logic.

kaushalmodi commented 4 years ago

@bep This can be made a non-breaking fix if the double-hyphens are retained at least when the user has explicitly set the permalinks option to use the :filename. Example:

[permalinks]
  "/" = "/:filename/"

Right now, even with this option set, when the file name is post--with--double--hyphens.md, the permalink is /post-with-double-hyphens/. Can the filename faithfulness be retained with the :filename used for permalinks option?

Thanks.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

kaushalmodi commented 4 years ago

@bep Can you please look at this issue? This one issue is still bugging me.

github-actions[bot] commented 2 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

gohugoio / hugo