Enable line un-wrapping of Markdown for GitHub releases

[jwodder]

If the source for the entries in a Markdown changelog are wrapped to some number of characters, then when scriv uses them as the body of a GitHub release, GitHub will render the intra-paragraph newlines as hard line breaks, so that this:

- Foo all the bars, and make sure the gnusto is no longer cleesh.
  Fixes [#XXX](https://github.com/datalad/datalad-next/issues/XXX) via
  [PR #YYY](https://github.com/datalad/datalad-next/pull/YYY)
  (by [@GITHUBHANDLE](https://github.com/GITHUBHANDLE))

ends up looking like this:

• Foo all the bars, and make sure the gnusto is no longer cleesh. Fixes #XXX via PR #YYY (by @GITHUBHANDLE)

Hideous!

I would thus like to ask whether it's possible for scriv to add in some sort of automatic line un-wrapping for Markdown changelogs when using them as GitHub release bodies. One possible implementation (I'm not sure if it would work) would be to send the Markdown through pandoc with the --wrap=none option. Alternatively, something like re.sub(r'(?<=\S)(\r?\n[ \t]+)*\r?\n[ \t]*(?=\S)', ' ', body) might work for 90%+ of cases.

[nedbat]

Is it possible to simply not wrap your original changelog fragments?

[yarikoptic]

Then it becomes one long line, harder to edit etc. And for scriv it is simply different behavior between rst (ok to consume wrapped) and markdown (not ok).

[nedbat]

Yes, it's one long line, so you use word wrapping in your editor. I guess I'm not sure why scriv needs to fiddle with the line wrapping. How do you edit Markdown files now so that the hard line endings aren't a problem? Or, if you have hard line endings, then how do they not become a problem when the text is on GitHub? For example, what about long commit messages?

[yarikoptic]

Dear @nedbat thanks for the quick responses. Now that I am at the keyboard let me expand on my previous reply to answer your questions/clarify the situation. As a sidenote, I think that we are dealing with a bug or peculiarity of how github reneders markdown of the itemized lists with wrapped items (pycharm is also funny - typically rewraps nicely unless previous line has 2 trailing white spaces which it takes as a signal to have clear newline there... I must read more on "markdown conventions"). But let's get back to scriv and use its already existing behavior:

I guess I'm not sure why scriv needs to fiddle with the line wrapping.

Well, the question is kind of back to you since it already does that for .rst - why does it fiddle with the line wrapping for .rst?

And with that we can come up with answers to rest of the comments/questions you had by simply looking at https://raw.githubusercontent.com/nedbat/scriv/main/CHANGELOG.rst and sample changelog snippets as in randomly chosen https://github.com/nedbat/scriv/commit/3fa23cedcebc6b60563342ad61abbe38e7efd939

• on large screens it is much easier to look at paragraphed text flow in limited in width "column" instead of lines of varying lengths
• even if the IDE/editor (I use PyCharm, emacs, vim) visually wraps lines (might be a non-default mode) it would wrap them into the beginning of the next line, thus breaking the "visual formatting" of the paragraphs as in .rst/.md to have consistent leading indentation with spaces -- makes it harder to parse
• in our template (posted in original description here and a can be found in https://github.com/datalad/release-action/blob/master/changelog.d/templates/new_fragment.md.j2) we would like to place issue description on one line (or possibly multiple if long), the issue PR fixes on the next one and PR itself with author on the next one. It allows for quick navigation to the necessary to edit component simply by going up/down instead of jumping through the long line or arbitrary visual formatting to get to the desired component to edit.

So, even if there are ways and means to work with long lines, it is more cumbersome and you yourself do not do that for scriv changelog entries. That is why we were asking to establish similar behavior for .md fragments. As a result it would also adhere to the Principle of Least Surprise by having different behavior between .rst and .md input formats handling and how they would render on github.

[nedbat]

TBH, the re-wrapping for .rst is an accident: scriv uses pandoc to convert .rst to .md for GitHub releases, since GitHub will only accept markdown. In theory there's no need for pandoc to add .md changelogs to releases. I suppose we could explore using pandoc for .md->.md also, but I'd hate to require people to install pandoc if they are already writing markdown. And if they aren't required to install pandoc, then we have a condition to check? It sounds like it will be very messy.

[yarikoptic]

I see the reservation! May be it could become a config option?

[yarikoptic]

just for another little piece of motivation against one lines -- look at https://github.com/datalad/datalad/pull/7107/files#diff-4fd7661b8fcafe326f99613d1de1a7c9cd23e2f7456019eefc2f3bd3d18dd401R3 and think how it would be in 1 liner and suggest a recommendation for its change: would be more cumbersome. I really hope that there could be an option so we could simply rely on pandoc. An alternative for us might be then to add it to our release workflow to run pandoc on all those .md first ourselves before calling out to scriv combine, wdyt @jwodder ?

[yarikoptic]

FWIW, we can't do it "ourselves" since we do not want oneliners in CHANGELOG.md but want them only within github release notes since otherwise they look quite suboptimal, see e.g. https://github.com/datalad/datalad/releases/tag/0.17.9

[yarikoptic]

@nedbat so what about a config option to trigger consistent use of pandoc?

[yarikoptic]

@nedbat ping on this issue.