multiline comments not working when generating markdown

[lucaferranti]

Description

I have a file test_file.jl with content

#=
# Title

## Subtitle

some text
=#

a = 1

b = 2

# some more text

c = a + b

after calling Literate.markdown("test_file.jl") the output in test_file.md is

```@meta
EditURL = "<unknown>/test_file.jl"

#=

Title

# Subtitle

some text
=#

a = 1

b = 2

some more text

c = a + b

---

This page was generated using Literate.jl.

I guess this is not the expected behavior?

### System information

```julia
(jl_2QYfMW) pkg> st
      Status `C:\Users\lucaa\AppData\Local\Temp\jl_2QYfMW\Project.toml`
  [98b081ad] Literate v2.9.3

julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code.cmd -g

[lucaferranti]

Similar issue for markdown strings, using

md"""
# Title

## Subtitle

some text
"""

a = 1

b = 2

# some more text

c = a + b

and calling Literate.markdown("test_file.jl"; mdstrings=true)

produces

```@meta
EditURL = "<unknown>/test_file.jl"

md"""

Title

# Subtitle

some text
"""

a = 1

b = 2

some more text

c = a + b

---

This page was generated using Literate.jl.

[lucaferranti]

I think this is a windows related issue, because on linux both seem to work as expected. My guess would be that windows doesn't understand the regexes at these lines.

[fredrikekre]

I thought this would have been fixed by https://github.com/fredrikekre/Literate.jl/pull/167. What version of Literate are you using?

[lucaferranti]

2.9.3

[lucaferranti]

I played with the code a little, the current regexes are

    content = replace_multiline(r"^#=+\R^(\X*?)\R=+#$"m, content)
    if config["mdstrings"]::Bool
        content = replace_multiline(r"^md\"\"\"\R^(\X*?)\R\"\"\"$"m, content)
    end

removing the $ from the regexes makes it work. I think the reason is that if you have

#=
blah blah
=#

a = 1

then you have an invisible new line character after =# and hence the regex doesn't match it since the =#$ matches a line that ends with =#.

note: I haven't checked (yet) whether this change breaks the behavior on other OSs, I can test on a linux machine later today

[fredrikekre]

Yea I am working on a fix. Just confused why the tests I added in #167 pass...

[lucaferranti]

maybe it's because the current tests have

write(inputfile, "#=\r\nhello world\r\nhej världen\r\n=#")

so in that case =# is indeed at the end of the line. I changed that line to

write(inputfile, "#=\r\nhello world\r\nhej världen\r\n=#\r\na=1")

and then chunks[2].lines returns

["#=", "hello world", "hej världen", "=#", "a=1"]

after removing the $ from the regex the tests pass again