search

fletcher / MultiMarkdown-4

This project is now deprecated. Please use MultiMarkdown-6 instead!
https://github.com/fletcher/MultiMarkdown-5
Other
306 stars 59 forks source link

Transclusion of files with special characters in filenames doesn't work on Windows #91

Closed mn4367 closed 9 years ago

mn4367 commented 9 years ago 
multimarkdown.exe --version
MultiMarkdown version 4.6 ...

transclusion.md:

Author: Me
Date: Today
Title: Test

Lorem ipsum dolor ...

{{pläne.md}}

Lorem ipsum dolor sit ...
multimarkdown.exe -f -t html -o transclusion.html transclusion.md

transclusion.html

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8"/>
    <meta name="author" content="Me"/>
    <meta name="date" content="Today"/>
    <title>Test</title>
</head>
<body>

<p>Lorem ipsum dolor &#8230;</p>

<p>{{pläne.md}}</p>

<p>Lorem ipsum dolor sit &#8230;</p>

</body>
</html>

Although pläne.md exists it is ignored. This is true at least on a german Windows 7 installation (using NTFS), but I think the same will happen with all Windows locales. Maybe this has to do with using fopen for opening files. I believe it could be fixed by using CreateFile or _wfopen instead for the Windows version of MMD, but it's probably not easy to do :-(.

fletcher commented 9 years ago

I'll leave this open, but no idea if/when I'll do anything about it. This seems to be a Windows issue, and it's hard for me to get excited about figuring out workarounds so that standard stuff works on Windows.... Would much rather put my time into new features, things like that.

It works fine for me on OS X, and presumably on *nix (though I haven't tested it there).

It's really time for Windows to join the modern world. They should have rebuilt Windows on top of a *nix like OS X, and then maybe it would be an OK operating system.... ;)

If someone comes up with a fix, I'll look at it. In the meantime, looks like Windows users should avoid non-ASCII characters in filenames if using transclusion,

mn4367 commented 9 years ago

I'm using OS X as my main system, too, but in this case it's not fair to blame it on Windows; NTFS uses UTF-16 for filenames, so in this regad it is already in the modern world and it is as standard as using UTF-8 (like OS X and, as far as I know, Linux do).

Anyway, I'm not a C programmer but I'll try to implement a fix.

fletcher commented 9 years ago

If the problem is UTF-16 instead of UTF-8, wouldn't that cause all filenames to fail?

mn4367 commented 9 years ago

It obviously doesn't, but this is probably covered by the the internals of fopen. I think fopen will work with characters in the ASCII range but not beyond.

abordercollie commented 9 years ago

I am not sure but perhaps this thread may be of interest. ownCloud has run into file name issues on OS X, unfortunately, and it may be related:

https://github.com/owncloud/core/issues/2377 https://github.com/owncloud/core/issues/2377

It seems they may have solved it just recently with Yosemite.

On Nov 25, 2014, at 10:01 AM, mn4367 notifications@github.com wrote:

It obviously doesn't, but this is probably covered by the the internals of fopen. I think fopen will work with characters in the ASCII range but not beyond.

— Reply to this email directly or view it on GitHub https://github.com/fletcher/MultiMarkdown-4/issues/91#issuecomment-64403043.

mn4367 commented 9 years ago

@abordercollie, thanks, but on OS X there is no problem, it's related to opening files with special characters on Windows.

fletcher commented 9 years ago

It's interesting. When I have struggled with UTF16 in the past, it's a mess. Even ascii characters are "double spaced" with empty bytes in between.

Perhaps they filter out empty bytes??

Sent from my iPhone

On Nov 25, 2014, at 9:01 AM, mn4367 notifications@github.com wrote:

It obviously doesn't, but this is probably covered by the the internals of fopen. I think fopen will work with characters in the ASCII range but not beyond.

— Reply to this email directly or view it on GitHub.

mn4367 commented 9 years ago

Honestly I don't know, so I'm a bit relcuctant to speculate about this. Maybe fopen talks to some sort of compatibility layer which enables opening files with ASCII characters, but I really don't know.

May I ask what development environment you use for the Windows version? I currently don't have any C develpment tools in my virtual machine, so any hints to speed up the setup for this is very welcome. Thanks.

fletcher commented 9 years ago

I use an Ubuntu virtual machine and used the directions from the README for peg-markdown to setup a MinGW environment for compiling.

It was actually surprisingly easy to do.

I don't use windows enough to have a native development tool chain.

Sent from my iPhone

On Nov 25, 2014, at 3:10 PM, mn4367 notifications@github.com wrote:

Honestly I don't know, so I'm a bit relcuctant to speculate about this. Maybe fopen talks to some sort of compatibility layer which enables opening files with ASCII characters, but I really don't know.

May I ask what development environment you use for the Windows version? I currently don't have any C develpment tools in my virtual machine, so any hints to speed up the setup for this is very welcome. Thanks.

— Reply to this email directly or view it on GitHub.

mn4367 commented 9 years ago

Thanks!