search

patrickdavey / vimwiki_markdown

A gem to use for converting vimwiki markdown files to html.
MIT License
144 stars 16 forks source link

The conversion will fail when filename of Markdown file is in Chinese. #35

Closed zzhirong closed 3 years ago

zzhirong commented 3 years ago

It will not works when the filename of the Markdown file being converted is in Chinese, I spent some time digging into the source code, and found the parameterize function which is called in output_fullpath returns empty string when the object string is in Chinese. I am not familiar with Ruby, so my dirty solution is removing the parameterize function call, but I don't think it is a perfect solution.

patrickdavey commented 3 years ago

Darn, that's annoying. Can you supply me with a filename that ought to work? I'll see if I can make it work.

I'm ashamed to admit that I know nothing about what sort of filenames I should be supporting on a chinese filesystem! I always cheat and (try) to make sure that everything is ascii, but, obviously that isn't going to work for you.

Anyway, if you can supply me with a filename I'll try to ensure that it at least works as a test-case.

zzhirong commented 3 years ago

Create a file whose filename is "世界", then you can reproduce the bug. I also upload a sample file 世界.txt, you may need to change the file's extension to "md", since the GithHub doesn't allow me to attach file with "md" extension. By the way, the Chinese has no case.

patrickdavey commented 3 years ago

I'm going to have to think about this one, and how to do it nicely. I'm sorry that it doesn't work (at all!) with Chinese.

I did get merged into vimwiki some support for matching (ish) rails's activesupport parameterize method which you can see here, so, I don't really want to change the default use of parameterize. Also, changing the default behavior now would break anyones site that uses vimwiki-markdown.

The setting needs to be user configurable somehow, but, that's not even that simple because I think I'm fairly restricted in what I can pass to vimwiki_markdown because of how it is called from vimwiki itself.

I could assume users are on mac/linux and have some kind of environment variable passed, and then use that to switch out the parameterize method. Seems pretty ugly though.

If you have any suggestions of a nice way to do this I'd love to hear it!

zzhirong commented 3 years ago

Maybe you could check the return result of parameterize method, if the result is empty, then you should keep the origin instead of a empty one.

patrickdavey commented 3 years ago

I have made that change and published 0.7.0 of the gem. Let me know if it fixes your issue :)

zzhirong commented 3 years ago

Now files with filenames in Chinese can be converted correctly. But I also found a minor issue when It come to a filename mixed in Chinese and some alphabets or numbers, the Chinese characters were completely tripped, for example, a note with a filename "世界1.md" which will be converted to a html file with name "1.html". So I think it is maybe a good idea to keep the origin filename when it encounters a filename which contains characters beyond ascii. Anyway, thank you for your effort.