Implement yaml frontmatter

[ghost]

Hi,

Love your script. Useful, simple, worked perfectly from what I have seen by a quick look at some of the 500+ notes it just converted. I wish it was this easy to get my other notes out of OneNote! I've recently swapped over to Obsidian and there is a growing community there looking for ways to extract their notes into markdown, I'll put a post on their forum about this script.

Can you please review this pull request? I have implemented the addition of a YAML header as an option. Many of the latest generations of md readers and systems like Obsidian and Typora allow the use of YAML headers for 'front matter' in markdown. This lets you tidy away information that is not key to the core of the note/document.

I also tweaked a couple of typos in the comments (my IDE has a spell checker else I would never have noticed)

I'll be passing a couple of smaller changes in the next day or so.

Thanks for the script and thanks for having a look at this request

Kevin

[Maboroshy]

I like your implementation. It is also compatible with QOwnNotes app that was my first target. But maybe we should reorganize the option settings. There are now twice of them. Maybe we could implement primary in-text/YAML option, and make YAML follow options previously used for in-text. Except title. I don't see use case for both in-text and YAML data at the same time. What do you think about it?

[ghost]

Hi,

Yes I agree. I didn't want to mess with your approach too much to start with :-)

How about

#  Select meta data options
meta_data_in_yaml - False  # True a YAML front matter block will contain the following meta data items.  # False any selected meta data options below will be in the md text
insert_title = True  # True will add the title of the note as a field in the YAML block, False no title in block.
insert_ctime = True  # True to insert note creation time in the YAML block, False to disable.
insert_mtime = True  # True to insert note modification time in the YAML block, False to disable.
tags = True  # True to insert an array of tags, False to disable
tag_prepend = ''  # string to prepend each tag in a tag list inside the note, default is empty
tag_delimiter = ', '  # string to delimit tags, default is comma separated list
no_spaces_in_tags = False  # True to replace spaces in tag names with '_', False to keep spaces

#Select file link options
links_as_URI = False  # True for file://link%20target style links, False for /link target style links
absolute_links = True  # True for absolute links, False for relative links

# Select File/Attachments/Media options
media_dir_name = 'media'  # name of the directory inside the produced directory where all images and attachments will be stored
md_file_ext = 'md'  # extension for produced markdown syntax note files
creation_date_in_filename = False  # True to insert note creation time to the note file name, False to disable.

Obviously the first line meta_data_in_yaml - False can be inverted if you wish. Let me know if the above or inverted first line is what you would like and I'll commit updated code into the pull request.

Now I digress a little but in a way this is related... (The below probably would be better in the discussions section if you turn it on, or if you want I can put this in the issues list for further discussion..)

The above approach of avoiding duplication and reusing options also helps with some additional features I am looking to implement (a couple are already written see this project board in my fork). For example one of these is adding the option of a GitHub flavoured markdown as this would allow image sizing to be included on the image links. (I found a post in 2016 that said QownNotes supports sizing but could not see anything in their documentation and their docker demo would not spool up last night to have a look at the actual program). This adds choices of 1 new md standard and 3 different formats of image links.

A lot of the fixes are not bugs in the code, but in the weird HTML synology produces and pandoc can not convert.....Is it just me or are bullet points not working properly? The first level has no bullets...

So as these new features add an optional setting and the list it will get longer and longer.. so I though about having a few choices at the beginning that would override all the settings and users can choose a 'standard' for their note software of choice. But also allowing power users to pick and choose. I see a lot of non-technical people struggle with scripts and it is good to give them a simple option.

Something like this

# Select a standard format for a markdown note system.  
# If all are False The manual selection of options will be used
# If any are True the manual selection is ignored  The first True in the list will be used
qownnotes = False
obsidian = False
github_flavoured_markdown = False

#  Manually selection of options
#  Select meta data options
meta_data_in_yaml - False  # True a YAML front matter block will contain the following meta data items.  # False any selected meta data options below will be in the md text
insert_title = True  # True will add the title of the note as a field in the YAML block, False no title in block.
insert_ctime = True  # True to insert note creation time in the YAML block, False to disable.

Obviously the code gets a little more complicated as there are more choices some refactoring to group them etc will help. I'm thinking refactoring as functions not OOP. The code has probably 3 main actions that could be split into functions .

1. Pre-pandoc processing
2. Pandoc conversoin
3. Post-pandoc conversion

These 3 functions that use individual functions as needed based on the options. We could use inner functions to keep these sub processing steps grouped in the above 3 functions until we know we want to reuse them and then move them out of the function they are in .

code is always clearer than words (or should be) for example using inner functions...

def pre_processin(page_content):
    def call_first_inner_function_for_that_option():
        # do some stuff

    def call_second_inner_function_for_that_option():
        # do some different stuff

    def call_third_inner_function_for_that_option():
        # do some different stuff

    if option:
        call_first_inner_function_for_that_option()
        call_second_inner_function_for_that_option()

    if second_option:
        call_first_inner_function_for_that_option()
        call_third_inner_function_for_that_option()

    return page_content

I know this is quite different from the current 'script' approach but the complexity is getting higher so some code gardening will help for clarity especially as additional options/features are added.

Ok for a refactor? I'm happy to have a go and combine your thoughts into a refactor as well.

Any input on the project board also very welcome. The order in the todo column is the current order I'm thinking of working on them and I believe everything in the todo list is achievable, as I have a vague idea of the solution for all of them, and would hope they are all done quite quickly.

To generate the list I made. test notebook with each feature synology could put on a page and then converted it to see what is supported/works or not yet supported/works. As I have a completionist personality I then have to do everything I see even the charts, which are so awful to use I can't believe anybody actually uses them! But there will be one person out there crazy enough to use them and they will appreciate the capability :-)

I've attached a zip of the test example showing the input and output (input is attached to the test page.md as pdf.). You can also see that Note Station can't even get its own data out cleanly to a pdf extra line breaks all over the place.

1 - Strict markdown - Qown Notes defaults.zip

Kevin

[Maboroshy]

I agree with your options vision. With your refactor proposals it's a bit more complicated.

Your vision is great, it's a whole new level. But it will more like rewrite than refactoring. For me this project hit all the targets I've set when I started it. I'm mostly maintaining it now adding some small features users ask about.

I think that I will slow you down on you journey. I suggest you to make a complete fork of my project or start from the scratch using any useful code you can get from it. I will be glad if this project has a "descendant" that will become more successful.

[ghost]

Hi, I have updated the options as described above and updated the readme to match.