feat: support transclude embedded links with "!"

[aarnphm]

Is your feature request related to a problem? Please describe.

Apparently this is a valid context to embed links within the vault

![pdf](./local.pdf)

![markdown](./relative/path.md)

![markdown](thoughts/index)

Describe the solution you'd like

Need to update the regex under ofm.ts

Additional context

See https://discord.com/channels/927628110009098281/1284799251821625354

[saberzero1]

Currently embed cases in ofm.ts are tied to the wikilinks option. ![...](...) is not a wikilink, ![[...]] is. Do we want to change the wikilink regex to accommodate this (despite not being a wikilink), or do we want to separate the embed functionality from the wikilink functionality?

[aarnphm]

Separate would be best.

I don't know you can embed files through ![...](...).

[saberzero1]

In Obsidian you can embed the following file types by default:

• Markdown files: md
• Image files: avif, bmp, gif, jpeg, jpg, png, svg, webp
• Audio files: flac, m4a, mp3, ogg, wav, webm, 3gp
• Video files: mkv, mov, mp4, ogv, webm
• PDF files: pdf

Obsidian lets you embed by adding an exclamation mark ! to any internal link.

Obsidian defines internal links like this:

• Wikilink: [[...]]
• Markdown: [...](...)

So yeah, every wikilink written like [[link|alias]] is functionally identical to [alias](link) to Obsidian, including embed cases.

I'm unsure if we want full vanilla Obsidian support (Obsidian Flavored Markdown + 'default' Markdown) or full Obsidian Publish support (What works in Obsidian Publish should work on Quartz).

Either way, I feel like we should probably do a larger pass to update/rework/fix the Obsidian parser to align it with the full Obsidian Markdown Syntax, as these small difference between Obsidian and Quartz keep popping up. If that is something you and @jackyzha0 would like, I could work on a proposal for a rewrite.

[aarnphm]

The problem with this is that we support many variants of markdown syntax (IIRC GFM, Obsidian, oxhugo, roam).

Please propose a simple interface that allows us to manage this better. For link parsing, maybe we can have a different implementation for each flavor.

right now Obsidian have pretty heavy coupling atm.

[saberzero1]

The problem with this is that we support many variants of markdown syntax (IIRC GFM, Obsidian, oxhugo, roam).

Please propose a simple interface that allows us to manage this better. For link parsing, maybe we can have a different implementation for each flavor.

Write now Obsidian have pretty heavy coupling atm.

My first thought is to split the markdown parsing into many small components, with every component parsing a small amount of markdown.

For example, we could have separate small components for wikilinks, tables, font styling (cursive, bold, underline, etc.), embeds, list items, checkboxes, etc.

We could provide a default template for each of Obsidian, Org mode (oxhugo), Roam, GitHub, whatever else we currently support. These would contain the default settings and order for parsing each flavor. Users with specific wishes (for example, Org mode in Obsidian, or Roam in Obsidian) could configure their own template to cherry pick their desired parsing features.

This would also allow us to more easily extend the markdown parsing, as well as allow users to more easily extend the default functionality.

[aarnphm]

My first thought is to split the markdown parsing into many small components, with every component parsing a small amount of markdown.

For example, we could have separate small components for wikilinks, tables, font styling (cursive, bold, underline, etc.), embeds, list items, checkboxes, etc.

We used to do this is v3 with partial templates. for parsing items ig we can refactor it out.

We could provide a default template for each of Obsidian, Org mode (oxhugo), Roam, GitHub, whatever else we currently support. These would contain the default settings and order for parsing each flavor. Users with specific wishes (for example, Org mode in Obsidian, or Roam in Obsidian) could configure their own template to cherry pick their desired parsing features.

a small ask is to keep the current behaviour desirable, meaning

Plugin.ObsidianFlavorMarkdown(),
Plugin.GitHubFlavorMarkdown(),

to keep working.

Though it sounds like to me it is going to be a big PR. feel free to work on this incrementally at your own time :))

Thanks a bunch for taking the initiative!

p/s: let's open a meta issue to keep track of these smaller items once we got confirmation from jacky

[saberzero1]

My first thought is to split the markdown parsing into many small components, with every component parsing a small amount of markdown.

For example, we could have separate small components for wikilinks, tables, font styling (cursive, bold, underline, etc.), embeds, list items, checkboxes, etc.

We used to do this is v3 with partial templates. for parsing items ig we can refactor it out.

We could provide a default template for each of Obsidian, Org mode (oxhugo), Roam, GitHub, whatever else we currently support. These would contain the default settings and order for parsing each flavor. Users with specific wishes (for example, Org mode in Obsidian, or Roam in Obsidian) could configure their own template to cherry pick their desired parsing features.

a small ask is to keep the current behaviour desirable, meaning

Plugin.ObsidianFlavorMarkdown(),
Plugin.GitHubFlavorMarkdown(),

to keep working.

Though it sounds like to me it is going to be a big PR. feel free to work on this incrementally at your own time :))

Thanks a bunch for taking the initiative!

p/s: let's open a meta issue to keep track of these smaller items once we got confirmation from jacky

Sounds good to me. I'll await Jacky's confirmation.

[exodrifter]

Hi there, I was the one who originally brought this up on the Discord server. I'm including some of my research here in case it will be helpful (note that I'm not terribly familiar with typescript or nodejs):

I tried to edit quartz/plugins/transformers/ofm.ts to support internal markdown link transcludes but I believe I determined that:

• remark is the library responsible for creating the mdast
• remark is compliant with Commonmark, which defines the ![]() syntax as an image
• So, if you try updating the wikilink regex or adding another replacement regex in ofm.ts, it won't work because the ![]() has already been turned into an image in the mdast.

If you try to parse a document containing ![](test.md), before mdastFindReplace(tree, replacements) is called, you'll have an element in the mdast tree that looks like this:

{
  "type": "paragraph",
  "children": [
    {
      "type": "image",
      "title": null,
      "url": "test.md",
      "alt": "",
      "position": [Object]
    }
  ],
  "position": [Object]
}

It looks like Obsidian is not compliant with Commonmark here (and it's not terribly surprising because it wouldn't be the first time I've seen Obsidian do this). So, if you want this to work, I believe you'd either have to write some code to look for images in the mdast and then convert them into links or replace remark with something else.

I'm in agreement with @saberzero1 that it would be better for the markdown parsing to be made up of smaller composable components.

[aarnphm]

I feel like exo brought up a pretty good point in discord. At which point should Quartz considered "compatible" with Obsidian format. I would also rather for it to be compliant with Commonmark.

What we can change is saying Quartz is "partially compatible" with Obsidian, and just document for ![]() to not be supported :))

[saberzero1]

I feel like exo brought up a pretty good point in discord. At which point should Quartz considered "compatible" with Obsidian format. I would also rather for it to be compliant with Commonmark.

What we can change is saying Quartz is "partially compatible" with Obsidian, and just document for ![]() to not be supported :))

I feel like this would be something that could be configurable when we rework the parser.

[exodrifter]

@aarnphm My opinion is that it will be better if Quartz is either fully compliant with Commonmark or fully compliant with Obsidian and not somewhere in between. If the project would like to commit to being partially complaint with both, I would point out that there are more incompatabilities between Commonmark and OFM you'd want to document:

# Which one should Quartz do?
![800](/url) # Alt text in Commonmark
![800](/url) # Width of the image in OFM

As I suggested in the Discord server, I would rather be complaint with Commonmark. However, I think there are going to be a lot of edge cases like this which will trip up new Quartz users who are used to Obsidian. If I could put my two cents in, I think it would be better for Quartz to be compatible with Obsidian, especially if it would like to keep the claim that it can "seamlessly interoperate with Obsidian". It was appealing to me and it was one of the reasons I started using Quartz.

[saberzero1]

@aarnphm My opinion is that it will be better if Quartz is either fully compliant with Commonmark or fully compliant with Obsidian and not somewhere in between. If the project would like to commit to being partially complaint with both, I would point out that there are more incompatabilities between Commonmark and OFM you'd want to document:

# Which one should Quartz do?
![800](/url) # Alt text in Commonmark
![800](/url) # Width of the image in OFM

As I suggested in the Discord server, I would rather be complaint with Commonmark. However, I think there are going to be a lot of edge cases like this which will trip up new Quartz users who are used to Obsidian. If I could put my two cents in, I think it would be better for Quartz to be compatible with Obsidian, especially if it would like to keep the claim that it can "seamlessly interoperate with Obsidian". It was appealing to me and it was one of the reasons I started using Quartz.

Why not support both?

I'm going to propose a change to the markdown parsing to support the current parsers separately. That way user can choose what flavor of markdown they want to use, or even if they want their pick and choose whatever combination of parsing features between them.