Support conversion of Delta to HTML and Markdown

[ArjanAswal]

This Library converts Zeyfr documents to HTML. There are many plugins that can help with the conversion of HTML to markdown. I was thinking that with some tweaking we can create our own converter too.

Or we can take another approach, we can first convert Delta to markdown like here and then to HTML. Again, this requires just some minor tweaking.

[singerdmx]

It is also possible to do it at server side. We found a Golang package to do it https://github.com/singerdmx/BulletJournal/commit/16fc1c2b82872a94a69880a711ed46377e9bb2d4

[singerdmx]

Note our underlying format is different than Notus

[singerdmx]

Zefyr's library won't work since it is missing a lot of attributes.

[ArjanAswal]

It is also possible to do it at server side. We found a Golang package to do it singerdmx/BulletJournal@16fc1c2

This could be a huge limitation for flutter apps looking to use this package. Any dart script that gets the job done will be very valuable for the flutter developers looking to work with this library.

[singerdmx]

You can modify Zefyr's library to submit a pull request if you want. Note that our data format is 100% sticking to the standard similar to what https://github.com/dchenk/go-render-quill did.

[singerdmx]

I mean you can copy Zefyr's code and make changes to it. It is much easier

[ArjanAswal]

@singerdmx In my opinion, there are only a few minor changes needed to be made in this code for it to work with this package.

Like replacing NotusAttribute with Attribute, and introducing getters and a few functions for the Style object.

[singerdmx]

  Map<String, String> _supportedElements = {
    "li": "block",
    "blockquote": "block",
    "code": "block",
    "h1": "block",
    "h2": "block",
    "h3": "block",
    "div": "block",
    "em": "inline",
    "strong": "inline",
    "a": "inline",
    "p": "block",
    "img": "embed",
    "hr": "embed",
  };

The above is what Zefyr supports. Yes, for flutter-quill it is https://github.com/singerdmx/flutter-quill/blob/master/lib/models/documents/attribute.dart

[ArjanAswal]

You can modify Zefyr's library to submit a pull request if you want. Note that our data format is 100% sticking to the standard similar to what https://github.com/dchenk/go-render-quill did.

That is why this library is leaps and bounds better than Zeyfr library in my humble opinion.

The above is what Zefyr supports. Yes, for flutter-quill it is https://github.com/singerdmx/flutter-quill/blob/master/lib/models/documents/attribute.dart

This is right. That is why only a few changes are needed to be made in this code before it can work with this library.

[ArjanAswal]

Here is my modified version (Doesn't work yet, needs a few getters and functions from the Flutter-quill library)

import 'dart:convert';

import 'package:flutter_quill/models/documents/attribute.dart';
import 'package:flutter_quill/models/documents/style.dart';
import 'package:quill_delta/quill_delta.dart';

class QuillMarkdownCodec extends Codec<Delta, String> {
  const QuillMarkdownCodec();

  @override
  Converter<String, Delta> get decoder =>
      throw UnimplementedError('Decoding is not implemented yet.');

  @override
  Converter<Delta, String> get encoder => _QuillMarkdownEncoder();
}

class _QuillMarkdownEncoder extends Converter<Delta, String> {
  static const kBold = '**';
  static const kItalic = '_';
  static final kSimpleBlocks = <Attribute, String>{
    Attribute.blockQuote: '> ',
    Attribute.ul: '* ',
    Attribute.ol: '1. ',
  };

  @override
  String convert(Delta input) {
    final iterator = DeltaIterator(input);
    final buffer = StringBuffer();
    final lineBuffer = StringBuffer();
    Attribute<String> currentBlockStyle;
    var currentInlineStyle = Style();
    var currentBlockLines = [];

    void _handleBlock(Attribute blockStyle) {
      if (currentBlockLines.isEmpty) {
        return; // Empty block
      }

      if (blockStyle == null) {
        buffer.write(currentBlockLines.join('\n\n'));
        buffer.writeln();
      } else if (blockStyle == Attribute.codeBlock) {
        _writeAttribute(buffer, blockStyle);
        buffer.write(currentBlockLines.join('\n'));
        _writeAttribute(buffer, blockStyle, close: true);
        buffer.writeln();
      } else {
        for (var line in currentBlockLines) {
          _writeBlockTag(buffer, blockStyle);
          buffer.write(line);
          buffer.writeln();
        }
      }
      buffer.writeln();
    }

    void _handleSpan(String text, Map<String, dynamic> attributes) {
      final style = Style.fromJson(attributes);
      currentInlineStyle =
          _writeInline(lineBuffer, text, style, currentInlineStyle);
    }

    void _handleLine(Map<String, dynamic> attributes) {
      final style = Style.fromJson(attributes);
      final lineBlock = style.get(Attribute.block);
      if (lineBlock == currentBlockStyle) {
        currentBlockLines.add(_writeLine(lineBuffer.toString(), style));
      } else {
        _handleBlock(currentBlockStyle);
        currentBlockLines.clear();
        currentBlockLines.add(_writeLine(lineBuffer.toString(), style));

        currentBlockStyle = lineBlock;
      }
      lineBuffer.clear();
    }

    while (iterator.hasNext) {
      final op = iterator.next();
      final lf = op.data.indexOf('\n');
      if (lf == -1) {
        _handleSpan(op.data, op.attributes);
      } else {
        var span = StringBuffer();
        for (var i = 0; i < op.data.length; i++) {
          if (op.data.codeUnitAt(i) == 0x0A) {
            if (span.isNotEmpty) {
              // Write the span if it's not empty.
              _handleSpan(span.toString(), op.attributes);
            }
            // Close any open inline styles.
            _handleSpan('', null);
            _handleLine(op.attributes);
            span.clear();
          } else {
            span.writeCharCode(op.data.codeUnitAt(i));
          }
        }
        // Remaining span
        if (span.isNotEmpty) {
          _handleSpan(span.toString(), op.attributes);
        }
      }
    }
    _handleBlock(currentBlockStyle); // Close the last block
    return buffer.toString();
  }

  String _writeLine(String text, Style style) {
    var buffer = StringBuffer();
    if (style.contains(Attribute.header)) {
      _writeAttribute(buffer, style.get<int>(Attribute.header));
    }

    // Write the text itself
    buffer.write(text);
    return buffer.toString();
  }

  String _trimRight(StringBuffer buffer) {
    var text = buffer.toString();
    if (!text.endsWith(' ')) return '';
    final result = text.trimRight();
    buffer.clear();
    buffer.write(result);
    return ' ' * (text.length - result.length);
  }

  Style _writeInline(
      StringBuffer buffer, String text, Style style, Style currentStyle) {
    // First close any current styles if needed
    for (var value in currentStyle.values) {
      if (value.scope == AttributeScope.line) continue;
      if (style.containsSame(value)) continue;
      final padding = _trimRight(buffer);
      _writeAttribute(buffer, value, close: true);
      if (padding.isNotEmpty) buffer.write(padding);
    }
    // Now open any new styles.
    for (var value in style.values) {
      if (value.scope == AttributeScope.line) continue;
      if (currentStyle.containsSame(value)) continue;
      final originalText = text;
      text = text.trimLeft();
      final padding = ' ' * (originalText.length - text.length);
      if (padding.isNotEmpty) buffer.write(padding);
      _writeAttribute(buffer, value);
    }
    // Write the text itself
    buffer.write(text);
    return style;
  }

  void _writeAttribute(StringBuffer buffer, Attribute attribute,
      {bool close = false}) {
    if (attribute == Attribute.bold) {
      _writeBoldTag(buffer);
    } else if (attribute == Attribute.italic) {
      _writeItalicTag(buffer);
    } else if (attribute.key == Attribute.link.key) {
      _writeLinkTag(buffer, attribute as Attribute<String>, close: close);
    } else if (attribute.key == Attribute.header.key) {
      _writeHeadingTag(buffer, attribute as Attribute<int>);
    } else if (attribute.key == Attribute.block.key) {
      _writeBlockTag(buffer, attribute as Attribute<String>, close: close);
    } else {
      throw ArgumentError('Cannot handle $attribute');
    }
  }

  void _writeBoldTag(StringBuffer buffer) {
    buffer.write(kBold);
  }

  void _writeItalicTag(StringBuffer buffer) {
    buffer.write(kItalic);
  }

  void _writeLinkTag(StringBuffer buffer, Attribute<String> link,
      {bool close = false}) {
    if (close) {
      buffer.write('](${link.value})');
    } else {
      buffer.write('[');
    }
  }

  void _writeHeadingTag(StringBuffer buffer, Attribute<int> heading) {
    var level = heading.value;
    buffer.write('#' * level + ' ');
  }

  void _writeBlockTag(StringBuffer buffer, Attribute block,
      {bool close = false}) {
    if (block == Attribute.codeBlock) {
      if (close) {
        buffer.write('\n```');
      } else {
        buffer.write('```\n');
      }
    } else {
      if (close) return; // no close tag needed for simple blocks.

      final tag = kSimpleBlocks[block];
      buffer.write(tag);
    }
  }
}

[singerdmx]

That is why only a few changes are needed to be made in this code before it can work with this library.

Hope so. We can add this ourself but it is lower priority for us - you may need to wait several months. Currently we are updating text/background color button in toolbar to show color attributes. If you cannot wait, you are welcome to submit a pull request. Let's keep this issue open until we implement it to keep track.

[singerdmx]

@ArjanAswal check out https://github.com/memspace/zefyr/pull/381/files

[ArjanAswal]

@singerdmx That PR you mentioned works with quill_delta: ^1.0.0 so there were some versioning conflicts but to resolve it I simply added notus to local files.

notusMarkdown.encode(controller.document.toDelta());

The above code doesn't work and shows this error:

The argument type 'Delta (where Delta is defined in C:\flutter\.pub-cache\hosted\pub.dartlang.org\quill_delta-2.0.0\lib\quill_delta.dart)' can't be assigned to the parameter type 'Delta (where Delta is defined in D:\Workspace\Projects\Apps\app\lib\notus\quill_delta.dart)'.

The quill_delta version is different that is the root cause of the problem.

[singerdmx]

OK, that is just for reference. As you know, our underlying data format are quite different and you cannot use notus for flutter-quill.

[ArjanAswal]

As you know, our underlying data format are quite different and you cannot use notus for flutter-quill.

This is unfortunately the crux of the matter. I actually found the way to convert Zeyfr document into flutter-quill document and vice-versa. My plan is to convert the flutter-quill doc to Zeyfr doc and then convert it into markdown.

Will keep you posted.

[ArjanAswal]

@singerdmx I have successfully created the Delta to markdown converter. Right now it has a few limitations, such as not able to convert 'color', 'background' and 'image' and 'strike' attributes (those attributes that flutter-quill has but Zeyfr doesn't) and leaves them. But it does convert successfully everything else.

It depends on quill_delta : 1.0.0 so how can I file the PR? Or should I create a package?

[singerdmx]

1.0.0 is just one file right? Just copy it over

[ArjanAswal]

1.0.0 is just one file right? Just copy it over

It also depends on notus and it is quite large.

[singerdmx]

Taking dependency on notus is fine. There is no conflict.

[ArjanAswal]

Taking dependency on notus is fine. There is no conflict.

There is, as notus itself depends on quill_delta 1.0.0. The best way is to create a package in my opinion.

[singerdmx]

Ok it seems so

[ArjanAswal]

I have created the quill_markdown package and it converts quill to markdown and vice-versa. Here is the repo.

[singerdmx]

I think it is private repo

[ArjanAswal]

However, there are a few limitations. This package works by converting flutter-quill document to Zeyfr and then converts it into markdown. This is a temporary solution until this library implements some drastic changes to its Style Attributes so that it can be parsed to markdown. For now, only those attributes that are common to both flutter-quill and Zeyfr are supported. Luckily that covers most of the Attributes.

For markdown to quill conversion, the markdown format needs to be very basic. Any fancy attributes are not supported yet.

Let's keep this issue open till we implement our own conversion code without being at the mercy of Zeyfr. Fortunately, for now, quill_markdown gets the job done.

[singerdmx]

I cannot access this repo

[ArjanAswal]

I cannot access this repo

Check now.

[singerdmx]

I can access now

[singerdmx]

So this is converting to markdown instead of html?

[ArjanAswal]

So this is converting to markdown instead of html?

There are existing packages for the conversion of md to HTML. So you can first convert delta to markdown and then to HTML or vice-versa. But it all depends on the HTML tags used(whether they are compatible with Quill) and also on the converted markdown attributes as well.

[ArjanAswal]

Pub Package link

[KrishnaBros]

You guys are working on it seriously or not? Please atleast tell this

[ArjanAswal]

No not right now. Feel free to submit a PR if you want. In the meantime you can use the package.

[KrishnaBros]

Ok Thanks For Informing

No not right now. Feel free to submit a PR if you want. In the meantime you can use the package.

[friebetill]

I created a new converter delta_markdown. It's an adapted version of an existing Markdown - Delta (Zefyr) converter so that it can now convert between Markdown - Delta (flutter_quill). It works in simple cases and I will probably extend it further in the coming weeks. One advantage is that images are supported.

Currently it only works with flutter_quill ^0.3.5, as I am in the process of migrating to Flutter 2.

[singerdmx]

@friebetill if it works out, you can update our REAMDE to add a section to use yours

[kdela]

In case if someone is looking for workaround

import 'dart:convert';

import 'package:delta_markdown/delta_markdown.dart';
import 'package:flutter_quill/models/quill_delta.dart';
import 'package:markdown/markdown.dart';

String quillDeltaToHtml(Delta delta) {
  final convertedValue = jsonEncode(delta.toJson());
  final markdown = deltaToMarkdown(convertedValue);
  final html = markdownToHtml(markdown);

  return html;
}

[friebetill]

As it's possible with the library delta_markdown to convert between delta and markdown and with the library markdown to convert between markdown and html, I close the issue.

If you disagree, please write a comment and I will reopen it.

[njovy]

@friebetill You saved my day. Much appreciate it!

[Hamza5]

@friebetill instead of dropping support from many features (like colors) because they are not supported in Markdown, you should allow the user to customize it as he wants. For example, the user may like to encode them as elements with a style, since using HTML inside Markdown is very popular.

[mrverdant13]

For us to be able to create other extensible packages, it would be handy to isolate the delta implementation used for flutter_quill in a separate package.

[TarekkMA]

I have created a new library that can convert from and to markdown that addresses the following:

• Uses latest version of markdown library.
• ~95% test coverage with complex full markdown articles and test cases from GFM Spec. Also, I used @friebetill tests from delta_markdown
• Support for:
  • inline code
  • strikethrough
  • image
  • horizontal rule
• Ability to define custom behavior (styles/embeds) (currently I'm working on adding markdown table as an embeddable element in quill)

Contribution and feedback are most certainly welcome.

Library Repo: https://github.com/TarekkMA/markdown_quill Pub Link: https://pub.dev/packages/markdown_quill

I will release it soon to pub.dev.

[Hamza5]

I have created a new library that can convert from and to markdown that addresses the following:

* Uses latest version of [markdown](https://pub.dev/packages/markdown) library.

* ~95% test coverage with complex full markdown articles and test cases from [GFM Spec](https://github.github.com/gfm/). Also, I used @friebetill tests from [delta_markdown](https://pub.dev/packages/delta_markdown)

* Support for:

  * inline code
  * strikethrough
  * image
  * horizontal rule

* Ability to define custom behavior (styles/embeds) (currently I'm working on adding markdown table as an embeddable element in quill)

Contribution and feedback are most certainly welcome.

Library Repo: https://github.com/TarekkMA/quill_markdown

I will release it soon to pub.dev.

Good luck bro,

Unfortunately, I found that Markdown is not the most suitable language for me since it needs a lot of customization, so I switched to HTML because it has all the rich text stuff out-of-the-box. I am currently writing my own code to export to and import from HTML, using the available parser library called html.

[TarekkMA]

@Hamza5 thank you, in my current project markdown is a requirement so I will need to stick to it.

[friebetill]

Nice @TarekkMA! I will try your version and if it's better than my hacky package, I will link to your package, deprecate my package and archive my repository.

[TarekkMA]

Published on pub https://pub.dev/packages/markdown_quill