igormironchik / md4qt

Markdown parser for Qt6 or ICU
16 stars 1 forks source link

Can you show an example of opening a markdown file and saving as an html file #2

Closed Light-Wizzard closed 1 year ago

Light-Wizzard commented 1 year ago

Looking at your example it seems I need to cast to get the information in the iterator (it) you use, would I also need to make a case for each type?

There must be a better way, but I did not figure it out.

I want to use this in a Qt Desktop C++ Widget App, in an editor, so I want to pass in the text and not the file name and return the text, I started off by using the Qt example for the Markdown editor, but it is way too slow, it uses JavaScript.

Most code I have seen only deals with files, which means I have to save the file, every time I enter a character so that I can update the HTML previewer, which is a Web Engine Widget, so I will have to write a wrapper around them to deal with this issue because this makes the editor GUI slow, plus the hourglass for saves, so I need figure out a better solution, and why I am looking at your code instead of C, only thing is your example does not help me much.

The App I am working on is at https://github.com/Light-Wizzard/QtWidgetCpackInstaller, the app is just an Editor, and the focus is on deploying this application using cmake, cpack, and NSIS, in this case, I want to use your parser to keep this a Qt example application for deploying Web Engine enabled apps.

The Qt Web Engine Widget Markdown editor uses a JavaScript implementation that is too slow, I would have to figure out how to replace that parser with yours, to know how much faster it is, my guess is that it will be fast enough.

Thanks for any help.

igormironchik commented 1 year ago

Hello.

You can parse text from QTextStream, so you don't need to give an existing file. You can store Markdown text as you need in memory and use alternative method to parse from stream:

//! \return Parsed Markdown document.
std::shared_ptr< Document< Trait > > parse(
    //! Stream to parse.
    typename Trait::TextStream & stream,
    //! This argument needed only for anchor.
    const typename Trait::String & fileName );

As shown in example:

for( auto it = doc->items().cbegin(), last = doc->items().cend(); it != last; ++it )
{
    switch( (*it)->type() )
    {
        case MD::ItemType::Anchor :
        {
            auto a = static_cast< MD::Anchor< MD::QStringTrait >* > ( it->get() );
            qDebug() << a->label();
        }
            break;

        default :
            break;
    }
}

You need to handle all possible types in this switch statement, and keep in mind that, for example, MD::List is an aggregate item, that contains other items, and you should loop through them and handle possible types there. As an example of handling MD::Document with all possible features, with except of raw HTML, you can have a look at my render of Markdown to PDF: https://github.com/igormironchik/md-pdf/blob/master/md-pdf/renderer.cpp

Your interest is only traversing through the MD::Document, start learning from this line: https://github.com/igormironchik/md-pdf/blob/b9129ab78c0f18cbcb4c30ea8c5ecca8edd1786c/md-pdf/renderer.cpp#L658 You only need to wrap text in HTML tags, combine QString with entire HTML document and ate this string to Qt's Web Engine to render.

During iterating through MD::Document you just combine QString with ready HTML code. Rendering to HTML is quite easy, just handle carefully all possible types during traversing through the MD::Document.

You don't need to save HTML file, as said, prepare in memory one QString, and give it to Qt to render HTML.

Good luck.

igormironchik commented 1 year ago

I can write a function that will receive MD::Document and return an HTML string. I thought about this function but decided to not provide it for some unknown reasons. But it's quite ease, I can do it.

igormironchik commented 1 year ago

I did two commits that enable converting MD::Document into HTML. It's not tested yet. But you can do following:

// md4qt include.
#define MD4QT_QT_SUPPORT
#include <md4qt/traits.hpp>
#include <md4qt/parser.hpp>
#include <md4qt/html.hpp>

using namespace MD;

int main()
{
    Parser< QStringTrait > parser;

    const auto doc = parser.parse( ... );

    // This will be an HTML representation of Markdown in QString.
    const auto content = toHtml( doc );

    return 0;
}
igormironchik commented 1 year ago

Well, I tried converter to HTML with some Markdown files, cleaned a little. It's not fully tested, to test it fully I need a lot of time. But you can play with it. I consider this issue as completed.

Light-Wizzard commented 1 year ago

Thanks, I will look into writing some tests for it.

I have thought about using the C version, as I have had to do this with other libraries I use for math, I did not find any examples using Qt, but it has an executable I could just call, but I like to do this all in Qt, and why I am interested in getting this to work.

I will write a small Markdown editor as an example, much like the one Qt has.

igormironchik commented 1 year ago

Minimal example of GUI application you want.

#include <QApplication>
#include <QWidget>
#include <QPlainTextEdit>
#include <QHBoxLayout>
#include <QWebEngineView>
#include <QTextStream>
#include <QTextDocument>

#define MD4QT_QT_SUPPORT
#include <md4qt/traits.hpp>
#include <md4qt/parser.hpp>
#include <md4qt/html.hpp>

int main( int argc, char ** argv )
{
    QApplication app( argc, argv );

    QWidget w;
    w.resize( 800, 600 );

    QHBoxLayout * l = new QHBoxLayout( &w );
    QPlainTextEdit e( &w );
    e.setSizePolicy( QSizePolicy::Expanding, QSizePolicy::Expanding );
    l->addWidget( &e );

    QWebEngineView h( &w );
    l->addWidget( &h );
    h.setHtml( "<html><head></head><body></body></html>" );
    h.setSizePolicy( QSizePolicy::Expanding, QSizePolicy::Expanding );

    QObject::connect( &e, &QPlainTextEdit::textChanged,
        [&]()
        {
            auto md = e.document()->toPlainText();
            QTextStream stream( &md );

            MD::Parser< MD::QStringTrait > parser;
            const auto doc = parser.parse( stream, "internal.md" );

            const auto html = MD::toHtml( doc );

            h.setHtml( html );
        } );

    w.show();

    return QApplication::exec();
}
igormironchik commented 1 year ago

markdowneditor.zip

This is Qt's example, that you tried, but with md4qt for Markdown parsing.

igormironchik commented 1 year ago

On my laptop I don't see significant difference, JS works fast, md4qt works fast. On eye you won't see anything, I guess. Maybe on some slow laptops the difference will be seen.

igormironchik commented 1 year ago

I did a small research, in the markdowneditor Qt's example. JavaScript version:

placeholder.innerHTML = marked(text);

runs ~2 milliseconds, the very first time is 25 milliseconds.

With md4qt and JavaScript:

placeholder.innerHTML = text;

JavaScript runs ~0.5 milliseconds, C++ conversion from Markdown to HTML is ~0.5 millisecond. Full time is ~ 1 millisecond, what is ~2 times fatser. But! 1 ms or 2 ms! Is it significant to you?

Light-Wizzard commented 1 year ago

Sorry for the delay, I have a partially collapsed right lung, and it hurts when it is cold, and we got over a foot of snow today.

You must be running Linux or Mac, I should have stated that I am using Windows 10 because it calls an external application called QtWebEngineProcess.exe to make that call to the web engine, so I was seeing a long delay due to that fact, I had to read up on this before I got back to you so I knew what I was talking about, but you are right, those numbers sound right, keep in mind that Qt compiles JavaScript into the executable so it talks back and forth much faster than using QtWebEngineProcess.exe as a bridge to talk to Qt Web Engine.

I need to run this in Linux and see what is going on here, this is almost funny, well it is because I am a Linux person stuck in a Windows world, and normally assumed everyone used Linux.

I am going to use your code because I need to alter tags and set CSS rules, and after looking at that JavaScript, it led me to your code, which I know is going to be so much easier for anyone to use that needs to change things based on tags.

You can run Windows in a VM, I am running from an SSD drive directly and really do not see the difference in speed, but to run Unreal I cannot do a VM, but you can check out this bizarre behavior, that must be a Windows thing, whereas Linux and Mac have a built-in web engine, being a web server, but Windows does not, and from what I read, even with Linux for Windows, or an IS server running, you still need this layer which is basically a process call to an external app, and I confirmed that by making the same type of call to cmark-gfm, and debugging why that took so long, so discovered the same thing affects any process due to timeouts, and startup and teardown, and every update to just one character will cause a flush, blank screen, then a few seconds of pause as it files the Web Engine Widget, so the delay is in a hidden process call, and I did find others talking about this issue on forums, so I now assume it is a known issue with Qt 6.5 at least, and that is beta, and what I am running, and this has a patch I am uploading now, did not read about, but will retest it to find out.

My goal is to make GitHub HTML to look like GitHub themed, currently the JavaScript with CSS looks flat, and maybe that CSS just needs to be adjusted to match the GitHub theme, I will figure that out.

I never thought about the fact that Qt WebEngine uses QtWebEngineProcess.exe in Windows till I ran a process in cmark-gfm, and thought that delay is not right, I was going to make a test for it, when I discovered the call in the stack, I am going blind so it was hard to follow at times, but it is clear that it creates a new instance instead of update the one, this is because it dumps the reference to the running process, and it goes out of scope, I was working on a way to fix that when it dawned me that this is the same issue I am experiencing in Qt Web Engine, so my guess is this is a bug in the process call, with cmark I want that instance to die, I only want the text, but with Web Engine, we need to keep that web server running, and clearly that is not working, it dies, and maybe by design, that I did not find out, becasue if you had more than one Web Engine Widget in one UI, you will have to have two in memory and that does not scale well.

I will have to look into this issue with QtWebEngineProcess.exe, but meanwhile, this code is what I needed, and it runs fast, well I think it will when I get back to Linux.

We use to make jokes about working with Windows when calling external apps for processing, looking at the debug log, I can see this call returns the rendered HTML code, much like calling cmark from a process, and that is the fact you have to Thunk about it, which is why it took so much time, I feel like an old programmer now having started in the late 60s, and now I am 62, and feeling old, and keeping up with Qt changes is crazy at times, testing out every beta release is hard work, I normally only use LTS, but needed a feature that was only released in 6.5, and should have told you all this from the start, I assumed everyone had this issue, but not everyone runs Windows, and now that they own GitHub, you would think Linux web servers would be on their way out and IS on its way in, but time will tell, but I did find all the benchmarks fascinating.

The issue about JavaScipt being slow turned out to be a false assumption, it turned out to due to a few factors in the way the Process runs now, in 5 it was different, they changed the way it works in 6, and Qt apps that use any process can feel this lag between when you call the app, and when it returns, I noticed a timeout call to a watchdog, so there are longs delays for a reason that would require me to install more debugging versions of Windows 10 Pro which I have a license for, I find Unreal Engine works better as does any Text to Speach (TTS) software.

I need to jump back into Linux to write the code for the cpack and in Linux I will need to use Qt Framework Installer, but that is tricky to get uninstall to work, and I have no idea what Mac uses, but I think NSIS only works on Windows, it will run on Windows 7, but Qt 6 apps will not, at least I could not get them to run, but I try to make the software I put out, easy to install, and an installer is the best way to do that.

Thanks for the code, I will see what I can do with your code, and hope Qt fixes this issue, but I have to ensure a bug report is filled first, beta is unstable at times, I did not remember this issue with earlier versions going back to when this first released, I have been using since it came out in other projects I have been working on, I would have to test them to find out.

igormironchik commented 1 year ago

Well, I booted to Windows and I don't see any lags with markdowneditor example with Qt 6.4.2. What feature do you need from Qt 6.5 beta?

Light-Wizzard commented 1 year ago

I need to go back to 6.4 to figure that out, something I read on the new features blog, and why I installed the beta, so I am going to remove beta while installing 6.4.2, this might be a beta bug.

Light-Wizzard commented 1 year ago

It works much better now, thanks for the help, I will stay with Stable, new features are not more important than it running.