V2: Non truncated blog article in feeds

MatanBobi commented 4 years ago

🚀 Feature

When working with the blog feed, in testing-library's, we would like a way to contain the full article and not just a truncated version of it within the feed since we don't really care about retention or something like that.

Motivation

Following this PR in testing-library, we think that having a way to send a non truncated version of an article can be helpful for people that just want to spread the word and don't care about traffic to their site rather than just spread updates.

Pitch

I believe we can add this by adding another config to the blog feed options, I'd be glad to help and create a PR for this one if you find it suitable.

Thanks for your hard work and help!

nickserv commented 4 years ago

Adding to the rationale here, I would expect most Docusaurus sites to be for community projects that don't need users to visit blog posts for monetization via advertisements or conversions. I think leaving blog posts non-truncated by default and adding an option for the old behavior (if encouraging site traffic or reducing feed size is necessary) would be even better.

nickserv commented 4 years ago

Also, this should be relevant to both V1 and V2. If approved, perhaps it should be backported from 2 to 1 or merged into 2 after 1.

slorber commented 4 years ago

Ok for an option to have non-truncated feeds.

I don't know much about RSS feeds and best practices regarding monetization/conversions, but if you can back up your claims by some authority link that explains why it's better to have non-truncated articles in feeds, we can make this the default.

slorber commented 4 years ago

Also, this should be relevant to both V1 and V2. If approved, perhaps it should be backported from 2 to 1 or merged into 2 after 1.

v1 is in low maintenance mode, it's not very important to backport this feature now, we'll encourage people to adopt v2 instead.

MatanBobi commented 4 years ago

I can take this one if it's ok :)

MatanBobi commented 3 years ago

Ok for an option to have non-truncated feeds.

I don't know much about RSS feeds and best practices regarding monetization/conversions, but if you can back up your claims by some authority link that explains why it's better to have non-truncated articles in feeds, we can make this the default.

Taken from the RSS 2.0 Spec:

A channel may contain any number of <item>s. An item may represent a "story" -- much like a story in a newspaper or magazine; if so its description is a synopsis of the story, and the link points to the full story. An item may also be complete in itself, if so, the description contains the text (entity-encoded HTML is allowed), and the link and title may be omitted. All elements of an item are optional, however at least one of title or description must be present.

I think we should keep the title and URL even if the description contains the whole item. Any thoughts? I've already started working on this btw.

nickserv commented 3 years ago

If I remember correctly the generated description didn't seem to include the title and URL

MatanBobi commented 3 years ago

@slorber I've implemented this and I did get the full post in the description but it was plain markdown and nothing rendered it.. I saw that in the classic-theme you use MDXProvider to render the MDX.. Any ideas what we can do here for the feeds? Thanks!

slorber commented 3 years ago

I see

The thing is, a user might be able to add mdx interactive components in the blog post. In such case, what should be the behavior of such feed? The feed reader won't be able to render the React component, but we can still render such component and get it as a static HTML string (it's just it won't be interactive anymore).

You should likely call ReactDOM.renderToString to generate the feed as HTML

MDXProvider only permits to replace default HTML elements with custom implementations, that does not seem 100% mandatory, but may be helpful to improve the default renderToString if not good enough. Unfortunately due to the modular architecture, the blog plugin does not know anything about the classic theme, and can't reuse this MDX provider. Maybe we need it for proper code block highlighting?

The question is: how do we transform markdown to a JSX tree on the server, so that we can feed renderToString with it?

The MDX apis I know compile markdown to React component source code, not a JSX directly. We generally write this source code to disk, and require it like regular source code. Wonder if we can avoid this step that seems a bit useless in such case.

slorber commented 3 years ago

Looks like mdx/runtime is what we are looking for: https://mdxjs.com/advanced/runtime

Something like this may work

import React from "react";
import ReactDOMServer from 'react-dom/server';
import MDX from "@mdx-js/runtime";

async function mdxToHtml(sourceFilePath) {
  const mdxString = await fs.readFile(sourceFilePath);
  return ReactDOMServer.renderToString(<MDX>{mdxString}</MDX>);
}

slorber commented 3 years ago

There's a pending PR here: https://github.com/facebook/docusaurus/pull/4330

Let me know if the implementation looks good to you, but to me it does not because it assume the blog post content is a regular string

MatanBobi commented 3 years ago

There's a pending PR here: #4330

Let me know if the implementation looks good to you, but to me it does not because it assume the blog post content is a regular string

Hi @slorber! Sorry, I didn't get a chance to solve this one and I have a WIP draft on my local machine.. Regarding what you wrote, I strongly agree. I'll try to add that functionality soon if this won't be solved in that PR already.

moonrailgun commented 3 years ago

In my pr https://github.com/facebook/docusaurus/pull/4330, I already change render from plain text to Full Markdown support, but without customize component.

and i dont think its a good idea for add config to choose whether show full content for blog. Its not a content website, you know.

For feed of docusaurus, I wanna know is any other suggest about that? I am glad with someone still use rss, and hope to improve user experience in docusaurus

MatanBobi commented 3 years ago

and i dont think its a good idea for add config to choose whether show full content for blog. Its not a content website, you know.

@moonrailgun Actually, I do think it should be a config, one of the purposes of RSS feeds is to attract people to your site and gain visitors, not all use cases should show all content.

If you're working on that, I'll delete my draft, but for further work you do on OSS, it's generally a good idea to ask in the issue if someone is still working on a feature :)

moonrailgun commented 3 years ago

and i dont think its a good idea for add config to choose whether show full content for blog. Its not a content website, you know.

@moonrailgun Actually, I do think it should be a config, one of the purposes of RSS feeds is to attract people to your site and gain visitors, not all use cases should show all content.

If you're working on that, I'll delete my draft, but for further work you do on OSS, it's generally a good idea to ask in the issue if someone is still working on a feature :)

I am soooo sorry for that because i have not found your issue before.

and for config, maybe your can add a config above my work? Also i dont think its right, but more customize way is not a bad work, right?

stnor commented 3 years ago

What's remaining for this feature to be fixed/merged @MatanBobi?

MatanBobi commented 3 years ago

This actually isn't my PR anymore as @moonrailgun did the work. If the current implementation isn't enough, I'll try to work on one next week :)

moonrailgun commented 3 years ago

sorry, but my pr #4330 is still pending and no reply. I have no idea how to continue.

@stnor @MatanBobi

slorber commented 3 years ago

I'm sorry, I didn't have time to review this PR because I have more important features to work on and this feature is not a top priority in my todo list, and I think the current PR is not totally ready (I find it strange to need to use remove-imports).

Unfortunately reviewing this PR is time-consuming because I have to conduct some additional tests locally to be sure it's working fine, and it's not time I'm willing to invest atm, but will come back to this when possible.

moonrailgun commented 3 years ago

I'm sorry, I didn't have time to review this PR because I have more important features to work on and this feature is not a top priority in my todo list, and I think the current PR is not totally ready (I find it strange to need to use remove-imports).

Unfortunately reviewing this PR is time-consuming because I have to conduct some additional tests locally to be sure it's working fine, and it's not time I'm willing to invest atm, but will come back to this when possible.

Well, thanks for your reply. Feel free to take time for other things, i will waiting for your review. If have any suggestion for my pr, I am grad to change it or modify it

facebook / docusaurus