Support full Markdown spec: +GitLab +GitHub #642

Open engelgabriel opened 9 years ago

https://github.com/gitlabhq/gitlabhq/blob/master/doc/markdown/markdown.md#special-gitlab-references

@engelgabriel looks very similar to GitHub flavored markdown. Do you know how it differs?

I am not sure.. but we must improve our Markdown support in general.. add the tables, etc. There is probably a good package somewhere :)

@engelgabriel Thanks for great work on Rocket.Chat. Sorry if this a dumb question, I'm new to this... Should the existence of bounty appear automatically in GitHub issues? It does not look like it. This issue now has a bounty - https://www.bountysource.com/issues/26990858-support-full-markdown-spec-gitlab-github.

Not sure why the plugin failed. I've edited it manually.

Tables would be very useful, I'm trying to write a better looking Redmine ticket announcement at the moment and being able to show the details in a table would look much better (and not use as many lines).

Mattermost has an excellent implementation, and I'd like to see equivalent for Rocket.Chat - see http://www.mattermost.org/open-source-slack-alternative-adopts-markdown/

Note that Mattermost has adopted Markdown for Channel header text as well as message text, which allows nice concise in-heading links.

Rocket.Chat could have equivalent treatment for the Topic text for each room

I agree, let's use that as a reference.

If no one else is working on this, I'll give it a go

:+1: I don't believe anyone is. Feel free to take a crack at it! We're here if you need us. Also on the demo server as usual :smile:

Ok. thanks guys. I started yesterday, but will be away till Sunday. I don't suppose fixing this next week will be a problem?

Not a problem :)

Ok so I'm almost done with implementing it myself, then again, I started wondering whether or not it would be easier/cleaner to implement a library. chjj/marked seems to be the one to do the trick... Is there some policy to this since all previous implementation has been done using custom code?

@klatys the most popular implementation that I'm aware of is codemirror

Hi, I commited markdown support, though against mattermost's markdown (as mentioned above to be taken as a reference) support there are some issues/questions I'd like a second opinion to

they use four spaces as a code indent which in my opinion colides with indentation of third level list so I would stick to using ticks only (as standard GFM documentation does so)
inlined image is always and not really inline and with full viewport width - is this intentional?

as for github issues / pull request:

Since without pinging them i cannot know whether number is either issue or pull request i suggest differing them as gitlab does
also since rocket chat is not in a repository scope, there are two options to determine which repository to use
- add default repository to settings
- use reference containing repo - e.g. RocketChat/feature-requests#642
furthermore if noone is against I would add option to define whether to use github/bitbucket/?? repositories
Gitlab allows Inline HTML... is that really a good idea?

@RocketChat/core let's give some feedback here? This is important :)

is this intentional?

In our markdown parser we are just using the same class on the image as we are for file uploads. So this would just be a matter of changing that.

Regarding the linking of issues / commits. I kind of wonder if we shouldn't treat this as an integration? Because I can see people wanting github / gitlab / bitbucket and more in the future. So this needs to be extendable, as well as potentially different per channel.

Gitlab allows Inline HTML... is that really a good idea?

If we could safely purge any and all javascript from it, this might not be a terrible thing. But the key thing would be to be able to purge it. I think this is a hard problem we may not want to tackle or allow.

Regarding the linking of issues / commits. I kind of wonder if we shouldn't treat this as an integration?

This is what we do now, and it works pretty well honestly.

Any news ? the actual parsing is really weird :eyes:

i just stumbled on that issue because i tried converting jira-matter-bridge to work with rocket.chat and the drawback is the same without the markdown/table support it looks really ugly.

I'm going to work on this over the weekend. Plan is to implement codemirror insertion if code block is detected, and inject the code into the injected codemirror object.

Any news on this one?

I had some issues setting up the dev environment the weekend after I posted this, and totally forgot about it. I did get everything installed though, and nothing going on evening this week, so... yeah.

ok... n/m... once again:

curl: (7) Failed to connect to meteorinstall-4168.kxcdn.com port 443: Operation timed out

I hate meteor. hate, hate, hate.

Any news @LongLiveCHIEF ?

Nope.... still hating on meteor.

Ok.. i just wasted another 3 hours trying to get meteor to simply install on a machine. I've tried several Mac's (all post 2014 MBP's), and several Linux machines, including a few DO droplets just to make sure it's not something with my networking equipment.

They all get:

curl: (7) Failed to connect to meteorinstall-4168.kxcdn.com port 443: Connection timed out
Retrying download in 5 seconds...

// repeated 9 more times

I have discovered this is pretty common with meteor installations, and it's not too often that these issues are resolved to the satisfaction of the user.

then I tried to create a new dev environment for this project by creating a docker-compose extended override, but using both danieldent/meteor and meteorhacks/meteord. In both cases here, mongo wouldn't start due to language issues. I'm going to try one more thing... and if that doesn't work, I'll have spent more than an entire working day (collectively) just trying to install a prerequisite for this project.

I really like Rocket.Chat... but this is my free time we're talking about, and I value it enough not to waste it carelessly. If anyone has run into this issue while attempting to contribute to this project, I'd appreciate knowing what tactic you used to get around the transient failures of meteor's packaging cdn.

@LongLiveCHIEF ~~what OS are you downloading this on?~~ nvm see it. Never seen someone have this much trouble just getting meteor its self installed :(

Just performed a new install my self with no issues.

Are you behind firewall / proxy etc that would interfere with your outbound connections?

@geekgonecrazy nope. I have very fine control over my firewall, and I can see the traffic leaving just fine, and I am getting headers back from the CDN. It's even keeping the initial connection open as it tries to re-attempt the download all 10 times.

I linked to an issue on the meteor/meteor site, and there are so many respondents that github collapses half of them in the middle and just displays a "134 more responses not shown" instead of making you scroll.

Many users seem to be reporting errors of failures on CI/CD builds as well... which are typically on a different network than their development machines. I'm surprised you guys have never run into this.

I got it to install once... towards the end of July, but when I came back to this 2 weeks ago, and tried to update meteor, it bombed out both my installation and the download... and I haven't been able to install on any of my machines, either at home, or in the cloud, since.

@geekgonecrazy See: https://github.com/LongLiveCHIEF/Rocket.Chat/commit/5687762dd9971eea370087c4a0a4404e26b2b8e0

I think I've had some luck by creating a docker-based environment to do development on. Instead of installing the meteor binary, you can use a dockerized meteor distribution, and mount the Rocket.Chat app onto it.

I'm going to switch out the base of the build, and create a canonical docker-compose.yml and a docker-compose.prod.yml if you like this idea and would like to see it go further.

alright, so even my CI/CD servers can't download meteor, either from NY or San Francisco. I've now spent 4.5 hours on this, today alone. I'm sorry, but I'm not going to try this again. Every.

I've seen this issue being reported by users as long as 2 years ago. I'm not going to waste another moment trying.

Good luck Rocket.Chat devs/contributors! You are great, and I wish you the best.

@LongLiveCHIEF regarding us not having the issue. We have meteor cached in travis. So i'm sure this helps keep us from hitting this as often. I think we've seen a few small occurrences in the past. But usually an hour or so later it would be back.

@RocketChat/core as you can see a lot of things have been tried. Ideas?

Before I totally lost it, my next step was going to be to create a Docker image that builds meteor using the developer instructions, instead of the CDN install, and then modify my docker-compose.dev.yml to use that as meteor.

had similar problems with meteor

using "54.192.225.217 warehouse.meteor.com" as resolve in /etc/hosts fixed that for me damm CDN still not working after month of having these problems

thanks @TheBelgarion i'll give that a try tomorrow

Is there table support? I can't get it to work if there is

Any news on this one?

As a temporary workaround, you might consider using LaTeX/KaTeX to generate tables. Since Rocket does support LaTeX, this is another way to generate tables. Not as simple as with Markdown, of course...

@LongLiveCHIEF do you continue to try to work on that ?

nope. Too many issues just with getting meteor installed. After wasting (literally) days just trying to get the platform installed, I decided not to bother. I won't be contributing to any meteor-based project anytime soon... at least until the meteor team fixes their delivery issues that have been plaguing their community since 2014.

Are we getting anywhere with this? is there any reason not to edit markdown.coffee to handle wanted markdown features?

I've been doing some tests with markdown-it. So far it's going well, but I still have some issues with styling.

However, I have some questions: What are the (for now) necessary features? I reckon at least as follows:

bold
italics
strike
inline code
multiline code
quote

There have been requests for the following:

bulleted and numbered lists
block quotes
headers
tables

Should there be a complete compatibility with Github Flavored Markdown, or should we conform to CommonMark or a subset of both? For example, I'd like to have the checked lists fro GFM, but that's only a wish! ;-)

Also, an important question: how are we going to treat the fact that the syntax is evolving? For example, in a lot of markdown parser I saw, bold is **some bold text** instead of *some other bold text* in Rocket.Chat right now. Will there be a change to be made in database (i.e. edit all messages to conform to new syntax where applicable) or will it just be "oh well, it will just look different"?

I'm fighting with the CSS now, where some settings (for ul and ol tags, for code tag) are overridden, but otherwise, it's working really well.

Is there some documentation somewhere regarding the organisation of CSS stylesheets in Rocket.Chat?

Hey everyone,

I have a preliminary version working well, the CSS may need a good bit of cleanup. I'll put up a PR tomorrow I think.

Please, give your opinion regarding the questions in my previous comment:

Should there be a complete compatibility with Github Flavored Markdown, or should we conform to CommonMark or a subset of both?
How are we going to treat the fact that the syntax is evolving?

I created the PR related to this issue. Please have a look and comment!

As discussed: please add a label "feature-needs-review". And maybe in this case also have a look at whether the PR is what is needed and whether it is still possible to merge it easily.

/cc @SeanPackham

Cheers Thomas

Hi everyone, After a break related to work, I'm planning on starting to work again on this. Is there anyone already working on this? Anything I should take into account?

Nothing on my side but would really love to see this coming as a reality as the formatting with latex is awful when you want to try to have tables.

@gromain Are there any updates from your side :)?

I think that we should use showdown js library. https://github.com/showdownjs/showdown. It has support for almost every command and it is released under the MIT version. I am working to implement it.

I think we should have just one markdown implementation and have it well implemented in all clients.

RocketChat / feature-requests

Support full Markdown spec: +GitLab +GitHub #642