markedjs / marked

A markdown parser and compiler. Built for speed.
https://marked.js.org
Other
32.33k stars 3.37k forks source link

Hi, I'm Josh, I'll be your co-pilot for at least a bit #956

Closed joshbruce closed 6 years ago

joshbruce commented 6 years ago

Hello,

If you've read #951 or #938 or chatter elsewhere, then you know I (and others) have been trying to revive the active status of this project. We're still working out the finer details, but for at least the next 30 days, @chjj and I have agreed to give me the ability to perform merges and publish to NPM on the main project.

Having said that, you all know the codebase, the history, and have more direct knowledge of what's going on than I do. So, I'm asking for your help in this endeavor.

This goes for anyone using Marked, not just the collaborators.

Current vision:

Marked is a high-speed, lightweight, and low-level JavaScript library for converting markdown to HTML.

(Correct me, if this is off-base in some way.)

Note: This is not the same as a full-featured HTML generator. Instead, it is an HTML generator with a specific and limited scope (the markdown specification).

Current actively supported Markdown specifications

Current reviving approach

  1. If there are no tests, it won't be merged.
  2. Fixing defects will take precedence over new features.
  3. I don't know enough about the codebase to make truly competent decisions about certain things; therefore, I'm leaning on the community to determine whether marked should even still be a thing.

Roadmap

We believe all XSS vulnerabilities have been addressed and released. Therefore, we have established 3 milestones based on tickets identified. They are as follows:

  1. No known issues (https://github.com/chjj/marked/milestone/1): This milestone is about focusing on issues flagged as broken, annoying, and refactoring & re-engineering. The goal is to address all of these issues - most likely resulting in a 0.4.0 release.
  2. Architecture and extensibility (https://github.com/chjj/marked/milestone/2): This milestone focuses on some community concerns and comments related to how the project itself is setup. Once these issues and proposals have been considered there will most likely be a 0.5.0 release.
  3. Improve developer experience (https://github.com/chjj/marked/milestone/3): Issues related to this milestone are about improving the consumability and contributability of Marked and will most likely result in a 0.6.0 release.

Help us out by linking duplicate issues and PRs...going through one issue and PR at a time is pretty time consuming. :)

joshbruce commented 6 years ago

Added option to preserve list numbering from input

rgbkrk commented 6 years ago

Thanks for taking up the mantle and being so thorough @joshbruce!

joshbruce commented 6 years ago

Hello,

The position from which this is written:

If I create it, or acquire it, I help maintain it.

I also don't know if the following was the initial intent or desire of @chjj.

The advent of package management and popularity of open source has resulted in a proliferation of packages available to consumers. Further, they have left in their wake many packages that are no longer maintained or participated in by the original authors (Marked), contain security vulnerabilities that go left unfixed (used to be Marked), or have been replaced by the native platforms for which they were written (https://www.npmjs.com/package/left-pad).


It's been more than the 30 days noted in the first message of this issue.

I was given permissions to publish the NPM package after invoking the NPM dispute process. The first 30 days was to see what the community wanted to do with Marked - up to and including letting it fade into the sunset. We did a lot in those first 30, which has now turned into over 60.

Having said that, having the repo here with no sign of it being transferred or being granted administration privileges from @chjj has proven problematic. I have reached out to NPM and @chjj again via email to discuss. Specifically with the request to transfer the repo to either my personal GitHub account or the 8fold organizational account; otherwise, we might change the repo referenced by the package to 8fold/marked. I have also forwarded the same message to GitHub itself through their contact form (I do not believe they have a dispute process as such).

I've never done (or been asked to do) something like this before; so, I don't know what "normal" looks like - other than stale projects strewn across the Internet. With the first dispute, NPM granted 4 weeks (roughly 30 days) for those in the dispute to come to a resolution - before the Marked package name would have been transferred to the fork I had initially created.

I have been assured that changing the repo pointer will have no negative affect on the package published to NPM.

I wanted to post here to let the community weigh in on this and remain as transparent and professional as possible. The following promises were made to @chjj and the community (I'll be adding a couple more that have come to mind).

  1. The project will remain under the MIT (or other open source) License.
  2. @chjj will remain the "initial developer" (as seen in the Common Development and Distribution License 1.0 license).
  3. Copyright would most likely shift to 8fold Productivity, LLC - which is a collective of independent professionals; therefore, would not be "owned" by a single individual.
  4. We will add CI tools per the requests by the contributors made elsewhere to ensure quality and reduce possibility of regression.
  5. @UziTech, @Feder1co5oave, and @KostyaTretyak would be added as contributors on the package.json file for their contributions over the last 60 days (not sure about the contributors of the past).

I would like to leave this public posting up for at least 30 days to let us discuss before taking any action to shift or stay. Again, I'm not sure if @chjj had intended for us to make this move or not; only that it is possible, and so far I have only heard from NPM on the subject of the latest NPM dispute - started February, 11 2018.

Thoughts? Questions, comments, concerns?

UziTech commented 6 years ago

My only issues with using the 8fold fork for the NPM module without @chjj transferring this repo are:

  1. We would loose all issues and pull requests.
  2. NPM is not the only package manager marked is in. The ones I know of are:
styfle commented 6 years ago

@UziTech

  1. Agreed, issues and PRs would be left behind. Maybe @github support could help transfer? It is not uncommon for a GitHub repo to change from personal to organization so they must have some process for that.
  2. jsDelivr and unpkg both read from npm, so this is not an issue if you have npm publish rights. However both Bower and cdnjs are manually published. But Bower recommends Yarn so it doesn't seem to really be a contender anymore.
Martii commented 6 years ago

@joshbruce

  1. The project will remain under the MIT (or other open source) License.
  2. @chjj will remain the "initial developer" (as seen in the Common Development and Distribution License 1.0 license).
  3. Copyright would most likely shift to 8fold Productivity, LLC - which is a collective of independent professionals; therefore, would not be "owned" by a single individual.

With item 1 and 2 I am a little confused why CDDL was mentioned here. If, and that's a strong if, the "fork" move changes to anything CDDL there's an issue with GPL found at https://www.gnu.org/licenses/license-list.en.html#CDDL ... which means that we most likely can't use it our organization.

With item 3 initial Copyright usually has a lengthy term (See https://www.copyright.gov/help/faq/faq-duration.html#duration). Not being an attorney (standard disclaimer) I would assume that the initial Copyright would remain in effect even if it is managed and maintained elsewhere. Some of my other contributor collaborations that I maintain along with the original author even states that I'm just a collaborator/contributor... keeping the original Copyright however has a known basis to myself through prior contract litigation.

We appreciate your fixes as they are important. Thank you very much. :smile:

joshbruce commented 6 years ago

Just to be clear.

This would not be a fork, this would be its own repository - completely cut off from this one...a separate product, so to speak, inspired by the codebase in this repository, which is inspired by the original Daring Fireball Perl (Markdown processing) in its execution (regex). The name "marked" is essentially being transferred just like when FedEx bought Kinkos (just dated myself a bit).

@UziTech:

Good points. We have 30 days or more.

Pretending this will happen (as it is the will of the actual maintainers and there is no word from @chjj requesting otherwise), what should I be doing to help facilitate as an owner listed on the package? (There seems to be a difference of thought between yourself and @styfle re the package manager question.)

I concur that we would lose the PRs and Issues - as well as the commit history - again, not a fork (or really a copy of the .git) - beginnings of a different product and codebase, just with the same name (trademark, so to speak, compared to IP related to the actual work itself); these concerns were also raised in the letter to @chjj with the request to transfer the repo. I think we do have the ability to reference the consolidated issues across the projects; so, not a lot to move - they all fit on the main page; we also have 30 days to close as many as we can.

We are, unfortunately caught between a rock and a hard place on this regarding what we want to do, what we can do under these constraints, and what's "fair" to those of us working to maintain this package...anyway, on a lot of levels.

@styfle:

I have contacted GitHub support with the full list of intentions and request. We have not received a response as of today. Will keep everyone posted.

If you and @UziTech can help us out on all things package managers and what needs doing, that would be wonderful.

Unless we would rather stay here.

Having said that, I'm not sure I could remain here - been wrestling with ethical dilemmas since arriving based on a lot of factors. Honestly, didn't think it would begin reviving and the mission would be to help people transition off of a dead package, including myself by extension (contacting 3,000 package maintainers to use something else). But, I will not leave you stranded, I hope.

@Martii:

Fair points. And, we should definitely take it under advisement.

  1. I only mention the CDDL as a point of giving credit where credit is due as in "This newness was inspired by a library created by @chjj, which was inspired by the work of ...." (Markdown has a license as well, which stipulates we can't use that term without written permission - not sure we've got that covered, and it's not 70 years out, but we might want to consider that moving forward: https://daringfireball.net/projects/markdown/license).

  2. The name "marked", I think, would fall under something akin to an unregistered trade or sales mark...not really related to IP of a person typically. (That's why, I believe, NPM has the name dispute process in place. If I had a package called "PhotoShop" - Adobe could show up and say something about it - think this instance is odd because it's more us saying, "This repository doesn't have the latest and greatest version of what people know as 'marked' in it.")

  3. The code itself is under the MIT License, which stipulates anyone can do anything so long as the copyright notice is left in tact. Having said that, I too am not an attorney, but the changes we intend to make would see the library changed so dramatically I'm not sure anyone would recognize the two as having similar origins. Further, using regex would have me betting that there aren't too many unique ways to solve the problems using that methodology based on the specifications; so, I'm not sure what specific areas of the codebase are considered IP or not.

  4. Unlike the Daring Fireball license, the MIT license does not have limitations in scope (for DF it's the Perl source, it would seem). So, again, I'm not sure what is actually covered by the copyright and how much of that original work has to be altered in order to be classified as a new work under fair use and common general knowledge of a skilled expert (see also music sampling and https://www.uspto.gov/web/offices/pac/mpep/s2141.html - understanding that patents and copyrights are not the same - and, again, stating that I don't think there's a lot of ways to skin the Markdown cat using regex).

  5. Given the open source nature, longevity of the project, and all the changes that have occurred since the original authorship, the number of hands who have touched it, how long it's been since @chjj actively participated in the creation or maintenance, and so on, I'm actually not sure how valid the single owner copyright is anymore based on the following; see collective works: https://www.copyright.gov/title17/92chap2.html

Having said all that, I'm really hoping attorneys don't end up getting involved here - seems a bit of a stretch and counter to at least some interpretations of the free and open source software movements. However, my knowledge in copyright comes from art and writing, which is a bit different, I believe, than software - it's pretty easy to tell if I wrote a paper or plagiarized it - same with a painting.

Martii commented 6 years ago

@joshbruce

I just wanted to mention the Chain of Title as being legal precedence e.g. it has to include the original Copyright and with OSL the license as well.

And to quote some OSL legal information:

The chain of title becomes important in open source licensing when someone wants to create a collective or derivative work of a previous work that itself consists of contributions by many people. The new authors are subject to the licenses of previous authors who preceded them, and each of those contributions may have different license restrictions on its use. * Lawrence Rosen

I'm really hoping attorneys don't end up getting involved here - seems a bit of a stretch and counter to at least some interpretations of the free and open source software movements.

That's why its best to follow 17 U.S.C. § 103[b] to avoid this. The subtle inference you mentioned is based on written fact (precedence) and actual litigation not an interpretation.

https://daringfireball.net/projects/markdown/license

Is there actual Code anywhere on that site? I've been viewing that page for more than a decade without finding it but I haven't spidered it. If there is no source code then it's probably a Document license only instead of a Code license. The idea/concept of markdown itself is free to develop however known Code derivatives, of any documented sort, follow what has been previously quoted and endured.

joshbruce commented 6 years ago

@Martii: Agreed. And it should maintain the Chain of Title, I think (thanks for helping work through clarifying).

Consider the US Web Design System from the US Gov't, which is licensed under the public domain and uses multiple open source projects: https://github.com/uswds/uswds/blob/develop/LICENSE.md

The license that would be in the new repo should, in my opinion, include the original from Gruber, the one from Marked, and our own. @chjj still retains full copyright over the work housed in this repo - unless transfer agreement is reached. Gruber still retains his. And we get ours, which includes the names of those who have helped keep it alive and moving forward.

Again, I do wonder how far this has to go all things considered. How much has to change before it is considered something else?

We are planning some pretty hefty changes. And, if the code in this repo (the work) begins taking "inspiration" from the 8fold/marked, does that mean this one has to reference ours? (Again, with artworks it's a lot easier.)

While I, personally, don't have an attorney in my back pocket, I will do what I can to ensure what is right, fair, and legal is performed and to the betterment of all involved - to the best of my knowledge at the time and leveraging the counsel of others.

Good question on the Markdown code: http://cpansearch.perl.org/src/SEKIMURA/Text-Markdown-Discount-0.11/xt/MarkdownXS.pl - Markdown is both the syntax and the tool for its conversion.

joshbruce commented 6 years ago

Also, just to make sure, there's a difference between the name and the code. Anyone can make a "Frappuccino" - they just can't call it that: https://en.wikipedia.org/wiki/Frappuccino - we're mainly working for the name; the repo, history, and so on may just not be able to come with us.

joshbruce commented 6 years ago

Oh! And @styfle - yes, in case you weren't aware (I wasn't when I first started using GitHub), transferring projects is a breeze; however, it is not GitHub that does it. Behind the "Settings" tab of any repo is the ability to "transfer ownership". The new owner gets an invite to confirm their acceptance. Done.

However, our roadblock there is that @chjj doesn't really talk to the community anymore. In the first dispute via NPM he mentioned he had tried multiple times to pass maintenance to others, but they always lost interest. Which is fair to a point. I'm not losing interest - my issues are more ethics- and principles-based. So, anyway, getting him to even say "no, I will not transfer the repo" or "yes, I will hit the button now" has not happened. He's got a lot of repos and a lot going on near as I can tell from the Issues we've triaged and whatnot; so, requests regarding Marked probably aren't high on the radar for him, which is fine...it's high on ours and we have a way to move forward under a new umbrella.

Honestly, I think he's kinda over marked - and so are the other contributors this Issue is assigned to. So, it's up to us to decide what happens next - individually and collectively.

Martii commented 6 years ago

@joshbruce

Again, with artworks it's a lot easier.

I would disagree... Content licensing is another animal altogether. (notice that some of those aren't OSI approved)

begins taking "inspiration" from the 8fold/marked

Inspiration would be from Daring Fireball's document... still have to check out the Perl code link you gave though. There may be times when ideas are developed simultaneously (take electricity and the light bulb for example) but in the Patent arena someone else got the credit in history... whereas your entity has the advantage of already examining the source and also collaborating.

How much has to change before it is considered something else?

Easy... look up "derivative" in a legal dictionary and apply it. Since the code would be derived from this one regardless of a refactor the licensing header would need to stay. MIT is a grey area which is why I usually don't use it... within certain aspects it can be "taken over" but you need legal counsel and appropriate written documentation by a qualified, accredited, attorney created for that (a forum post doesn't usually count for this... I certainly don't expect this project to hire an attorney).

... unless transfer agreement is reached.

This is one case where it could be changed if the license doesn't prohibit it like GPL (GPL is spelled out in detail and one can't add/remove restrictions or it invalidates the protective/permissive nature of it).

Usually with MIT (or BSD) you can add some restrictions (key word here and not contradictory e.g. if it's unenforceable no deal then) to existing licensing if needed (usually one doesn't unless there is an issue of protective rights against litigation for consequential or inconsequential damages e.g. misuse representing of a name including author name sometimes with malware).

Our concern here is to comply as much as possible with Copyleft as mentioned in its current form, evaluate any changes that might come with a transfer, if compatible continue using the module otherwise look into other modules (which I'm already doing since there have been some DDoS issues with this one... we have dynamic forum style content so we have to consider this... but we're also patient... everybody makes mistakes and sometimes there is the unexpected bug... regular expressions aren't the only way to do markdown syntax but it is native to the language in most node projects and usually faster)

difference between the name and code

This is usually where Trademark comes into play.

we're mainly fighting for the name, the repo, history, and so on may just not come with us.

I too would like to see the history saved in a refactor/transfer. I am all for access and distribution control rights with npmjs.com to be handled with care and active projects. As they are a provider they can be exempt from certain aspects in U.S.C (has precedence... pick an ISP for example) and that is why you can submit those requests to them. However Code licensing will always be an issue one way or the other. I would personally prefer to continue using marked as our code base sections are currently developed and extended with marked but we're also open to newer implementations of that idea of markdown.

With @chjj designating you as a Collaborator that is the equivalent of a maintainer management system. So I think this user has done this part to the best of abilities and the system that GH offers. If there is an issue with npmjs.com then @chjj should add you as an authorized person to push to their distribution system (don't know if you have that already... but I did do a dep update so probably :).

Martii commented 6 years ago

Btw before I forget there is also https://www.npmjs.com/docs/orgs/ to handle naming conflicts. We signed up for it months ago but it is idle at the moment.

chjj commented 6 years ago

I apologize to everyone here. I've been head down working on other projects for a while now and I'm just terrible at keeping track of things. I only just saw this through twitter. @joshbruce, thanks for taking over maintainership of this project. It's something I've neglected for a while. I will transfer marked to 8fold (or maybe a specific markedjs org that you are owner of?). Anyway, I'm glad this has finally found a good home. Is there anything else you need done on the npm side?

chjj commented 6 years ago

@joshbruce, I've transferred this repo to markedjs/marked. I've invited you as an owner. Thanks for all you work. I do appreciate it even though I've been so absent. Closing.

joshbruce commented 6 years ago

@chjj: Thank you so much for starting this project - it's one of the first - and I think it deserves to be recognized as such. Hadn't thought of a MarkedJS org, right on.

Should we make the same on NPM?

Not sure how this will play out with the CI bit - credit cards and whatnot. But, we shall see how it goes.

Thanks for the quick turnaround on setting it up!

joshbruce commented 6 years ago

@Martii: We've taken care of most of the, if not all of the XSS and REDOS things. If you find something, please let us know as Snyk.io and others are removing Marked from their watch lists.

https://github.com/markedjs/marked/issues/1039

chjj commented 6 years ago

@joshbruce, NPM is totally up to you. I don't think a marked org on NPM is necessary right now, but who knows.

Yeah, I'm not sure about billing either. I just checked and it looks like billing through orgs on github reveals the expiration and last 4 digits of your card to other members (wtf github?). Seems like it should only be visible to the user who entered it, or not visible at all. Not sure if the same is true of the paid CI support...?

joshbruce commented 6 years ago

@chjj: I know, right!? That's why we were aiming for 8fold. It's already a paid org account, which means 8fold would still ultimately be paying for it (me, specifically, because it would be unfair to the others), but at least there are the tax things.

Keeping it under here, from an entrepreneur-perspective could be weird. So, maybe the MarkedJS org on NPM makes sense from that perspective - package and teams there - code lives on 8fold for now unless or until we pass the torch again.

Gonna start a new issue to continue this conversation - as it has definitely changed. Give me about 10 minutes.

styfle commented 6 years ago

Linking #1051 for others to follow