search

laughedelic / literator

:memo: Generate literate-style markdown docs from your sources
GNU Affero General Public License v3.0
58 stars 6 forks source link

Migrate to FastParse #23

Open laughedelic opened 6 years ago

laughedelic commented 6 years ago

Although scala-parser-combinators are available for Scala 2.12, I want to try to migrate to FastParse (which btw, just released 1.0):

I'm not sure whether it's worth specializing the parsers to Scala (as FastParse provides a ready to use ScalaParse combinators). As a first try I want just to migrate existing parsers to FastParse.

mfirry commented 5 years ago

I think one would first need to check if fastparse offers something like RegexParsers, right?

laughedelic commented 5 years ago

Hi @mfirry! I'm sorry if you were interested in this during the Hacktoberfest and my reply is late. Anyways, I think regex parsers are not needed for literator, they are used here in a couple of places and could be replaced with some FastParse primitives. The rest is just combining simple parsers to combine a more complex ones, it should be quite straightforward to translate it to FastParse syntax.

mfirry commented 5 years ago

Thanks @laughedelic. I've been looking for a (toyish-)project to learn how to use parsers and parsers combinators for a while, and yeah Hacktoberfest was just an excuse but it's fine.

I'd be interesting in helping out with moving over to FastParse if you can guide me in the right direction.

When I looked at the code I found case class LiteratorParsers(val lang: Language) extends RegexParsers right away so that's why I figured that was needed.

Let me know how and if I can help.

laughedelic commented 5 years ago

Nice! Sure, I'll be glad to guide you if you want to try your hand at this. And I'll be more responsive in the future (you caught me in the middle of a move to another country 😅).

About RegexParsers: I see now where you got that idea. I pretty sure it the old parser combinators library-specific way to define parsers with some given context. So in FastParse it will be different and there's no problem with it.

SemanticBeeng commented 5 years ago

Now we have PythonParse : what would it take to make literator work same way for both Scala and Python? http://www.lihaoyi.com/post/Fastparse2EvenFasterScalaParserCombinators.html

@laughedelic what is your assessment on the value of doing this given your work on bio4j? Always been looking to link code to concepts in wiki, across languages. See 4-5 tweets around here https://twitter.com/semanticbeeng/status/1079445197484904450. If interested in this sort of thing I can document as a separate feature request.

Fastparse 2: Even Faster Scala Parser Combinators
Twitter
Nick Vintila on Twitter
“@olafurpg Wes, thanks for interest. My goal is to maintain rich knowledge in a wiki style but very close to code (Intellij ideal). See https://t.co/klVxm89iS6 for context. Like also what Specs2 (see https://t.co/aMdl2ktCED) does but mdoc is better because of scalameta foundation.”
SemanticBeeng commented 5 years ago

@laughedelic scala.meta already uses fastparse : https://github.com/scalameta/scalameta/issues/556#issuecomment-263530966 + https://github.com/scalameta/scalameta/issues/1800

Would you say it makes sense to use scala.meta in literator ?