Feature: scala-only parser via `fastparse` with cross platform

[Andrei-Pozolotin]

1. there are few incomplete implementations around:

   • https://github.com/olafurpg/metaconfig
   • https://github.com/akka-js/shocon
   • https://github.com/jokade/sconfig

2. it would help to have an official version here instead, based on fastparse

   • https://github.com/lihaoyi/fastparse
   • http://www.lihaoyi.com/fastparse/

3. there is also a need to publish it cross-scala-platform

   • scala
   • scala-js
   • scala-native

4. we suggest to contact relevant project owners and ask them to collaborate here

related:

• HOCON.g4
• hocon-parser.y
• BNF https://github.com/lightbend/config/issues/459

[havocp]

You're welcome to discuss how to get started here if you need a forum for it. This would need someone else to maintain it though - I think we lack bandwidth to ideally-maintain even the Java version here (since the library is done-ish, there aren't a lot of urgent tasks, but we aren't actively moving anything forward).

Handling all the corner cases in the same way as the original is quite difficult; much of the rationale of HOCON for better or worse is to handle a bunch of corner cases to optimize for the user instead of for implementation simplicity. An idea on the project is to port ALL the test cases from the Java version - possibly even working toward a TCK that multiple implementations could use. The weird cases of withFallback (merging configs) are downright mind-bending sometimes.

Happy to try to answer questions.

[olafurpg]

I would LOVE to have a standard pure-Scala implementation for HOCON. The main use-case is for cross-platform (jvm + js + native) tooling for Scalafmt/Scalafix.

I think a 1:1 rewrite of the Java sources to Scala would a be a better approach than clean rewrite of the parser with fastparse. Ideally, the API should be 100% source compatible with the Java API so that it's truly a drop-in replacement even for the more "advanced" corners of the API like unwrapped().

I personally don't have cycles to invest in this, however, but would be very grateful if someone stepped up to work on this 😄

[Andrei-Pozolotin]

@havocp thank you for "happy to try" :-)

1. the motivation is to find a real project home

   • and this looks like it
   • it seems there is a need for 2 sub projects:
   • scala-parser - scala-only parser via fastparse, first priority
   • scala-facade - scala-only replacement for java name space, second priority

2. project management

   • re: need someone else to maintain - say, some of the people mentioned above bring project to testable state, can you handle the releases after that?
   • concider that lightbend is married to config till death you part, so it seems there is no escaping scala-only support in near future anyway, especially as scala-js and scala-native become more mature

3. project quality

   • re: handling all the corner cases - not a first priority, current implementations are very poor even on the basic level
   • port ALL the test cases - not needed at first, scala-parser can re-use java tests

[Andrei-Pozolotin]

@olafurpg re: 1:1 rewrite of the Java sources to Scala would a be a better approach - tried, looks more problematic then fastparse for purposes of js and especially native

[ktoso]

Cool idea, adding some info:

• definitely rewriting config to be scala instead of java is NOT a good idea -- since this library is "everywhere" including "root projects" from which the entire ecosystem bootstraps every new scala release, having one less project to release to get the bootstrap running is a great benefit -- keeping config without scala dependency is useful.
• having an implementation that would be useful for js / native sounds interesting, many people like the HOCON format after all
• we don't have bandwidth to maintain such a thing
• I also would like to stress the quality aspect; "handling corner cases is not a priority" is scary; Since you have all the tests, making sure things are handled "properly" as covered by the tests should be much easier than going the first time around with it

Not sure if scala-hocon can live under the lightbend repository, it could imply we're working on it and maintaining it. Maybe it's fine though, but then who actively maintains it must be made clear, and also the quality must be a focus then -- otherwise it bounces back at us

[Andrei-Pozolotin]

@ktoso

re: "handling corner cases is not a priority" is scary I meant not first priority: current implementations do not even handle include and ${variable}, so after we support that, corner cases should be easy :-)

[ktoso]

Yeah; Just wanted to reaffirm that if we're building "the official hocon library for scala-js/native" it should cover and work exactly like the main one

[andreaTP]

I would love to have such a standard and do not waste effort in different projects.

I think a 1:1 rewrite of the Java sources to Scala would a be a better approach than clean rewrite of the parser with fastparse.

is not a good idea IMO, and the required effort is enormous if you look at the actual codebase.

What we can fairly ask to lightbend is a formalization of HOCON as a standard. Right now, basically the living implementation define the handling of corner cases and any re-implementation should go through a trial and error process to find the correct encoding.

Having a formal definition of the spec would help a lot and splitting the implementation into "standard API" and "JVM specific details" would help even further (i.e. cache management, files handling, etc. etc.)

From my side shocon was developed specifically to support akka.js use-case, generalizing it to cover more projects could be possibly long process.