Publish for Dotty

[travisbrown]

I've started working on this, primarily in order to unblock Dotty cross-building for http4s. I've got most of the combinators ported, including a simplified version of ~>, and will push a WIP branch soon (probably later this week).

I'm not aware of any other attempts to set up Dotty cross-building, and am opening this issue to track progress and help avoid duplicated work.

[sirthias]

Thank you, Travis! Without having looked at things more closely I could imagine that porting pb2 to Dotty entails quite a bit of work, which I wasn't sure is worth the effort, since it's hard for me to gauge the current user base.

So, my initial approach was to simply wait until someone opens the first "please port to Dotty" ticket.

What is http4s using parboiled2 for? HTTP header parsing? Would it make maybe sense to simply rewrite the parsing parts to a hand-rolled parser instead of pulling pb2 along?

The reason I'm a bit caution wrt putting in a lot more work into parboiled2 is that, from my current perspective, I'd regard the basic approach as only half successful. While parsing performance is generally good and error reporting not bad I would, today, go for a different, more flexible and more composable design for modeling parsing rules.

[travisbrown]

@sirthias http4s has dozens of Parboiled parsers (~190 rules) for different header types, IP addresses, etc., and currently uses a parboiled2 fork (not exposed in any public APIs) to avoid Shapeless conflicts for users. The fork generally stays pretty close to upstream (right now it's a month and a half behind).

We could rewrite these parsers, but my feeling right now is that it'd be less work to get the subset of parboiled2 that's needed to support them working on Dotty. I think I'm about an half-day of work away from that after getting started last weekend—it's close enough to complete that I can work through the non-Parboiled-related compilation errors on Dotty, which was my primary goal at the moment.

My plan has been to finish this minimal port in the http4s parboiled2 fork, outline what's missing, and see if there's interest in completing the migration upstream. I definitely understand if there's not, or if that's not a priority for anyone in the immediate future!

[sirthias]

Ok, thank you for this update! I'd say we keep this ticket open and use it a discussion channel for your progress. If things really go as smoothly as you expect than it sounds like cross-building for Dotty might actually be worth the effort. In any case it'd certainly be valuable to make the work in your fork available to us here in the upstream repo, maybe simply as a partial PR serving as a seed for the full port to Dotty, which could be completed at some later point in the future.

[rossabaker]

@travisbrown Did your parboiled branch ever get published? I'm beginning some core model cleanups in http4s, which includes the parsers, and am wondering whether we still think Parboiled2 on Dotty is plausible or if it's time to move on. I agree with @sirthias's comments on composability, but it's nice how the parboiled parsers resemble the RFCs, and the project has treated us well for many years.

[travisbrown]

@rossabaker I don't think I've pushed the branch yet, but I'll try to dust it off tomorrow.

[phdoerfler]

The reason I'm a bit caution wrt putting in a lot more work into parboiled2 is that, from my current perspective, I'd regard the basic approach as only half successful. While parsing performance is generally good and error reporting not bad I would, today, go for a different, more flexible and more composable design for modeling parsing rules.

@sirthias Don't let anyone stop you from making an even better parboiled3 using, e.g., scala.meta ;) Seriously though, I would love being able to use parboiled2 or something even better with Dotty / Scala 3. But I am not a significant user base, of course. I only use parboiled for hobby and for small open source projects. And obviously such a nice library does not write itself over night, so I don't expect this to be happening any time soon. One can dream though!

[rossabaker]

@travisbrown Do you still have that branch anywhere? I'm getting ready to sort out what still needs to be done for http4s so we can plan our Dotty release roadmap, and this is the scariest dragon we've thought of.

[SethTisue]

alternatives that are already available for Scala 3 include https://github.com/typelevel/cats-parse (the up-and-comer) and https://github.com/scala/scala-parser-combinators (the old standby; not industrial-strength, but works fine if you don't care about performance)

(fastparse has been a popular choice for Scala 2, but Scala 3 support hasn't landed, as per https://github.com/com-lihaoyi/fastparse/issues/246)

[yanns]

We are extensively using parboiled2, for internal applications, and for sangria

If there is any WIP branch, I'd be willing to spend some time continuing the work.

[yanns]

Travis was very kind and pushed his last changes here: https://github.com/travisbrown/parboiled2/commit/03483e391a239b4536cd83963244e1855f6a153e

I'm taking a look at those.

[yanns]

It seems that we must choose between scala native & scala 3 for now: https://github.com/scala-native/scala-native/issues/2317

[lolgab]

It seems that we must choose between scala native & scala 3 for now: scala-native/scala-native#2317

You don't have to choose, you can exclude Scala Native/Scala 3 from the matrix, while keeping Scala Native/ Scala 2

[yanns]

It seems that we must choose between scala native & scala 3 for now: scala-native/scala-native#2317

You don't have to choose, you can exclude Scala Native/Scala 3 from the matrix, while keeping Scala Native/ Scala 2

👍 I figured that myself:

lazy val commonNativeSettings = Seq(
  // Currently scala-native does not support Dotty
  crossScalaVersions := crossScalaVersions.value.filterNot(Scala3 == _)
)

[yanns]

The dependency on shapeless is an issue right now. As shapeless is using macros, I cannot use .cross(CrossVersion.for3Use2_13). https://github.com/milessabin/shapeless/issues/1043

I'm trying to remove this dependency temporary by copying hlist into parboiled2. It's not a long term solution.

[rossabaker]

http4s' fork of parboiled2 existed to remove the shapeless dependency. That might give you a head start.

On Mon, May 31, 2021, 11:24 AM Yann Simon @.***> wrote:

The dependency on shapeless is an issue right now. As shapeless is using macros, I cannot use .cross(CrossVersion.for3Use2_13). milessabin/shapeless#1043 https://github.com/milessabin/shapeless/issues/1043

I'm trying to remove this dependency temporary by copying hlist into parboiled2. It's not a long term solution.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sirthias/parboiled2/issues/210#issuecomment-851558159, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABC22WJTHOR6P2PXLIKHSDTQOS2BANCNFSM4LELKDFA .

[jrudolph]

http4s' fork of parboiled2 existed to remove the shapeless dependency. That might give you a head start.

Same for akka-http's fork which includes only the essential parts from shapeless for building parboiled2 (which is hlists).

[yanns]

If some people want to help, I've started a new branch trying to keep a clean history that can be used as base: https://github.com/sirthias/parboiled2/pull/274

It's only the basic structure for now, nothing is working. If you want to help, you can try hacking on the macros, or moving one test from scala-2 to scala and fix the compilation issues.

[yanns]

New base branch: https://github.com/sirthias/parboiled2/pull/280

[sirthias]

After the awesome work in https://github.com/sirthias/parboiled2/pull/280 this is finally done. Version 2.4.0 is cross-build against Scala 3, has been published to sonatype and should appear on Maven Central soon.