typelevel / cats-tagless

Library of utilities for tagless final encoded algebras
https://typelevel.org/cats-tagless/
Apache License 2.0
310 stars 41 forks source link

Scala 3 feature parity #170

Open larsrh opened 3 years ago

larsrh commented 3 years ago

Type classes from cats-tagless:

Type classes from cats:

Other:

joroKr21 commented 3 years ago

https://scalacenter.github.io/scala-3-migration-guide/docs/macros/migration-tutorial.html

smarter commented 3 years ago

This library uses macro annotations which don't exist in Dotty, but things like @autoFunctorK trait Foo[F[_]] should be rewritable as trait Foo[F[_]] derives FunctorK using https://dotty.epfl.ch/docs/reference/contextual/derivation.html. In fact it looks like shapeless-3 already has support for that! (there's an example in the readme)

joroKr21 commented 3 years ago

@smarter the macro annotations are not essential. They just generate an implicit instance in the companion object but the heavy lifting is done by blackbox macros. If we want to use shapeless 3 we would need to somehow generate mirrors for traits.

Fristi commented 3 years ago

First of all, great work with this library.

Just doing some evaluation for scala 3. We use it in some projects and we would like to know if we could use it when Scala 3 comes out? Or do you plan to port the most used macros first? FunctorK and Tupled2K are essential, aspect weavng is nice but can be added later.

All the macros need to be rewrite right? I am not an expert :) Contributed only by looking at the examples of the work of the maintainers. Let me know if I can help a hand.

joroKr21 commented 3 years ago

Just doing some evaluation for scala 3. We use it in some projects and we would like to know if we could use it when Scala 3 comes out? Or do you plan to port the most used macros first? FunctorK and Tupled2K are essential, aspect weavng is nice but can be added later.

It's on my todo list to work on this. I can't promise that it will be done by the time Scala 3 final out though. But - if we manage to port one macro, we can port them all. The implementation is very consistent. The only odd ones are Instrument and Aspect. Note that there are no macro annotations on Scala 3 so they will have to be replaced by support for derives FunctorK etc. clauses.

All the macros need to be rewrite right? I am not an expert :) Contributed only by looking at the examples of the work of the maintainers. Let me know if I can help a hand.

Embarrassingly, the most demotivating part of the work is setting up the build and CI for cross publishing πŸ™ˆ We also need to apply simulacrum-scalafix before we can start because we use @typeclass. So here are two self-contained tasks we need to do first:

  1. Apply simulacrum-scalafix
  2. Setup build and CI to cross-publish the core module (it doesn't contain any macros)
joroKr21 commented 3 years ago

We also need to resolve #192 - our build is really outdated πŸ˜•

joroKr21 commented 3 years ago

We are blocked by lampepfl/dotty#11685 - looks like we will have to wait until Scala 3.1 😞 I didn't think such a feature could be missing from the new macros.

smarter commented 3 years ago

Out of curiosity, what part of cats-tagless requires generating arbitrary class implementations? I would have though that at least some part of cats-tagless could already be implemented today given that shapeless-3 supports derives FunctorK for example which I assume is equivalent to @autoFunctorK

joroKr21 commented 3 years ago

@smarter on a high level think Mirror for traits. Given trait UserRepo[F[_]] we want to have FunctorK[UserRepo]. shapeless-3 supports case class UserRepo[F[_]] or enum but that is not quite the same.

smarter commented 3 years ago

Would https://github.com/lampepfl/dotty/pull/11686 be relevant?

joroKr21 commented 3 years ago

No, what we are talking about here is the equivalence between a trait and the Mirror.Product of all it's methods as functions. BTW I'm not sure, does every method type in Scala 3 have a corresponding function type? Can you mix polymorphic functions and dependent functions and contextual functions arbitrarily? But even then type members of the trait would be a problem.

smarter commented 3 years ago

You can mix them yes, but you can't represent method with default parameters or methods with varargs losslessly as functions.

joroKr21 commented 3 years ago

You can mix them yes, but you can't represent method with default parameters or methods with varargs losslessly as functions.

That's not a problem. We are just forwarding to an underlying instance and modifying the arguments and/or the result. Varargs just have to be passed correctly but otherwise can be viewed as a Seq. Default parameters are a property of the interface, not the implementation so they don't matter.

So in theory, if we had Mirror.Product[trait, tuple of functions] we could use shapeless-3

joroKr21 commented 3 years ago

But then you have edge cases like what about abstract vs concrete methods? You shouldn't really have the AnyRef methods in that pile.

ScalaWilliam commented 2 years ago

@joroKr21 @smarter https://github.com/lampepfl/dotty/issues/11685 is now merged into dotty (it was noted as a blocker on an earlier message of joro's). Is it now possible to support this library on Scala 3?

smarter commented 2 years ago

Well, someone has to go and try to port the macros to find that out :). That would be a good exercise since this feature is still experimental in the compiler (only available in nightlies) and user feedback would help us decide if it's good enough to stabilize.

ScalaWilliam commented 2 years ago

How about we first make it work without any cross-building and release it as cats-tagless3 - since configuring a build is super complicated, this could be decoupled?

I had a go, the following compiles core (except some places I had to comment). I am not particularly knowledgeable on HKTs, so this might be completely wrong.

It fails to compile when running testsJVM / testQuick - in particular, in Derive.scala under the macros project, with an error like:

[error] java.lang.StackOverflowError
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs(ImportSuggestions.scala:277)
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs$(ImportSuggestions.scala:23)
[error] dotty.tools.dotc.typer.Typer.distinctRefs(Typer.scala:107)
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs(ImportSuggestions.scala:278)
...
smarter commented 2 years ago

That stackoverflow is fixed in 3.1.1 but your build uses 3.1.0 for some reason.

ScalaWilliam commented 2 years ago

@smarter yep that works. compilation is Ok in core but not in macrosJVM and lawsJVM. 138 errors

joroKr21 commented 2 years ago

Yeah we need to reimplement the macros in Scala 3 - a good place to start is with FunctorK

pomadchin commented 1 year ago

Hey πŸ‘‹ , we've (with @dos65) recently tired to play around with https://github.com/lampepfl/dotty/issues/11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works πŸŽ‰ πŸŽ‰ πŸŽ‰ ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo https://github.com/pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

Swoorup commented 1 year ago

Hey πŸ‘‹ , we've (with @dos65) recently tired to play around with lampepfl/dotty#11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works πŸŽ‰ πŸŽ‰ πŸŽ‰ ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

Great news

jchapuis commented 1 year ago

Hey πŸ‘‹ , we've (with @dos65) recently tired to play around with lampepfl/dotty#11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works πŸŽ‰ πŸŽ‰ πŸŽ‰ ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo https://github.com/pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

Great! are you still working on this front?

pomadchin commented 1 year ago

Hey @jchapuis yea, just a bit slow due to the lack of time 😞

I finished the POC a ~ day ago in a separate repo (https://github.com/pomadchin/tagless-derivation); hopefully will have time soon to start working on a PR against this repo.

ScalaWilliam commented 1 year ago

https://github.com/lampepfl/dotty/issues/11685 seems closed but not sure if all the pieces are in place for this yet.

ScalaWilliam commented 1 year ago

@smarter @joroKr21 is there anything else missing for this? I understand @nicolasstucki has a PR https://github.com/lampepfl/dotty/pull/15024 but I am not sure if this will be sufficient to fix it.

pomadchin commented 1 year ago

@ScalaWilliam the current scala3 macro capabilities are sufficient enough to work on the cats-tagless scala 3 support; check out the fully functional POC here https://github.com/pomadchin/tagless-derivation

I still didn't have a chance to fully commit into porting this POC into this repository.

pomadchin commented 1 year ago

I've been mostly interested in the ApplyK derivation for the simple aglebras composition; I have macro porotytpes for the FunctorK, SemigroupalK, ApplyK, InvariantK, ContravariantK only.

Don't hesitate to create PRs against https://github.com/pomadchin/tagless-derivation with macro implementations for other concepts!

I'll try to create the initial PR against this repository so it can be picked up if needed.

ScalaWilliam commented 1 year ago

That is fantastic @pomadchin, I will not have the time to spare to contribute, but am hugely interested in using this. Also had set myself as as a watcher for the repo.

joroKr21 commented 1 year ago

Yes I think we have all the tools necessary, it's just a matter of making the time work on this. The beginning is most difficult because we have to setup all the infrastructure. But we can do this one type class at a time, starting with FunctorK and ApplyK as the most useful ones.

bpholt commented 1 year ago

There's been a lot of work on this, which is really great! I've been wondering, as an interim step, could the core module be published for Scala 3 while work on the macros continues? It obviously would be tedious to use, since you'd have to write out algebra instances manually, but it would unlock downstream Scala 3 support for things like libraries that use the typeclasses and assume their users make the instances available. (Our async-utils and natchez-tagless libraries are examples.)

pomadchin commented 1 year ago

@bpholt i can split my PR in two if that’s really needed; also it can be helpful in case we don’t want to merge poor persons macro implementations as is (macros work but don’t support all the cases) cc @joroKr21

joroKr21 commented 1 year ago

Let me take a look first. It might take a while though. I'm really busy these days because I changed jobs recently and I'm also in the process of changing apartments.

joroKr21 commented 2 months ago

Now that we have decent infrastructure, missing features are up for grabs

pomadchin commented 2 months ago

@joroKr21 I'll try to make some PRs ~ to help keeping momentum going

joroKr21 commented 2 months ago

@pomadchin that would be amazing! πŸ‘

pomadchin commented 2 months ago

Ha, bifunctor and profunctor could be tricky πŸ€”

joroKr21 commented 2 months ago

Ha, bifunctor and profunctor could be tricky πŸ€”

That is true, we might need new helpers for those. The same applies for stuff in the "other" category.