Total transformation for Cats' `NonEmptyX[A] -> NonEmptyX[B]`

[arturaz]

Is there a reason why these aren't in cats module?

given nonEmptyVectorTransformer[From, To](using
  t: Transformer[From, To]
): Transformer[NonEmptyVector[From], NonEmptyVector[To]] =
  _.map(t.transform)

given nonEmptyListTransformer[From, To](using
  t: Transformer[From, To]
): Transformer[NonEmptyList[From], NonEmptyList[To]] =
  _.map(t.transform)

given nonEmptySetTransformer[A, B: Order](using t: Transformer[A, B]): Transformer[NonEmptySet[A], NonEmptySet[B]] =
  _.map(t.transform)

[MateuszKubuszok]

Yes, long story short: it's unmaintainable.

At first it's one given mapping each F[A] into F[B]. So we'll have one for: NonEmptyChain, NonEmptyVector, NonEmptySet, because they require only implicit Transformer (or, maybe it should be implicit Transformer.AutoDerived?). Or, maybe someone would do it with a Functor + Transformer

Next request will be for NonEmptyChain into NonEmptySet. Suddenly we should rather use Applicative and NonEmptyTraverse to convert the collections. Except, this breaks binary compatibility because we cannot just remove the implicit from the scope and keep the linker happy.

Then, someone might ask why they cannot convert List into NonEmptyList (as Partial) or NonEmptyList into List (as Total transformer). Build-in collections used Factories and Iterable interface, Cats would be on Traversable and Applicative, using std with Cats would be... adding new Cats instances for std types? Pulling Cats into core module which struggles to be dependency-free?

Then we add Java Collections, because you might want to convert from java.util.List into scala.collection.List... but what if one attempt sto convert java.util.List into cats.data.NonEmptyList?

Finally: DSL allows overriding items. You can write:

case class Foo(a: Int)
case class Bar(a: Int, b: String)

val list: List[Foo] = ...

list.intoPartial[NonEmptyList[Bar]].withFieldConst(_.everyItem.b, "value").transform

For this DSL to work, conversion has to be handled in the macro, the macro has to be aware of how it can iterate over/build a collection, this mechanism has to be extensible so that it would work (for now) with: Standard Library, Cats, Java Collections, and without pulling all dependencies for everyone.

In practice, we developed our own type classes which allow us to iterate over and build data types. And non-empty collections has to be build with a builder which returns partial.Result[NonEmptyX[A]]. It currently has the limitation that it won't allow to call a map over the collection - we iterate over the transformed collection, and call a builder to build a new one, which has the downside that requires all NonEmpty to be constructed as partial.Result even if there is .map available.

For now I am not planning to work on it, but I would not add given NonEmptyX(using Transformer) as they would not cooperate with overrides DSL, and would force us to keep in the source code a temporary solution even when it would become obsolete.

[arturaz]

Thank you for the detailed explanation!

[MateuszKubuszok]

Just to be clear: it sounds like a good idea (and it might be surprising to users that it does not work currently), but it would require some thinking to come up with a solution which:

• would be extensible for every user-provided collection
• keep cooperating with DSL (withFieldConst(_.everyItem, ...), withFieldComputed(_.foo.everyMapKey.bar, ...), etc)
• could be merged and released in one go, because we are on semver 1.x.y and we are promising to keep code backward compatible, so adding and removing implicits and types is a no-no
• would not have some weird, surprising corner cases when sometimes something is silently ignored (e.g. user's overrides)

It's doable and probably the solution will be rather simple, but it will take some time to figure it out, and go through several possible options and their ups and downs.