Untupled parameters in context functions aren't provided as givens

coreywoodfield commented 5 days ago

Compiler version

3.5.1

Minimized code

case class A(a: Int)

def foo(f: ((Ordering[A], Any)) ?=> Unit) = ()

// compiles: parameter untupling occurs for the ContextFunction
foo { (ordA, ordB) ?=>
  (0 to 5).map(A(_)).sorted(using ordA)
  // doesn't compile: ordA isn't provided as a given
  (0 to 5).map(A(_)).sorted
}

def bar(f: (Ordering[A], Any) ?=> Unit) = ()

bar { (ordA, ordB) ?=>
  // compiles: if no untupling takes place, all parameters are provided as givens
  (0 to 5).map(A(_)).sorted
}

Output

-- [E172] Type Error: .../Test.scala:9:29 
120 |    (0 to 5).map(A(_)).sorted
    |                             ^
    |No implicit Ordering defined for B..
    |I found:
    |
    |    scala.math.Ordering.comparatorToOrdering[B](
    |      /* missing */summon[java.util.Comparator[B]])
    |
    |But no implicit values were found that match type java.util.Comparator[B]
    |
    |where:    B is a type variable with constraint >: MultiServiceTest.this.A
    |.
    |
    |One of the following imports might make progress towards fixing the problem:
    |
    |  import scala.math.Ordering.comparatorToOrdering
    |  import scala.math.Ordering.ordered
    |
one error found

Expectation

I expected the parameters to be provided as givens whether or not the context function was subject to untupling. The context functions passed to foo and bar are identical except in body, so it's surprising that they don't function identically.

The reference on parameter untupling says

The tuple parameter is decomposed and its elements are passed directly to the underlying function.

which to me suggests that the "underlying function" shouldn't ever know that its parameters were untupled.

Gedochao commented 4 days ago

@coreywoodfield The example seems to be missing

case class B(b: Int)

So I'm guessing the full repro (in a raw .scala file, rather than a .sc script) is:

case class A(a: Int)
case class B(b: Int)

def foo(f: ((Ordering[A], Any)) ?=> Unit) = ()
def bar(f: (Ordering[A], Ordering[B]) ?=> Unit) = ()

@main def main() = {
  // compiles: parameter untupling occurs for the ContextFunction
  foo { (ordA, ordB) ?=>
    (0 to 5).map(A(_)).sorted(using ordA)
    // doesn't compile: ordA isn't provided as a given
    (0 to 5).map(A(_)).sorted
  }
  bar { (ordA, ordB) ?=>
    // compiles: if no untupling takes place, all parameters are provided as givens
    (0 to 5).map(A(_)).sorted
  }
}

prolativ commented 4 days ago

This might not be obvious at a first glance, but IMO this is not a bug against the language specification. The linked documentation page describes parameter tupling for functions, which is a mechanism that could be described as below: If in some context a function from a single tuple parameter (e.g ((X, Y)) => Z) is required, but a function literal with multiple parameters (e.g. a lambda of type (X, Y) => Z, i.e. { (x: X, y: Y) => z: Z }) is provided in the codebase instead, the compiler converts the multi-param function to a function with a single tuple parameter.

However, context functions are not a subcategory of functions (kind of like guinea pigs are not pigs) and they don't work exactly the same way. Let's consider the snippet below:

trait Qux
trait Quz

def foo(f: ((Qux, Quz)) ?=> Int) = ()
def bar(f: (Qux, Quz) ?=> Int) = ()

bar expects a 2-parameter context function as it's argument, while foo expects a 1-parameter context function as an argument (with the type of the parameter being a tuple). Both of the parameter context functions should return an Int. In practice these signatures mean that both foo and bar expect a block of code returning an Int as its argument, but inside this block the context parameters of the context function are available in the given (implicit) scope. In case of bar there are 2 context parameters, of types Qux and Quz. In case of foo there is a single context parameter of tuple type (Qux, Quz). So these lines do compile:

  foo { 1 }
  bar { 2 }
  foo { summon[(Qux, Quz)]; 3 }
  bar { summon[Qux]; summon[Quz]; 4 }

but these don't

  foo { summon[Qux]; summon[Quz]; 5 }
  bar { summon[(Qux, Quz)]; 6 }
  foo { summon[(Quz, Qux)]; 7 }

and this is an expected behaviour.

Now we could slightly desugar these lines by adding explicit context lambda parameters (with explicit types, or letting the compiler infer them from the signatures of foo and bar). Compiling:

  foo { (quxQuz: (Qux, Quz)) ?=> 1 }
  foo { quxQuz ?=> 1 }

  bar { (qux: Qux, quz: Quz) ?=> 2 }
  bar { (qux, quz) ?=> 2 }

  foo { (quxQuz: (Qux, Quz)) ?=> summon[(Qux, Quz)]; 3 }
  foo { quxQuz ?=> summon[(Qux, Quz)]; 3 }

  bar { (qux: Qux, quz: Quz) ?=> summon[Qux]; summon[Quz]; 4 }
  bar { (qux, quz) ?=> summon[Qux]; summon[Quz]; 4 }

Not compiling:

  foo { (quxQuz: (Qux, Quz)) ?=> summon[Qux]; summon[Quz]; 5 }
  foo { quxQuz ?=> summon[Qux]; summon[Quz]; 5 }

  bar { (qux: Qux, quz: Quz) ?=> summon[(Qux, Quz)]; 6 }
  bar { (qux, quz) ?=> summon[(Qux, Quz)]; 6 }

  foo { (quxQuz: (Qux, Quz)) ?=> summon[(Quz, Qux)]; 7 }
  foo { quxQuz ?=> summon[(Quz, Qux)]; 7 }

Having this in mind, let's have a look back at the initial snippet of this issue. Trying to simplify things we could change

  foo { (ordA, ordB) ?=>
    (0 to 5).map(A(_)).sorted(using ordA)
    // doesn't compile: ordA isn't provided as a given
    (0 to 5).map(A(_)).sorted
  }

to

  foo { (ordA, ordB) ?=>
    summon[Ordering[A]]
  }

This still wouldn't compile, which is contrary to what @coreywoodfield would expect, if I understood the problem correctly. But IMO this expectation seems wrong, taking into account my explanations above, because what we have in the implicit scope inside foo { ... } is not given instances of Ordering[A] and Any, but a single given instance of (Ordering[A], Any). And in general, not even taking parameter tupling into account at all, there's a difference in the language between

given (X, Y) = (x, y)

and

given X = x
given Y = y

Making these 2 equivalent would require quite serious changes to the language and as far as I know there are no plans to introduce such behaviour

prolativ commented 4 days ago

Or maybe just my understanding of desugaring of context lambdas is quite limited, because TBH I'm quite surprised that

foo { (ordA, ordB) ?=>
    (0 to 5).map(A(_)).sorted(using ordA)
}

actually works

coreywoodfield commented 4 days ago

@coreywoodfield The example seems to be missing case class B(b: Int)

Ah whoops, yes, I initially had that, but then realized that the B class wasn't needed. I just forgot to also change bar to def bar(f: (Ordering[A], Any) ?=> Unit) = () (I've updated the original message to no longer reference B as a type)

coreywoodfield commented 4 days ago

@prolativ I mostly agree with you. I also wasn't sure if

foo { (ordA, ordB) ?=>
    (0 to 5).map(A(_)).sorted(using ordA)
}

would work, but when it did, I reasoned that if the parameter was untupled, the givens also should be. I could reasonably see this going either way: we stop untupling context functions, or we also start untupling the givens.

Really my problem here is that I want to do something like the following:

trait Context[T] {
  // ...
}

trait Trait {
  type Types <: Tuple
  val values: Types

  def withContext[T, R](t: T)(f: Context[T] => R): R = ???

  def curried[R: ClassTag](f: Tuple.Fold[Tuple.Map[Types, Context], R, ContextFunction1]): R = {
    def inner[T <: Tuple](values: T, fOrR: Tuple.Fold[Tuple.Map[T, Context], R, ContextFunction1]): R = {
      type Head = Tuple.Head[T]
      type Tail = Tuple.Tail[T]
      type F = Context[Head] => Tuple.Fold[Tuple.Map[Tail, Context], R, ContextFunction1]
      (values, fOrR) match {
        case (EmptyTuple, r: R) => r
        case ((head: Head @unchecked) *: (tail: Tail @unchecked), f: F @unchecked) =>
          withContext(head)(context => inner(tail, f(context)))
    }
    inner[Types](values, f)
  }

  def tupled[R](f: Tuple.Map[Types, Context] ?=> R): R = {
    def inner[N <: Int](values: Tuple.Drop[Types, N], contexts: Tuple.Take[Tuple.Map[Types, Context], N] = EmptyTuple): R = {
      (values: Tuple) match {
        case EmptyTuple => f(using contexts.asInstanceOf[Tuple.Map[Types, Context]])
        case head *: tail =>
          withContext(head) { context =>
            inner(
              tail.asInstanceOf[Tuple.Drop[Types, N + 1]],
              (contexts *: context).asInstanceOf[Tuple.Take[Tuple.Map[Types, Context], N + 1]]
            )
          }
      }
    }
  }
}

class Class extends Trait {
  type Types = (Int, String)
  val values = (1, "foo")

  // This mostly works how I want, except it suffers from https://github.com/scala/scala3/issues/21792
  // and I'd prefer to just have a single context function instead of many
  curried { intContext ?=> stringContext ?=>
    ...
  }

  // tupled _looks_ how I want, but doesn't provide the contexts as givens
  tupled { (intContext, stringContext) ?=>
    ...
  }
}

I want something in between curried and tupled. Ideally I'd just have a ContextFunctionN that took the members of the tuple as parameters, but I don't see any way to do that without macros, as I don't statically know the size of the tuple while in Trait. (Also, as noted, both curried and tupled have small problems that can be worked around but that are annoying)

Again, I did expect

  foo { (ordA, ordB) ?=>
    summon[Ordering[A]]
  }

to work, but not because of anything in the language spec. I just expected the second line to work because the first line works, and because it looks the same as

  bar { (ordA, ordB) ?=>
    summon[Ordering[A]]
  }

which does work. I think making the foo case work, or making it fail on the first and second lines (i.e. fail to untuple the parameters) rather than just failing on the second line, would both be reasonable courses of action here.

coreywoodfield commented 4 days ago

For my specific case I guess I can add

  given elem[A <: Tuple.Union[Tuple.Map[Types, Context]]: Typeable](using t: Tuple.Map[Types, Context]): A =
    t.toList.collectFirst { case a: A => a }.get

I don't love that, but it works for now (this probably actually wouldn't work for my example as written, because of erasure, but in my real code, Context is a type alias that ends up resolving to unique types without type parameters for each member of Types)

prolativ commented 3 days ago

If I understand your use case correctly:

You want to abstract over the context parameters of a context function (arity and types of parameters)
You already have a way to do that for curried context functions
At use site all of these should be valid:
```
foo { summon[Ordering[A]] }
foo { (ordA, ordB) ?=> summon[Ordering[A]] }
foo { (ordA, ordB) ?=> summon[Ordering[A]](using ordA) }
```
If so I think as a workaround you could try doing something like
```
def foo(f: ((Ordering[A], Ordering[B])) ?=> Ordering[A] ?=> Ordering[B] ?=> Unit) = ()
```
Then you would get all the given instances in scope thanks to currying but you could also write (ordA, ordB) ?=> at the beginning of your lambda to reference each of the contextual parameters directly. Although that's not perfect either and the signature of foo would look even more horrifying for bigger arities. IMO the way to go here would be to rely on currying rely on summon if explicit access to a specific contextual parameter is needed

scala / scala3