Kotlin / KEEP

Kotlin Evolution and Enhancement Process
Apache License 2.0
3.3k stars 356 forks source link

Collection literals #112

Closed KyLeggiero closed 3 years ago

KyLeggiero commented 6 years ago

Discussion of this proposal: https://github.com/BenLeggiero/KEEP/blob/collection-literals/proposals/collection-literals.md

Related issue is #113

udalov commented 6 years ago

I couldn't understand from the proposal, what declarations exactly are you suggesting to add to the standard library, so that collection literals would be available for all standard Kotlin collection types?

For example, to make this code possible:

val a1: List<String> = ["a", "b", "c"]
val a2: Array<String> = ["a", "b", "c"]

We should add at least the following declarations:

operator fun <T> sequenceLiteral(vararg elements: T): Array<T> = ...
operator fun <T> sequenceLiteral(vararg elements: T): List<T> = ...

But these are conflicting overloads and Kotlin does not allow that.

ilya-g commented 6 years ago

@udalov I believe this proposal implies allowing these conflicting overloads for sequenceLiteral operator.

KyLeggiero commented 6 years ago

Great observation, @udalov. I only found that issue late into the final draft. @ilya-g is correct. There might also have to be a second KEEP for resolution of such overloads, but... I am not a bytecode engineer so I'm not sure how that would work under the covers. I would need help from more experienced JVM engineers.

helmbold commented 6 years ago

Having the same syntax for lists and sets contradicts the point of beeing concise, since you have to specifiy the type explicitly:

val myList = [1, true, "three"]
val myEmptySet: Set<String> = []

And it would be inconsistent. Why is List a "first-class citizen" and Set not?

Combining a collection literal with a type conversion makes the whole approach absurd:

val b2 = [1, 2, 3] as MutableList<Int>

The existing approach is much cleaner and consistent:

val b2 = mutableListOf(1, 2, 3)

If the goal is to make the code more concise, one could use mnemonics like "s" for "set" or "m" for "map":

val myMap = m("a": 1, "b": 2)
val myList = l(1, 2, 3)
val mySet = s("a", "b", "c")

However this doesn't play nicely with mutable collections -- but this is true for the syntax proposed in the KEEP as well. I'd go as far as to say that the literal syntax should only exist for immutable collections. But if we leave it as is we have a consistent way for mutable and immutable collections.

Personally I like the way of Scala best:

val myMap = Set("a" -> 1, "b" -> 2)
val myList = List(1, 2, 4)
val mySet = Set("a", "b", "c")

It looks much better than the collectionOf methods in Kotlin. From my point of view this is pure beauty, very readable, and in the best sense of a "scalable" language. I've never heard anyone complaining about the "verbosity".

All in all I'd prefer to leave the syntax for collections as is, if the alternative (as proposed in the KEEP) would introduce so much inconsistencies.

accursoft commented 6 years ago

Collection literals may be more relevant for passing as parameters. For example, if you have fun foo(list: List<Int>) = ... You could call it with foo([1, 2, 3]). This is particularly relevant for DSLs. For example, listOf and mapOf can make Gradle scripts somewhat ugly when translating from Groovy to Kotlin.

helmbold commented 6 years ago

@accursoft If collections literals are used for passing them as parameters, one could pass the collection elements as vararg, at least if there is only one such parameter in a method call. Otherwise it would possible to use a builder.

To translate your example in this style:

fun foo(vararg list: Int) = ...
foo(1, 2, 3)

The builder style would look like this:

SomethingComplex.a(1, 2, 3).b(2, 4, 6, 8).c("one" to 1, "two" to 2).build()

Where

The builder in this example would look like this:

class SomethingComplex(a: List<Int>, b: Set<Int>, c: Map<String, Int>) {

    // do something useful with a, b, and c

    class Builder(private val a: List<Int>) {
        private var b = setOf<Int>()
        private var c = mapOf<String, Int>()

        fun b(vararg b: Int) : Builder {
            this.b = b.toSet()
            return this
        }

        fun c(vararg items: Pair<String, Int>) {
            this.c = items.toMap()
        }

        fun build() = SomethingComplex(a, b, c)
    }

    companion object {
        fun a(vararg a: Int) = Builder(a.toList())
    }
}

So it is relatively easy to make nice APIs without collection literals.

gildor commented 6 years ago

@helmbold You right, but if you want to pass existing collection to vararg method it looks not so nice: SomethingComplex.a(*someA.toTypedArray()).b(*someB.toTypedArray()).c(*someC.toTypedArray()).build()

instead of

SomethingComplex.a(someA).b(someB).c(someC).build()

Also vararg copies array on a method call. So the solution with vararg is not so universal

helmbold commented 6 years ago

@gildor But if you want to pass existing collections, you could overload the respective functions. In my example this would be:

fun b(b: Set<Int>) : Builder {
    this.b = b
    return this
}

I don't want to deny that collections literals could be useful in some scenarios, but if I see all the compromises and inconsistencies in the proposal, I think the already possible solutions are not so bad.

However, I would suggest to introduce the Scala way to Kotlin and allow something like List(1, 2, 3) instead of listOf(1, 2, 3). It would look so much better, even if it wouldn't be the Kotlin Way ™.

ilya-g commented 6 years ago

@helmbold So you propose an alternative where one is required to write a builder class for each function he wants to pass collection literal to? I don't think this approach scales at all.

Let's keep the discussion and critics focused on this particular proposal. If there's an alternative that goes completely in another direction, it's better to start a distinct KEEP proposal for that.

KyLeggiero commented 6 years ago

Archived Slack discussion of first draft: https://imgur.com/a/NKw0MCF

ilya-g commented 6 years ago

What this proposal currently lacks is the analysis of situations when there are multiple sequenceLiteral functions applicable and the rules how to choose one of them or report an ambiguity.

Examples are:

accursoft commented 6 years ago

There is a lot of work (and complication) here to cover the non-default use cases ... are they actually common enough to be worth it? I suspect that >=90% of actual requirements would be met with just List and Map, <10% of the proposed implementation.

gildor commented 6 years ago

@accursoft Arrays is least useful case for literals imo, usually you use lists or any other collections. Arrays use case is only performance critical parts and mostly you avoid to expose arrays on you public API (they are mutable). Also array literals have own poblems with ambiguous types: val a = [1,2,3] What is type of a? IntArray or Array[Int]?

gildor commented 6 years ago

Why is List a "first-class citizen" and Set not?

@helmbold List is much more widely used collection than set or any other data structure. I think it's completely fine solution to choose most common collection as default (it's also common practice in many languages)

accursoft commented 6 years ago

@gildor I'm proposing that [a,b,c] creates a List, the same as listOf(a,b,c) would. I only mentioned arrays in the context of having one hard-coded type for collection literals. Arrays in annotations, Lists in code (and perhaps Maps for dictionary literals).

Not completely consistent, but highly pragmatic.

fvasco commented 6 years ago

I agree with @accursoft, data structure type is important and sequenceLiteral operator hides this kind of information.

Moreover the "Motivation" section should be revisited

Having a cool synstax to define read-only, random-access list is pratical, same for maps.

Instead having

takesAmbiguousType(["1", true, 3] as Set<Any>)

working and

val list = ["1", true, 3] 
takesAmbiguousType(list as Set<Any>)

not working looks strange, IMHO.

helmbold commented 6 years ago

Let's say we could settle with parenthesis for all collection literals (a, b, c) and that we would distinguish the actual types with a prefix what would syntactically be similar to Scala string interpolation, for example:

s"Hello, $name"
f"$name%s is $height%2.2f meters tall"

... then we would have the syntax I suggested above (s(1, 2, 3) for sets, m("a" to 1, "b" to 2) for maps and so on). I just want to show with this Scala example that we are possibly discussing an overly complex solution for an actually small problem.

voddan commented 6 years ago

Why is List a "first-class citizen" and Set not?

@helmbold List is much more widely used collection than set or any other data structure. I think it's completely fine solution to choose most common collection as default (it's also common practice in many languages)

@gildor don't forget the motivational factor. If lists are much easier to instantiate than sets, then they will be used instead of sets even when a set is more appropriate.

In my practice about 80% are lists, and 20% are sets because logic does not require ordering. If we give lists special treatment, that ratio will shift to something like 95%-5% for lists-vs-sets.

We can see that effect in Java APIs, where arrays prevail over lists or sets just because they have the [] syntax. Such APIs are confusing, inflexible, and a pain to work with. I would not want that disease for Kotlin!

fvasco commented 6 years ago

Some considerations:

[1, 2, 3] as Set<Int>

has same performance of

[1, 2, 3].toSet()

so we consider

val s1 = [1, 2, 3] as Set<Int> // deny
val s2 = [1, 2, 3].toSet() // allow
val s3 : Set<Int> = [1, 2, 3] // allow

As alternative using "prefix dictionaries with a # symbol" is a "natural way to distinguish a literal for a List<Pair<*, *>> from a Map<*, *>"

fvasco commented 6 years ago

A way to fix the follow issue without any special treatment

operator fun <T> sequenceLiteral(vararg elements: T): Array<T> = ...
operator fun <T> sequenceLiteral(vararg elements: T): List<T> = ..

is to shift this operator into the Compantion object:

operator fun <T> Array.Companion.sequenceLiteral(vararg elements: T):  Array<T> = ...
operator fun <T> List.Companion.sequenceLiteral(vararg elements: T): List<T> = ..

this avoid any kind of ambiguity (I hope :)

Deeping into this way (merging with the Java style) it is possible rename the sequenceLiteral operator to of, ie:

operator fun <T> Set.Companion.of(vararg elements: T): Set<T> = ..

this allows replacing

val set = ([ "a", "b", "c"] as Set<String>).map { transform(it) }

to

val set = Set.of( "a", "b", "c").map { transform(it) }
gildor commented 6 years ago

@fvasco There is a problem with this approach: You cannot implement it for a collection without existing companion object, so it's just impossible to add collection literal syntax for a Java class or for a Kotlin class without companion object

fvasco commented 6 years ago

Hi @gildor, Kotlin 1.2 does not support this syntax, right. However I cannot understand your opinion about this way to solve the question.

Is the original propostal more appropriate than this one?

Do we discuss in a different KEEP a solution for the current Kotlin limitation (ie: declaring a operator companion fun <T> Set.of(vararg elements: T): Set<T> = .. extension function)?

Do we consider some alternative way to current proposal, ie: considering the syntax

val a : MyType = [ 1, 2, 3]

as syntactically equivalent of

val a : MyType = MyType(1, 2, 3)

so it is possible to write

fun MyType(vararg ints : Int) : MyType = TODO()

// or

class MyType(vararg ints : Int) { ... }

// or

class MyType {

  companion {

    operator fun invoke(vararg ints : Int) : MyType = TODO()

  }

}

note: this last proposal allows the follow syntax

data class Point(val x: Int, val y: Int)

fun distance(p1: Point, p2: Point) = ...

val delta = distance([2, 3], [4, 5])

without adding more code

zxj5470 commented 5 years ago

Enable type hint, the type of s is Array image And I think specify type is OK such as val s:IntArray = [1,2,3] or val s:List<Int> = [1,2,3] with implicit cast

hastebrot commented 5 years ago

Using type annotations like this

val foo: List<Int> = [1, 2, 3]

can get very cumbersome when used in nested structures (which might actually be the biggest reason to introduce this syntax sugar):

val foo: List<???> = [10, 20, [1, 2, [25, 50]], ["Groovy"] as Set]
ilya-g commented 5 years ago

@zxj5470 Given that collection literals are currently not supported outside of annotations, it doesn't make sense to reason what type is inferred for a collection literal when the code doesn't compile.

helmbold commented 5 years ago

Oracle had planned to introduce collection literals in Java 8, but dropped the idea. The arguments are valid for Kotlin as well.

bennylut commented 5 years ago

I have couple of performance oriented questions:

it seems that in this proposal, in order to create maps the "key value sequence" will first be wrapped in a list of pairs just to be destructed back into a map.

This seems to be quite wasteful. To the best of my knowledge, the JIT will not be able to do an escape analysis on such usage profile which means that the list and each pair will be actually allocated, filled, then iterated on inside the sequenceLiteral function. Do you have any plans to "inline" the function in some way to avoid this performance overhead?

In addition, I did not find any reference to primitive collections and arrays while they are extremely important (performance wise) - do you have any plans to avoid boxing/unboxing the entries in primitive collections? (collections like Matrix, Vector, Int2DoubleMap, etc. which are especially common in scientific stuff that will probably use collection literals a lot)

helmbold commented 5 years ago

@accursoft I haven't said that Kotlin should follow every decision of Oracle, but considering their reasoning is a chance to learn. It is a good intellectual habit to study prior art.

accursoft commented 5 years ago

Swift:

accursoft commented 5 years ago

Oracle is referring to Java, a language which already has array literals. A monomorphic collection literal is sufficient for many use cases, e.g. DSLs.

Dico200 commented 5 years ago

A couple of things:

Comments on the Motivation as per the KEEP:

These are just my opinions with some added facts. Please tell me where you disagree.

Dico200 commented 5 years ago

For implementation, it would probably be a good idea to implement the collectionLiteral functions as some sort of collector interface, such as Java's Collector which is used in Streams. This would avoid the performance overhead introduced by instantiating arrays and/or arrays of pairs (for maps) as mentioned by @bennylut

noncom commented 5 years ago

Just a thought... as for syntax on such shorthands, the best one I know is in Clojure (adapted to be more Kotlin-friendly):

The ! variant is usully used for something dynamic and mutable, so it would be a nice choice for the mutable counterparts. It should not contradict with the boolean negation semantics, since that semantics is out of question when you're about sequence literals. So it would be !(1, 2, 3), ![1, 2, 3], !{1 to 2, 3 to 4} and !#{1, 2, 3} for mutables. The ! can be doubled to get !!(1, 2, 3) which is more of a visual aid and also an allusion to the already existing !! in Kotlin that reminds us that all things are temporary.

Other variants are possible like ~(1, 2, 3) or *(1, 2, 3), but these are not as comfortable to type, I think that ! or !! are better.

helmbold commented 5 years ago

@noncom I think the syntax with {} is problematic since Kotlin uses this syntax already for anonymous functions.

noncom commented 5 years ago

@helmbold Well, yes, in case with a single pair it would work as an anonimous function literal.. I think that some symbol, like :: or \ or ~ could be added in front, like ::{}, like for the sets to announce that it's a map. Maps and functions are similar in the way that both map one value to another, so the {} part of syntax is not completely irrelevant here.

tieskedh commented 5 years ago

I was always in favor of literals, as literals are more easy to read as text. But then it needs to be kept simple: that is not a lot of different syntaxis to learn, so it would be easy.

That having said, I have an (im?)possible idea based on @noncom:

I think they should be created by operator functions:

inline operator fun <T> createList(vararg args: T)  : List<T>
inline operator fun <T> createArray(vararg args: T)  : Array<T>
inline operator fun <S, T> createMap(vararg args: Pair<S,T>)  :  Map<T>
inline operator fun <T> createSet(vararg args: T)  : Set<T>

inline operator fun <S, T> createGenericCollection(vararg args: Pair<S,T>)  :  ...
inline operator fun <S> createGenericCollection(vararg args : S)  :  ...
// ,,, means unrestricted

These operator functions places the functions in the companion-object, such that the import says:

import LinkedArrayList.companion.createList
import HashMap.companion.createMap
import Array.companion.createArray
import HashSet.companion.createSet
import IntArray.compantion.createGenericCollection

This can be combined with the ~ for mutability, (! would clash a bit with the not-operator). I however prefer that the GenericCollection is not implemented as I feel that it could be abused very easily, but then IntArray doesn't have a literal...

pro:

con:

Dico200 commented 5 years ago

@tieskedh

Can't we just convert these functions to operator functions?

inline fun intArrayOf(vararg elems: Int): IntArray
...
inline fun <T> arrayOf(vararg elems: T): Array<T>

As for the other ones, (I think) they allocate an array to create the collection or map, which would be something to consider avoiding.

pdvrieze commented 5 years ago

I would like to specify one of the requirements I would see for the topic. From my perspective a custom written collection should be supported such that its use is as convenient as a standard collection. This means that a collection literal must be able to work to initialise such custom collection. Currently the way lists etc are supported meets this (except for the MutableList magic applied to java classes, but that's a separate issue). A naive collection literal breaks this in some cases. For example

  val list1: List<Int> = [1, 2, 3, 4] // works like `listOf(1,2,3,4)`
  val list2 = listOf(1, 2, 3, 4) // notice this is shorter
  val list3 = [1, 2, 3, 4] // What should the type of list3 be? Is it List<Int> ?

  val list4: MyList<Int> = [1, 2, 3, 4] // Should be possible, has clear semantics, but how does it work? how is a My list created
  val list5 = myListOf(1, 2, 3, 4) // Already possible, clear how and where, still shorter
  val list6 = [1, 2, 3, 4] // This would not be of type MyList<Int>, but of type List<Int>
AarjavP commented 5 years ago

Just want to share my thoughts on this with respect to what I like about kotlin. I am not really considering the implementation for these, just the usage. in no particular order:


As per a solution, I have not thought too much into this, however this is what I propose:

Here is a simple data class without nested collections/maps

data class Foo(val name: String, var ver: Int, val l: List<Int>, val map: Map<String, Boolean>)

println(Foo(name = "a", ver = 1, l = listOf(3,4,5), map = mapOf("test" to true)))
// prints:
// Foo(name=a, ver=1, l=[3, 4, 5], map={test=true})

//with literals:
Foo(name="a", ver=1, l= listOf [3, 4, 5], map= mapOf {"test"=true})

I have not thought of the implementation, and honestly I do not know kotlin in depth, but could it be possible to use the same approach as SAM conversions? have the xxxOf methods marked somehow so the compiler knows to treat the [] or {} blocks specially, which may be something simple as the parameter types and return type needed to make the collection.

helmbold commented 5 years ago

@AarjavP I agree with your arguments to keep the functions like listOf, however it doesn't make sense then to introduce square brackets and curly braces along with these functions. Why should anyone want to introduce this inconsistency without any benefit?

Despite that I doubt that it could be implemented without massive compiler hacks.

shubhang93 commented 5 years ago

I think we can borrow some ideas from Clojure, by far all the languages I have worked, Clojure has the best literal syntax I feel. A list can be declared as

val list = l(1,2,3,4,5)

An array

val myArray = [1,2,3,4,5]

A Map

val myMap = {:key value,:key2 value}
//But since clojure uses prefix notation it wouldn't make sense in Kotlin
val myMap = {key->value, key2->value} would also work, I guess
janvladimirmostert commented 5 years ago

The only place where collection literals would make sense is where the type is already known, otherwise we go into a debate whether [1,2,3] means List, MutableList, Set or Array, etc

val list: ArrayList<Int> = [1, 2, 3, 4];
val set: Set<Int> = [1, 3, 4, 5];
val map: Map<String, String> = {"1":"one"}
val mutableMap: MutableMap<String, String> = {"2":"two"}

You're not gaining much here in terms of the number of characters you're typing. Annotations already allow collection literals and it makes sense to do so here, since the type is already known and it actually saves a few characters of typing.

@Blah(data = [1,2,3]) VS @Blah(data = listOf(1,2,3))

The one other place where there might be some minor benefit is when making function calls:

fun doSomethingWithList(list: List) { .... }

doSomething(list = [1,2,3]) VS doSomething(list = listOf(1,2,3))

Maybe when returning data as well since the type is known there too:

fun blah(): List { return ["1", "2", "3"] }

In both cases, the compiler would just treat [...] as it would have treated listOf

-- Jan Vladimir Mostert janvladimirmostert.com

On Sat, 16 Mar 2019 at 21:15, shubhang93 notifications@github.com wrote:

I think we can borrow some ideas from Clojure, by far all the languages I have worked, Clojure has the best literal syntax I feel. A list can be declared as

val list = l(1,2,3,4,5)

An array

val myArray = [1,2,3,4,5]

A Map

val myMap = {:key value,:key2 value} //But since clojure uses prefix notation it wouldn't make sense in Kotlin val myMap = {key->value, key2->value} would also work, I guess

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Kotlin/KEEP/pull/112#issuecomment-473576184, or mute the thread https://github.com/notifications/unsubscribe-auth/ACKxdk6kTAudLigL6tVwqi-W-UVAtjKjks5vXUJKgaJpZM4T867L .

hastebrot commented 5 years ago

Why should anyone want to introduce this inconsistency without any benefit?

@helmbold Absolutely. The benefit of collection literals is small. And we even did not talk about rest spread operators like [1, 2, ...otherList] or {foo: 1, bar: 2, ...otherMap} or even shorthand property names like {foo, bar} given val foo = 1; val bar = 2;. Or using variables for map keys (computed property names in ES2015): {[key]: 42} given val key = "answer";.

Above examples in Kotlin:

fun main() {
    run {
        val otherList = listOf(3, 4, 5)
        println(listOf(1, 2) + otherList)
        // [1, 2, 3, 4, 5]

        // collection literals: [1, 2, ...otherList]
    }

    run {
        val otherMap = mapOf("baz" to 3, "quux" to 4)
        println(mapOf("foo" to 1, "bar" to 2) + otherMap)
        // {foo=1, bar=2, baz=3, quux=4}

        // collection literals: { foo: 1, bar: 2, ...otherMap }
    }

    run {
        val otherMap = mapOf("foo" to 1, "bar" to 2)
        println(otherMap + mapOf("bar" to 3, "baz" to 4))
        // {foo=1, bar=3, baz=4}

        // collection literals: { ...otherMap, bar: 3, baz: 4 }
    }

    run {
        val foo = 1
        val bar = 2
        println(mapOf("foo" to foo, "bar" to bar))
        // {foo=1, bar=2}

        // collection literals: { foo, bar }
    }

    run {
        val key = "answer"
        println(mapOf(key to 42))
        // {answer=42}

        // collection literals: { [key]: 42 }
    }
}
BreimerR commented 4 years ago

Don't really understand the difficulty here because I take it javascript also associates the curly braces with a scope but still manage to make sense of it in for as a map literal

BreimerR commented 4 years ago

when(a){ is String -> { false } else -> { true } }

this is completely different from val b = { a : 12, b : "bar" }

jnorthrup commented 4 years ago

i wonder if we can get a cameo from @gvanrossum to give a temperature on collection literals in hindsight of his experience bdfl'ing a popular language that opted in. ?

gvanrossum commented 4 years ago

No regrets, except there’s a problem with empty sets that could have been solved differently if we had had sets from day 1. But since you have type declarations it may not even be a problem for you.

jnorthrup commented 4 years ago

sadly, it looks like this ship has sailed for the most part. https://blog.jetbrains.com/kotlin/migrating-tuples/?_ga=2.67367192.867902301.1576357503-1820613938.1571294470

gvanrossum commented 4 years ago

Well it's kind of cool that Kotlin adopted PEP 557. :-)

KyLeggiero commented 4 years ago

@jnorthrup

sadly, it looks like this ship has sailed for the most part. https://blog.jetbrains.com/kotlin/migrating-tuples/?_ga=2.67367192.867902301.1576357503-1820613938.1571294470

That blog post seems unrelated to collection literals, but thank you for sharing