Open ilmirus opened 3 years ago
I've been looking into a Kotlin version of TensorFlow's tf.function
which is similar to this in a lot of ways and ideally would make use of it. For those not familiar, this is a decorator that calls the function once w/ placeholder inputs to build a graph, and then uses the graph to calculate function calls. The design is still up in the air a bit (I'll update once it's more defined), but at a minimum I'll be handling:
Int
would be typed as Number
, which is currently impossible to express. even with a plugin (symbol replacement is limited to the same type).remember
function, which would need to "exit" the current context.I'm not clear on how much you want to enable handling of this stuff (specifically input/output transforms) without needing a compiler plugin, but it would be nice to have. My only need there is captures need to be handled via producing lambda rather than variable, incase they change between executions (they are added as hidden inputs to the graph, the value would be gotten from the lambda).
@rnett
Thanks a lot for the example!
If I understood you correctly, you want something like this in the end:
@TFFunction
fun caller(param: Type) {
if (param == captured) {
callee(param)
}
}
@Pure
fun callee(param: Type) {
...
}
@Serializable
class Type {
...
}
fun main() {
val graph = TFGraph.fromFunction(::caller)
tf(graph) {
// use the graph
}
}
Now, let's break this down.
Sure, it can be a single plugin, but I see how pure functions can be useful outside of your use-case. So, looks like, like libraries the plugins should be able to list their dependencies and gradle should download them. In addition, the compiler should load and run them in order.
However this is slightly weird for variable creation (a specific function), since its forbidden on all calls but the first.
This is indeed odd and notes do not cover this use-case. Can you, please, give an example, how the function is used and which usage should be forbidden? Maybe, the function should instead be provided by delegate.
Code coloring by design requires compiler support, unfortunately. It is a way to extend the language for each individual use-case, but keep the language consistent. Guidelines for plugin authors, in other words.
Yeah, that's pretty close, although the TFGraph.fromFunction(::caller)
would be implicit and done by IR rewriting, you would just call the function.
There's also my issue with captures, with how I want to transform some to a supertype (i.e. LinkedList
-> List
, which would be actually an ArrayList
inside the colored function). That's the only thing here that would require language support.
I'd like the purity checking to be done implicitly (since it's mostly going to be on 3rd party code anyways). The compiler already has a definition for it (IrExpression.isPure
) although that doesn't seem particularly complete. My thoughts here was that it seems like something useful for lots of different contexts (Compose and coroutines both don't like side effects, although coroutines can be ok), and something that should be calculated once and cached, so it would be nice to have an official or semi-official implementation in the compiler. In general it would be nice to have compiler support/helper methods for the common color scope operations, although they aren't terribly hard to implement. Probably not possible until the compiler API is finalized though.
The variable creation thing is probably going to go away at some point, the python docs are here. For the Kotlin version, an example would be:
@TFFunction
inline fun <reified T: TNumber> KotlinOps.sum( list: List<Operand<T>>): Operand<T> {
x = tf.Variable(tf.constant<T>(0))
list.forEach{ x.assignAdd(it) }
return x
}
where I'd like to forbid (for now) the tf.Variable
call. Although something like this should definitely be supported someday.
Allowed usage would be something like:
val W: Variable?
@TFFunction
fun KotlinOps.call(x: Operand<TFloat32>): Operand<TFloat32> {
init{
W = tf.Varable(0)
}
return W!! matMul x
}
where init
is a function that removes you from the colored context (and is only executed once), similar to remember
but not returning a value.
How will this affect code coloring due to @DslMarker
?
Let's take my current Kotlin/JS project with plenty of React & CSS DSL coloring for example.
As soon as I see something green I know this is React-related code. As soon as I see something pink I know this is CSS-related code.
Not every part of the DSL has a receiver, so basing color purely on the receiver won't work here.
For example public inline val Int.px: Length
for 0.px
has Int
as the receiver and nothing else.
Action(…)
is a factory function without receiver.
Here px
is colorized for good reason (it's a CSS value) but there is no way to have a specific receiver in this case.
react
in react.componentWithChildren
and children()
should also be green but sometimes aren't due to an IDE bug. The former is a global variable which serves as a namespace for global React functions.
Same for none
in UserSelect.none
. IDE bug.
Another open question is if types should also be considered colorizable instead of just function invocations (or function-like, e.g. getters).
What does the following mean regarding composition?
So, the compiler will implicitly color any given function with as many colors as it needs.
It's possible that we see @Composable suspend fun
in the future for async UI functionality similar to React Suspense. How would the IDE colorize a function invocation to make it clear that it's composable
and suspend
?
Maybe the IDE can be customized with a simple prioritized list of coloring/styling rules. Maybe the IDE allows the developer to assign more than color, just as it does today.
One developer might prefer this:
suspend
-> make italiccomposable
-> make redAnother developer might prefer this:
composable & suspend
-> make greensuspend
-> make red & italiccomposable
-> make redRules 1 conflicts with 2 & 3 in terms of color. Because it's ordered, 1 wins over 2 & 3. "italic" doesn't conflict and will be applied.
If there isn't a specific rule for code which (through annotation or otherwise) is supposed to be colorized, it'll pick a default one similar to what it does already for @DslMarker
.
Another example of mine - a Kotlin GraphQL library:
Different colors for type definitions (pink), built-in types & values (lime green) and the rest (turquoise).
Note that due to an IDE bug colors are applied inconsistently. E.g. ID
and String
are supposed to be lime green.
Like here:
One developer might prefer this:
- If
suspend
-> make italic- If
composable
-> make red
Code Coloring
is a metaphore. It does not actually relate to real world colors. You could also call it code flavours or whatever you like. It just means that a certain part of the code is compiled with different rules/strategies/context/language/features.
😅 I must have read over that. Thanks for pointing that out. Too bad. Would be amazing.
At least it's related. I currently try to impose such restrictions using @DslMarker
and sometimes by using subinterfaces as receivers that get replaced by parent interfaces when certain functionality is not supposed to be available in a scope.
A typical example are React Hooks. They can only be used in React code. And there only within functional components. And there only within the top-level "builder" block and not within nested builders/scopes. This is tricky to model with @DslMarker
and when mixing it with other DSL (e.g. CSS) limiting it becomes increasingly complex.
I hope that fits the topic better 😁
Another topic that comes to mind is an application setup library that I'm building. It's a mix of how Gradle and Ktor set up projects. There are different phases (assembly, completing assembly, assembly completed) and functionality becomes increasingly restricted with each phase. E.g. you can only add or configure new components before the assembly completed and access them in a read-only fashion afterwards but you can still finalize the configuration of your own component when the assembly completes.
Another use case is Flow
. It happens occasionally that there's an unexpected coroutine context switch that's only obvious at runtime.
Then there's blocking code within non-blocking code or coroutines.
--
For the sake of new developers I'd suggest to not call it "color(ing)" - at least in annotation names or alike. That's confusing.
@fluidsonic Thanks for the examples! I will go through them one by one.
You got the idea right. It is called "code coloring", since we do not only develop the language, but also an IDE. Thus, we can use code highlighting feature to color certain blocks of code. I did not explain it in the DN, but I probably should have. You screenshot sum up the intent perfectly! Thumbs up!
@DslMarker
and code coloring. Coloring based on (contextual) receivers will work, if the receiver is used only to indicate current color. Here we stumble across another intent for code coloring - restricting context. Currently, there is no way to say, that these types or functions are inaccessible in the context, but are accessible outside of it. Receivers and suspend functions add functionality, but do not subtract it. And no, @DslMarker
does not restrict context. It merely reports an error, when there is possible ambiguity. In my web-workers example I do not want to access DOM API. Moreover, I want to restrict the access to it in the worker.
In your example, react
should be contextual receiver.
Another example of data coloring (without types) is Compose, which does not like captured mutable variables, to say the least.
React Hooks. The clear winner. Because, it is clear, how they can be restricted to react code only. They should have react
contextual receiver, so they are not accessible from outside react code. The same with the setup library. Three interfaces, one is extending another, so code with Assembly
receiver is not accessible from AssemblyCompleted
color.
Flow and context switch. I am not sure, that I get what you mean. Do you mean switching CoroutineContext
? Year, the word "context" is kinda overused. This, by the way, is why "code coloring" has word "color" in it instead of "context" - to avoid confusion. Given, that we intent to color the code in the IDE, I do not think, that word "color" is more confusing, than "context".
Blocking code and coroutines. You are right, we would like to have a general solution, instead of current ad-hoc one. Ideally, blocking code should be accessible only from "blocking" color, which is different from more restricted "suspend" color. There was an idea floating around about "nosuspend" modifier or annotation, when the coroutines were experimental, but the idea was abandoned before 1.3. I think, this is a time to revive it in the new light of code coloring.
hi there. With Loom virtual threads, will Kotlin perhaps support a new coroutine launcher that will allow use of regular (non-suspeding/colourless) functions? Thanks in advance.
@davidmeredith hi there. With Loom virtual threads, will Kotlin perhaps support a new coroutine launcher that will allow use of regular (non-suspeding/colourless) functions? Thanks in advance.
Yes. But that's out of topic here. This enhancement will be a part of kotlinx.coroutines
Mentioning it here by request of @ilmirus on https://youtrack.jetbrains.com/issue/KT-58655 : one use-case we've run into is imposing architectural constraints rather than relying on developer conventions that may or may not be followed by the individual developers.
E.g. in the specific example prompting above issue, marking a function as being callable only from within certain other identified functions. (which we tried with ArchUnit's annotation checks, cf. https://www.archunit.org/use-cases )
This issue is for discussing and gathering feedback for Design Notes on Code Coloring: https://github.com/Kotlin/KEEP/blob/master/notes/code-coloring.md