search

enso-org / enso

Enso Analytics is a self-service data prep and analysis platform designed for data teams.
https://ensoanalytics.com
Apache License 2.0
7.39k stars 323 forks source link

@widgets for extension functions of `Table` may not be found #6955

Closed radeusgd closed 1 year ago

radeusgd commented 1 year ago

As experienced in #6925, the primary_key annotation on select_into_database_table operation seems to be not registered.

Actual behaviour

image

The widget on primary_key is a Vector Editor but it only allows to add a Nothing, not the actual column names as expected. Checking Widgets.get_widget_json we can see that the annotation for @primary_key seems to not be applied for some reason.

Expected behaviour

The "primary_key" widget JSON should not be null - it is defined to be Widget_Helpers.make_column_name_vector_selector.

The above should then allow the vector editor to display the proper column selector dropdown.

Context

We checked if this may be due to the fact that select_into_database_table was defined on renamed types:

import Standard.Table.Data.Table as In_Memory_Table

...

@primary_key ...
In_Memory_Table.select_into_database_table : ...
In_Memory_Table.select_into_database_table self ... primary_key ... =
    ...

but we split the extensions file into separate versions for in-memory and Database Tables, allowing to get rid of the renames and the issue persisted, so it is probably something else.

jdunkerley commented 1 year ago

The method is an extension in the Database module.

When the IDE calls through it call with the type and module name correctly and gets passed onto Meta.get_annotation but the methodFunction cannot be resolved.

JaroslavTulach commented 1 year ago

What is the program to test on? What shall I search for in that program? I tried to create something, but it doesn't work:

obrazek

radeusgd commented 1 year ago

What is the program to test on? What shall I search for in that program? I tried to create something, but it doesn't work:

obrazek

Looks like a typo in SQLite - note the single L.

radeusgd commented 1 year ago

Also note that the select_into_database_table operation was added pretty recently, in #6925, so ensure that you've got a recent build.

JaroslavTulach commented 1 year ago

Thank you for spotting the typo. I got over it. Now I cannot use Widgets:

Image

What's your import statement? I have no idea how you code works. I had to:

enso$ git diff
diff --git distribution/lib/Standard/Visualization/0.0.0-dev/src/Main.enso distribution/lib/Standard/Visualization/0.0.0-dev/src/Main.enso
index 7dbda46d96..ca56cf6f68 100644
--- distribution/lib/Standard/Visualization/0.0.0-dev/src/Main.enso
+++ distribution/lib/Standard/Visualization/0.0.0-dev/src/Main.enso
@@ -9,3 +9,6 @@ from project.File_Upload export file_uploading
 export project.Id.Id
 export project.Helpers

+import project.Widgets
+export project.Widgets
+

I wish bugreports had simple steps to reproduce. But I think I see the problem now.

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-07):

Progress: - failed to reproduce: https://github.com/enso-org/enso/issues/6955#issuecomment-1580055828

Next Day: Bugfixing & Runtime Type Checks

Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
JaroslavTulach commented 1 year ago

This is a dirty fix:

enso$ git diff engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/GetAnnotationNode.java
diff --git engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/GetAnnotationNode.java engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/GetAnnotationNode.java
index bbd28da286..efc90d9e26 100644
--- engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/GetAnnotationNode.java
+++ engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/GetAnnotationNode.java
@@ -1,7 +1,16 @@
 package org.enso.interpreter.node.expression.builtin.meta;

+import com.oracle.truffle.api.CompilerDirectives;
+import com.oracle.truffle.api.Truffle;
+import com.oracle.truffle.api.dsl.Cached;
+import com.oracle.truffle.api.dsl.Specialization;
+import com.oracle.truffle.api.frame.FrameInstance;
+import com.oracle.truffle.api.frame.VirtualFrame;
+import com.oracle.truffle.api.library.CachedLibrary;
+import com.oracle.truffle.api.nodes.Node;
 import org.enso.interpreter.dsl.BuiltinMethod;
 import org.enso.interpreter.node.BaseNode;
+import org.enso.interpreter.node.EnsoRootNode;
 import org.enso.interpreter.node.callable.thunk.ThunkExecutorNode;
 import org.enso.interpreter.node.expression.builtin.text.util.ExpectStringNode;
 import org.enso.interpreter.runtime.EnsoContext;
@@ -13,12 +22,6 @@ import org.enso.interpreter.runtime.library.dispatch.TypesLibrary;
 import org.enso.interpreter.runtime.scope.ModuleScope;
 import org.enso.interpreter.runtime.state.State;

-import com.oracle.truffle.api.CompilerDirectives;
-import com.oracle.truffle.api.dsl.Cached;
-import com.oracle.truffle.api.dsl.Specialization;
-import com.oracle.truffle.api.frame.VirtualFrame;
-import com.oracle.truffle.api.library.CachedLibrary;
-
 @BuiltinMethod(
     type = "Meta",
     name = "get_annotation",
@@ -43,6 +46,15 @@ public abstract class GetAnnotationNode extends BaseNode {
     Type targetType = types.getType(target);
     ModuleScope scope = targetType.getDefinitionScope();
     Function methodFunction = scope.lookupMethodDefinition(targetType, methodName);
+    if (methodFunction == null) {
+      methodFunction = Truffle.getRuntime().iterateFrames((FrameInstance fi) -> {
+        if (fi.getCallNode() instanceof Node call && call.getRootNode() instanceof EnsoRootNode ensoRoot) {
+          return ensoRoot.getModuleScope().lookupMethodDefinition(targetType, methodName);
+        } else {
+          return null;
+        }
+      });
+    }
     if (methodFunction != null) {
       String parameterName = expectStringNode.execute(parameter);
       Annotation annotation = methodFunction.getSchema().getAnnotation(parameterName);

at least with this fix I get proper result from:

from Standard.Base import all
from Standard.Base.Data.Boolean import Boolean
from Standard.Base.Data.Text.Location import Location
from Standard.Database.Connection import Connection_Details
from Standard.Table import all
from Standard.Database import all
from Standard.AWS import all
from Standard.Visualization import Widgets

main =
    operator17 = Database.connect (SQLite location=In_Memory)
    operator18 = Table.new [ ["Y", ['a', 'b', 'c']] ]
    operator19 = operator18.select_into_database_table operator17  temporary=Boolean.True
    operator20 = Widgets.get_widget_json operator18 "select_into_database_table" [ "primary_key" ]
    operator20
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-08):

Progress: - dirty "fix" found for https://github.com/enso-org/enso/issues/6955#issuecomment-1582365442

Next Day: Bugfixing & Runtime Type Checks

Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-09):

Progress: - test cases for Meta.get_annotation: https://github.com/enso-org/enso/pull/7000

Next Day: Meta.get_annotations

GitHub
More Literal Types... · enso-org · Discussion #6945
We currently have literal support for Number and Text. We have previously spoken about adding ``` (backtick) to be identified as an expression (so we can make it explicit rather than implied). Foll...
Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for the last Saturday (2023-06-10):

Progress: - automatic conversions of arguments: https://github.com/enso-org/enso/pull/7009

Next Day: Meta.get_annotations

Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-12):

Progress: - Meta.Function proposal: https://github.com/enso-org/enso/pull/7000/commits/5d87c9c7d539cf17d068bbe2bc312f139976cdaf

Next Day: Meta.get_annotations

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new 🔴 DELAY for yesterday (2023-06-14):

Summary: There is 7 days delay in implementation of the @widgets for extension functions of Table may not be found (#6955) task. It will cause 7 days delay for the delivery of this weekly plan.

Still investigating

Delay Cause: The issue crosscuts all the layers we have IDE, LS, compiler. I am still investigating what the proper solution should be. Design and discussion with owners of other parts of the system needed.

Possible solutions: The most promising solution right now seems to be to let IdExecutionInstrument provide access to instrumented ModuleScope to the visualization. Whether that can solve the problem remains to be seen.

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-18):

Progress: - Saturday & Sunday coding

Next Day: Meta.get_annotations

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-19):

Progress: - debugging QueryData::new & request_widget

Next Day: Meta.get_annotations

JaroslavTulach commented 1 year ago

I am trying new approach to propagate the ModuleScope via State. It doesn't work, because of

The state is lost in Vector.newFromFunction which is using InteropLibrary to invoke a Function and Function.java:190 is using emptyState() instead of provided state in @ExportMessage and Execute message handler.

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-20):

Progress: - Vector.map & State: https://github.com/enso-org/enso/issues/6656#issuecomment-1598258539

Next Day: Meta.get_annotations

Google Docs
Notes - Imports/Exports discussion
Spring 2023 | 5th meeting: 3:15pm – 4pm, Join with Google Meet 4th meeting: June 13, 2023 3:15pm – 4pm, Join with Google Meet 3rd meeting: June 6, 2023 3:15pm – 4pm, Join with Google Meet 2nd meeting: May 30, 2023 ⋅ 3:15pm – 4pm, Join with Google Meet 1st meeting: May 24, 2023 ⋅ 3:15pm – 4...
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new 🔴 DELAY for yesterday (2023-06-21):

Summary: There is 7 days delay in implementation of the @widgets for extension functions of Table may not be found (#6955) task. It will cause 7 days delay for the delivery of this weekly plan.

Got it working somehow

Delay Cause: IdExecutionInstrument can provide access to instrumented ModuleScope to the visualization via State - however different solution is requested.

Possible solutions: Take a shortcut, integrate what we have right now. Or investigate better fix. Let's investigate.

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-21):

Progress: - got get_annotation working: https://github.com/enso-org/enso/pull/7078#issuecomment-1600267081

Next Day: Meta.get_annotations

enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-22):

Progress: - organisation issues: https://discord.com/channels/401396655599124480/1120986567570370651/1121456406915198976

Next Day: Meta.get_annotations

Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-23):

Progress: - breakthru in get_annotations PR: https://github.com/enso-org/enso/pull/7115

Next Day: Get reviews and QA for Meta.get_annotations and integrate

GitHub
Always call instance methods on Any by Akirathan · Pull Request #7033 · enso-org/enso
Fixes #5612 and #6473. Fixes some static method invocations on Any, like Any == Boolean, or Any.to_text. Pull Request Description Previously, static method calls on Any have not worked as expected....
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-26):

Progress: - addressing Adam's & Pawel's comments: https://github.com/enso-org/enso/pull/7115/commits/15b9639dd5445e75464147b4e7f285161ccf4e70

Next Day: Finish Meta.get_annotations and integrate

GitHub
AI Completions by kustosz · Pull Request #5910 · enso-org/enso
Pull Request Description Added a special escape hatch in the searcher to run AI completions. Screencast was posted previously on discord. You'll need to export OPENAI_API_KEY to use the feature...
enso-bot[bot] commented 1 year ago

Jaroslav Tulach reports a new STANDUP for yesterday (2023-06-27):

Progress: - Merged: https://github.com/enso-org/enso/pull/7115

Next Day: Investigate performance of node changes

Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
GitHub
Encapsulation of project with private keyword · enso-org · Discussion #7088
Motivation - encapsulation Currently, all the entities (modules, types, methods, constructors, fields) from a project/library can be imported and used, even the entities that are meant to be intern...