Access clients of user-defined resources via simulator API

Chriscbr commented 10 months ago

Feature Spec

Today, the simulator API only allows you to access a client for a resource if the resource's class was implemented in the SDK:

import { testing } from '@winglang/sdk';

const sim = new testing.Simulator({ simfile: "hello.wsim" });
const fn = sim.getResource("root/Default/cloud.Function");
const response = await fn.invoke("hello!");

If you try calling getResource with a resource path corresponding to a class written in Wing, the simulator API throws an error.

We would like to lift this restriction so that clients of all resources in the app can be accessed.

Use Cases

For Wing Console features

Implementation Notes

No response

Component

Compiler, SDK

Community Notes

Please vote by adding a 👍 reaction to the issue to help us prioritize.
If you are interested to work on this issue, please leave a comment.
If this issue is labeled needs-discussion, it means the spec has not been finalized yet. Please reach out on the #dev channel in the Wing Slack.

ainvoner commented 10 months ago

This feature also enables us to meet our basic requirements here: https://github.com/winglang/wing/issues/4493

In short: If every user defined resource will also have a client which will enable us to call every public inflight method defined in this resource - it is exactly what we need. It means that pub inflight visualModel(): VisualModel will be accessible from the console.

Chriscbr commented 10 months ago

It looks like this isn't as easy as I first thought. My initial approach was to codegen toSimulator() for every preflight class, so that every resource in a Wing app gets listed insimulator.json when wing compile -t sim is run. That part is straightforward.

Now, how does the simulator know how to run a user defined resource? One idea is to have the simulator config point to the JavaScript code for that resource's inflight client. For example, if I have an app with a user-defined Store class and SDK-defined Bucket class, I can store a new field called simulationPath that refers to JS code for the client:

    {
      "type": "@winglang/sdk.cloud.Bucket",
      "path": "root/Default/Store/cloud.Bucket",
      "simulationPath": "/Users/chrisr/dev/wing1/libs/wingsdk/lib/target-sim/bucket.inflight.js",
      "props": {
        "public": false,
        "initialObjects": {},
        "topics": {}
      },
      "attrs": {}
    },
    {
      "type": "Store",
      "simulationPath": "/Users/chrisr/dev/wing-test/target/example.main.wsim/.wing/store_c8ce3f27.js",
      "path": "root/Default/Store",
      "props": {},
      "attrs": {}
    },

When the simulator is creating a client for each resource, I thought to try to require() this code. The issue is the JS code produced by the compiler isn't in a form that's easy to run. For example, when you write this in Wing:

class Store {
  var name: str;
  bucket: cloud.Bucket;
  init(name: str) {
    this.name = name;
    this.bucket = new cloud.Bucket();
  }
  setName(name: str) {
    this.name = name;
  }
  pub inflight save() {
    this.bucket.put("store-${this.name}", this.name);
  }
}

it produces this JS:

"use strict";
module.exports = function({  }) {
  class Store {
    constructor({ $this_bucket, $this_name }) {
      this.$this_bucket = $this_bucket;
      this.$this_name = $this_name;
    }
    async save() {
      (await this.$this_bucket.put(String.raw({ raw: ["store-", ""] }, this.$this_name), this.$this_name));
    }
  }
  return Store;
}

But this JS has several variable placeholders that need to be filled in. This code is a function that needs several other inputs.

If we call toInflight() on the class, we get code that performs the instantiation of this Store client class, providing all of the necessary inputs like $this_bucket and $this_name`:

(await (async () => {
  const StoreClient = require("./inflight.Store-1.js")({});
  const client = new StoreClient({
    $this_bucket: (function (env) {
      let handle = process.env[env];
      if (!handle) {
        throw new Error("Missing environment variable: " + env);
      }
      return $simulator.findInstance(handle);
    })("BUCKET_HANDLE_7248178e"),
    $this_name: "test",
  });
  if (client.$inflight_init) { await client.$inflight_init(); }
  return client;
})())

But this code assumes the client is being called from an inflight host that has all of its environment variables set! The simulator itself isn't exactly an inflight host though, so when I tried testing this, I just got "Missing environment variable: BUCKET_HANDLE_34274719" errors.

To work around this I might need to: a) find some other solution where we generate code that doesn't require these environment variables, or b) treat the simulator as some kind of special inflight host, or c) create some fake / mock host that implicitly calls onLift to every other resource on the app, just for the purpose of the simulator

Chriscbr commented 10 months ago

One more idea I had was to create a cloud.Function corresponding to each user-defined resource and make invoking the function proxy to calling an arbitrary method on the other resource - but this feels pretty hacky (and it will inflate compile times massively)

eladb commented 10 months ago

I am wondering if the culprit is that the Console is calling inflight APIs directly instead of a network interface.

Considering that soon enough we will need the Console to be able to interact with production Wing applications deployed to the cloud, the Console service won't even have permissions to execute inflight client code (eg call s3.PutObject).

I am wondering if the direction should be to use the simulator remoting API that we just introduced, and then we can evolve it to support production as well.

Maybe we can even get rid of some of the resource specific code we have in the console backend in this case and give the front end direct access to the app's api.

winglang / wing