Custom resources / compile time hooks

[sam-goodwin]

We should add a way to run eventual code, like workflows, during deployment/compile time.

export const updatePostgresSchema = resource({
  input: async () => ({
    schema: await fs.readFile("./schema.sql")
  })
}, async (oldProps, newProps) => {
  // run arbitrary code in a lambda function
  await myUpdateWorkflow(..)
});

The input of the custom resource is determined at compile time. This allows for static files checked in to code to trigger workflows during cloudformation deployment time.

[sam-goodwin]

I modeled it as a resource since it will be implemented as a custom resource. Whatever data it returns should be available for other lambda functions to access

[thantos]

the file system call will run during any handler that doesn't tree shake this out. Should the input be a callback?

[sam-goodwin]

It should be possible to pass credentials securely to a resource, this wil be very helpful for onboarding with external services without CDK silliness

[sam-goodwin]

the file system call will run during any handler that doesn't tree shake this out. Should the input be a callback?

Good point, let me update that.

[sam-goodwin]

It should be possible to pass credentials securely to a resource, this wil be very helpful for onboarding with external services without CDK silliness

export const actualElasticSearch = resource("my-elastic", {
  input: () => {
    // TODO: how to pass credentials reliably here
  }
}, async (elasticAPIKey) => {
  // create the elastic cluster, wait for stabilization, etc.
});

[thantos]

We'll make sure the callback runs during import in infer, but not at runtime. At runtime we can inject the stored value back in.

Also should have a name. We'll put it's data into the spec.json under the name with the config.

And a lambda function/CDK custom resource will be created with it on the CDK side.

[sam-goodwin]

Also should have a name.

Should it also have a resource type? We can use that to namespace things in the Service. Enables abstractions (like constructs) developed directly in eventual.

export function myCustomResource<Name extends string>(name: Name, options) {
  return resource(name, {
    type: "MyCustomResource",
    ...options
  });
}

[sam-goodwin]

And a lambda function/CDK custom resource will be created with it on the CDK side.

Perhaps one Lambda Function per resource type instead of per resource instance?

[thantos]

configurable? I assume we'd want to be able to model the per resource lambdas we have now.

[sam-goodwin]

Playing with a twilio example. Here I introduce the CRUD lifecycle and as a new primitive. It can call workflows, events, etc. - whatever it needs to do its job. In this example i'm just calling the APIs in flight.

Everything is type-safe and inferred.

The final init function call takes the Resource's attributes (their outputs) and instantiates a client. This client is then available for use within functions.

We can pass attributes from one resource to another - although this will have edges. We can use a Proxy to intercept references to these values but not sure what to do if they're used before they're available during runtime.

import twilio from "twilio";
import { resource } from "@eventual/core";
import type { AddressListInstanceCreateOptions } from "twilio/lib/rest/api/v2010/account/address";
import type { IncomingPhoneNumberListInstanceCreateOptions } from "twilio/lib/rest/api/v2010/account/incomingPhoneNumber";

const client = twilio(
  process.env.TWILIO_ACCOUNT_SID,
  process.env.TWILIO_AUTH_TOKEN
);

export const Address = resource("twilio.Address", {
  async create(input: AddressListInstanceCreateOptions) {
    return {
      sid: (await client.addresses.create(input)).sid,
    };
  },
  async update({ newResourceProperties, attributes: address }) {
    const updatedAddress = await client
      .addresses(address.sid)
      .update(newResourceProperties);
    return {
      sid: updatedAddress.sid,
    };
  },
  async delete({ attributes: address }) {
    await client.addresses(address.sid).remove();
  },
  async init(address) {
    return client.addresses.get(address.sid).fetch();
  },
});

export const PhoneNumber = resource("twilio.PhoneNumber", {
  async create(input: IncomingPhoneNumberListInstanceCreateOptions) {
    return {
      sid: (await client.incomingPhoneNumbers.create(input)).sid,
    };
  },
  async update({ newResourceProperties, attributes: address }) {
    const updatedAddress = await client
      .incomingPhoneNumbers(address.sid)
      .update(newResourceProperties);
    return {
      sid: updatedAddress.sid,
    };
  },
  async delete({ attributes: address }) {
    await client.incomingPhoneNumbers(address.sid).remove();
  },
  async init(address) {
    return client.incomingPhoneNumbers.get(address.sid).fetch();
  },
});

export const samGoodwin = Address("Sam Goodwin", {
  customerName: "Sam Goodwin",
  city: "Seattle",
  region: "Seattle",
  postalCode: "(redacted)",
  isoCountry: "US",
  street: "(redacted)",
});

export const samGoodwinCell = PhoneNumber("Sam Goodwin Cell", {
  addressSid: samGoodwin.attributes.sid,
});