search

everywhere-computer / roadmap

The roadmap for the Everywhere Computer
https://everywhere.computer
3 stars 0 forks source link

Writing Custom Functions in Rust #5

Open depatchedmode opened 11 months ago

depatchedmode commented 11 months ago

Feature Description

As we solicit feedback from folks, they need to be able to write custom functions in order to:

  1. Evaluate and understand the capabilities of the Everywhere Computer
  2. Imagine clear use cases within their business

Use Cases

Signals of Success

More function diversity in each run, and overall.

Current Workarounds (optional)

Technically possible write now, but requires making changes to core code. See Philipp's tear down of how it's done presently: https://www.loom.com/share/19c4651b5ae6419ea091cd6c2fcb049f?sid=8739f44e-e88e-4a29-9446-5eb9a39d59aa

Summary of current process is:

  1. in your local Homestar repo:
    • write a function in homestar-functions/src/lib.rs
    • add type definitions to 'homestar-functions/wit/host.wit`
    • compile by running cargo build -p homestar-functions-test --target wasm32-unkown-unknown --profile release-wasm-fn
    • now turn it into a Wasm component: wasm-tools component new ... -o functions.wasm
    • ingest into local IPFS node
  2. in your local Control Panel repo:
    • launch the control panel npm run dev
    • make function available in dropdown. update the VITE_WORKFLOW_RESOURCE with the newly created module's CID in .env
    • add the function signature to the const FUNCTION_NAMES array in NewFunctionNode.svelte
    • in function-template.ts:
    • rewrite generateFunction as so: 
      // philipp's snippet goes here
    • specify default parameters in the const DEFAULT_PARAMS object
    • in graph.ts add function definitions to const FUNCTION_PARAMS

Extreme MVP is to document the workaround of manually writing and ingesting wasm modules. Consider this a map of the work to be done in pursuit of a cleaner DX for authoring -> publishing in your local registry of functions.

Evidence of Value (Optional)

Table stakes! You can't have a computer if there's nothing to compute.

Tasks & Jobs Stories

Additional Context (Optional)

Boris is going to start by dog fooding the process.

matheus23 commented 11 months ago

Haha this so very closely re-created my "script" I wrote for the video, so here's my script for reference:

Custom Homestar Function Script

    fn unsafe_encrypt(data: Vec<u8>, key: u32) -> Vec<u8> {
        let loaded = image::load_from_memory(&data).unwrap();
        let stream_bytes = loaded.width() * loaded.height() * 4;
        let mut output = blake3::Hasher::new_derive_key(
            "poor man's stream cipher. Please use a proper AEAD with a proper password kdf instead.",
        )
        .update(&key.to_le_bytes())
        .finalize_xof();

        let mut key_stream = vec![0u8; stream_bytes as usize];
        output.fill(&mut key_stream);

        let mut rgba_image = loaded.into_rgba8();

        for (i, pixel) in rgba_image.pixels_mut().enumerate() {
            let offset = i * 4;
            pixel.0[0] ^= key_stream[offset];
            pixel.0[1] ^= key_stream[offset + 1];
            pixel.0[2] ^= key_stream[offset + 2];
            pixel.0[3] ^= key_stream[offset + 3];
        }

        let mut buffer: Vec<u8> = Vec::new();
        rgba_image
            .write_to(&mut Cursor::new(&mut buffer), image::ImageOutputFormat::Png)
            .unwrap();
        buffer
    }

    fn unsafe_encrypt_base64(data: String, key: u32) -> Vec<u8> {
        let base64_encoded_png = data.replace("data:image/png;base64,", "");
        let decoded = general_purpose::STANDARD
            .decode(base64_encoded_png)
            .unwrap();
        Self::unsafe_encrypt(decoded, key)
    }
  const homestarFunctionName = `${functionName}${base64 ? '-base64' : ''}`

  const inv = {
    name: label,
    resource: import.meta.env.VITE_WORKFLOW_RESOURCE,
    ...(needs ? { needs } : {}),
    args: Object.values({
      data,
      ...(args ? args : DEFAULT_PARAMS[functionName])
    }),
    func: homestarFunctionName
  };

  return HSWorkflow.invocation(inv)
depatchedmode commented 10 months ago

Discussion to start: where should we deposit function/module configuration?

My proposal would be to rename the getting-started repo to default-workspace and have both homestar and the control panel be pointed to config/definitions that life there. That can be the source for both custom local functions, and things pulled down from any other registries.

Issue where I propose the rename is here https://github.com/everywhere-computer/getting-started/issues/3

Thoughts?

bmann commented 10 months ago

This is roughly "here's what I want to run on my local node / network of nodes", yes?

So this would in part be booting up your homestar node and pointing at this set of configs, expanded to include "on startup, always cache / load these functions"

So, it would either be a list of functions by content address or their equivalent DNS link, or local file paths, which then get uploaded / cached by the local node.

depatchedmode commented 10 months ago

Yep, exactly. And when we have a CLI that boots up homestar, control panel, etc it'd know about that workspace and pass along important bits from it to the initialization of any other processes.