every canvas

Connect your canvas to the open web.

draw everything

Context

Sawyer Hood built an app that uses tldraw and the gpt-4-vision api to generate html based on a wireframe you draw.

tldraw forked that app, then I forked tldraw.

I had originally adapted the prompt and preview shape to create near.social widgets and embed them via near-bos-webcomponent. I was able to create valid, stateful widget code:

A demo of the app

It worked by taking the current canvas SVG, converting it to a PNG, and then sending that png to gpt-4-vision with instructions to return a single jsx file with styled components. The JSX code was then passed to a near-bos-webcomponent so it could be rendered via the Near Social VM](https://github.com/NearSocial/VM), but now I'm thinking there should be a widget with an iframe inside it -- then this iframe holds the html that gets generated from the canvas. Possibly sharing some resemblance with bos-web-engine.

Overview

This repository is built using the NEAR open web stack.

Right now, it is a Near Social gateway and bos-workspace combined in one. This gateway provides tldraw as a custom element to the Near Social VM. This enables us to use <Tldraw /> and it's editor via:

<Canvas
  initialSnapshot={data || {}}
  persistance={path}
  autoFocus={true}
/>

in a widget you can find in apps/canvas/widget/index. Widgets are reusable composable components for Near Social. You can view and create widgets at near.social.

This means the draw.everything.dev gateway provides you with the ability to hack your own components using the <Canvas> element.

This structure will evolve for a better development experience.

Setup & Development

Initialize repo:

yarn

Start development version:

yarn dev

This will start both the gateway at localhost:3000 and serve the widgets from http://127.0.0.1:4040.

Once these are running, copy this server address (http://127.0.0.1:4040), go to localhost:3000/flags (Chrome recommended), and paste it as the BOS Loader Url.

Save and you should see the following banner:

Next Steps

[x] Merge PR to near-bos-webcomponent and replace bundles from CDN
[x] Ability to publish JSX widget to the social-db
[ ] Load in existing widgets and make modifications to them
[x] Option to provide your own API key (near-openai)
[ ] Feature to provide custom prompts and option to swap chat gpt model - IN DEV
[ ] Store JSON on IPFS and create hyperfiles for saving canvas and communicating with Obsidian.
[ ] Deploy to web4

Contributing

Pull requests are welcome! Happy hacking!

If you'd like to help improve the developer experience, help out here.

near-everything / canvas