High friction getting started experience

sam-goodwin commented 3 months ago

Problem

I am trying to get started with o1js and am encountering a ton of friction. I just want to test it out in NextJS 14 app and am finding that terribly difficult.

Proposal

I propose that the getting started guide should be dedicated to:

How to dropp o1js into an existing NextJS application
How to create an new idiomatic NextJS project with o1js installed and an extremely simple example app to learn from.
Once people have a solid foundation to work from, then have the tutorials walk them through adding different zk-app features.

Instead, the tutorials start from zk-app features and then neglect helping people actually build a working app.

[!Note] I believe copying what SST (and many other frameworks) do would be the safest and best approach. Just use the official NextJS setup script (or whichever other frontend framework) and then provide instructions (or a script) to setup o1js in it.

See: SST's NextJS Getting Started Guide.

Friction Log

Below is a log of the friction I have run into.

First, I ran this command, expecting to get a simple web app to play with. I suspect many people will want to start this way.

zk project test-app --ui next

It created a project that is just a folder with two nested folders and no files at the root.

contracts
ui

[!Warning] This is bizarre and leads to a bad experience. VS Code can get confused and it's not immediately obvious how to install dependencies and build the project.

[!Note] If monorepos are desired, then I'd suggest following the best practices recommended by package managed, e.g. PNPM and make use of TypeScript Project References

That said, I'd like to ask if it is strictly necessary to separate contracts into its own package? This much complexity early on is excessive and causes confusion. Can we instead just drop them in a contracts/ or src/contracts/ folder within a standard NextJS project?

E.g. I expected a simple (!) and standard TypeScript + TailWind + Next 14 application:

package.json
tsconfig.json
next.config.json
postcss.config.js
tailwind.config.js
src/
  app/
    page.tsx
    layout.tsx
  contracts/
    Add.ts

I then saw the bizarre next.config.js. Is this strictly necessary? What kind of magic is o1js doing? Imposing these kinds of configs on developers is going to lead to friction and abandonment. Can o1js be refactored into a standard ESM module?

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: false,

  webpack(config) {
    config.resolve.alias = {
      ...config.resolve.alias,
      o1js: require('path').resolve('node_modules/o1js')
    };
    config.experiments = { ...config.experiments, topLevelAwait: true };
    return config;
  },
  // To enable o1js for the web, we must set the COOP and COEP headers.
  // See here for more information: https://docs.minaprotocol.com/zkapps/how-to-write-a-zkapp-ui#enabling-coop-and-coep-headers
  async headers() {
    return [
      {
        source: '/(.*)',
        headers: [
          {
            key: 'Cross-Origin-Opener-Policy',
            value: 'same-origin',
          },
          {
            key: 'Cross-Origin-Embedder-Policy',
            value: 'require-corp',
          },
        ],
      },
    ];
  }
};

module.exports = nextConfig

harrysolovay commented 3 months ago

So much good feedback in this issue. Gonna riff on this a bit.

How to drop o1js into an existing NextJS application

Just use the official NextJS setup script (or whichever other frontend framework) and then provide instructions (or a script) to setup o1js in it.

Agreed. Devs will want to integrate o1js into existing projects. In new projects, devs will have all kinds of preferences (which package manager to use, TS config, test runner, misc.). While a bootstrap may be valuable to beginners / the docs experience, it certainly won't be zkapp-cli, but rather the CLI of a more popular app framework (create-next-app in this case).

This is bizarre and leads to a bad experience. VS Code can get confused and it's not immediately obvious how to install dependencies and build the project.

Bizarre indeed.

I then saw the bizarre next.config.js. Is this strictly necessary? What kind of magic is o1js doing?

Oof that's rough. Making this unnecessary seems like a very high priority.

The SST docs look smooth indeed. o1js docs need the same treatment: for each major runtime and framework, there should be a corresponding minimal installation guide and example repo.

Headless
- Node
- Deno
- Bun
Front-end framework workspaces
- Vite
- Next.js
- SolidJS
React Native / Expo
Library (to be published to NPM, denoland/x and jsr.io)

mitschabaude commented 3 months ago

Added some more context in the other issue https://github.com/o1-labs/o1js/issues/1575#issuecomment-2054175461

mitschabaude commented 3 months ago

It created a project that is just a folder with two nested folders and no files at the root. ... This is bizarre and leads to a bad experience. VS Code can get confused and it's not immediately obvious how to install dependencies and build the project.

That's interesting feedback to me, and I don't necessarily disagree.

I think part of the original motivation was separation of contracts and UI, and to make it easy to install contracts to npm. (Which is necessary for contract composability.)

I'm not sure if that trumps the possibly better experience of a single project with src/contracts though.

mitschabaude commented 3 months ago

I propose that the getting started guide should be dedicated to:

How to drop o1js into an existing NextJS application How to create an new idiomatic NextJS project with o1js installed and an extremely simple example app to learn from.

I agree that our docs should contain that info, but I want to push back against it a bit. In my experience, people really like the zkapp cli and most don't mind at all that a specific set of templates are imposed on them. Having a default one command getting started flow which includes project scaffolding and more zk-related commands in the same CLI was a great idea.

I'm saying this as someone who was sceptical at first, because I always set up projects from scratch myself and never liked using templates

sam-goodwin commented 3 months ago

and most don't mind at all that a specific set of templates are imposed on them

Templates are good. I do want a template but one that produces an idiomatic project.

This is a broken project configuration.

ui/
contracts/

I should be able to do this:

zk new my-project --ui next
cd my-project
pnpm install
pnpm dev # boots straight to a localhost:3000, start iterating

For a monorepo setup (ui and contracts), i'd also expect a workspace to be configured. That way in the ui project we can use ordinary package.json dependencies.

{
  "dependencies": {
    "contracts": "workspace:^"
  }
}

Then, in my JS code:

import contracts from "contracts"

Instead of references reaching outside the scope of the ui project:

import contracts from "../../../contracts/lib/..."

This means that my code is set up to look identical to the experience of using the contracts package from NPM.

Finally, we'd have a root tsconfig.json that connects these two projects so that the TypeScript language server understands their relationship and also so that tsc -b will build in the right order:

{
  "references": [
     { "path": "./ui" },
     { "path": "./contracts" }
  ]
}

And in ui/tsconfig.json:

{
  "references": [
    { "path": "../contracts" }
  ]
}

Now, tsc -b will build contracts first and then ui and the language server will connect everything nicely.

pnpm install is made to work for all packages with a pnpm-workspace.yaml at the root (see https://pnpm.io/pnpm-workspace_yaml):

packages:
  - ./ui
  - ./contracts

Other tools like NPM and Yarn have their own mechanism.

harrysolovay commented 3 months ago

In my experience, people really like the zkapp cli and most don't mind at all that a specific set of templates are imposed on them

Not Minding the Templates vs. Not Needing them

Should we aim to have setup be so easy, that one can (1) npm/yarn/pnpm install o1js, (2) create a contract.ts file anywhere in their project, and (3) get cruising? I'm still not sure what value the bootstrap/template offers. There are many powerful bootstrapping tools for specific app frameworks. I think we can point to those from the getting started documentation. However, recommending (let alone maintaining) an o1js-specific bootstrap seems unnecessary.

Alternative Approach

We could implement an init script within o1js.

npx o1js

This script could possibly...

Install o1js with the developer's package manager of choice.
Scaffold a contract file and perhaps.
A script for spawning the lightnet (if that's the desired approach).
Add a contract deploy script.

sam-goodwin commented 3 months ago

As a developer, I would like the freedom to make these decisions on my own. Every decision the zk CLI makes is one that could be wrong (as is the case here) and something that needs to be maintained. I think push that responsibility to the app frameworks and have o1js be "just another dependency" that I install and configure. Or at least prioritize making that work seamlessly and documenting it as one of the ways of getting started.

In summary, I think we should put together a plan to have a super lean getting started guide that "just works" in a standard NextJS 14 app with minimal config or "weirdness" like imports inside useEffect. The header config in next seems unavoidable, but that's all we should require (hopefully).

In my case, I really want to explore what's possible with SST and o1js. If I can get https://github.com/o1-labs/o1js/issues/1575 unblocked I'd happily take the lead on creating a sample project with what I think is a good setup. I can create 1 for a simple nextjs app and one for monorepo with CI/CD to NPM.

mitschabaude commented 3 months ago

As a developer, I would like the freedom to make these decisions on my own.

Me too

Or at least prioritize making that work seamlessly and documenting it as one of the ways of getting started.

+1

I'm still not sure what value the bootstrap/template offers. ... Alternative Approach We could implement an init script within o1js.

That alternative doesn't give developers an example UI with an example integration of a contract though. We get many beginner-level TS devs who are trying this out mainly for their interest in zkApps, and that kind of code to get started with has proven really important to them.

Tbh, the direction we are looking at based on feedback is to provide more, not less, guidance in starting a project. The zkapp CLI will have a dropdown of multiple example projects which are a bit more meaningful and complex than the Add contract, and that demonstrate how to implement important use cases.

Every decision the zk CLI makes is one that could be wrong (as is the case here) and something that needs to be maintained.

Every decision that is not made by a bootstrap is one that (potentially beginner) devs have to make.

Maybe the docs should describe two ways of getting started as equally valid options:

Invoke zk project - suitable for everyone, opinionated
Use o1js in your own project setup - suitable for intermediate TS developers, flexible
- here's how to install o1js in your project: npm i o1js
- here's the extra config you'll need for XYZ framework: ...
- here's the single command to initialize the zk CLI in your project: zk config
- here are examples of contracts integrated in XYZ UI framework: ...

mitschabaude commented 3 months ago

Btw, this discussion is gold, and I really appreciate both of your inputs @harrysolovay @sam-goodwin

harrysolovay commented 3 months ago

That alternative doesn't give developers an example UI with an example integration of a contract though.

Examples are indeed important. We should have many example projects for each runtime/framework/lib distribution. However, I'd advise against baking example boilerplate and project scaffolding decisions into the beginner docs (let alone docs geared towards more sophisticated devs, who will likely disagree with the current workspace templates).

the direction we are looking at based on feedback is to provide more, not less, guidance in starting a project.

Then let's do exactly this, but let's not take on zkapp-cli maintenance overhead with little to gain. Ie. let's create runtime/framework-specific guides that showcase their ecosystem-specific project setup. The o1js integration step should––for most of these guides––be the same: npm/pnpm/yarn install o1js.

ymekuria commented 3 months ago

Thanks for all the feedback @sam-goodwin and @harrysolovay.

First, I ran this command, expecting to get a simple web app to play with. I suspect many people will want to start this way.

This is on our roadmap currently. We want to give devs an example to expand on.

I should be able to do this:
zk new my-project --ui next
cd my-project
pnpm install
pnpm dev # boots straight to a localhost:3000, start iterating
For a monorepo setup (ui and contracts), i'd also expect a workspace to be configured. That way in the ui project we can use ordinary package.json dependencies.

I agree that the there should be workspace configuration at the root of the project. This was always the plan in future iterations. I think it is time to prioritize this.

I'd advise against baking example boilerplate and project scaffolding decisions into the beginner docs

This is a tough balance. We baked in decisions into the example boilerplate that were necessary at the time for o1js to work in a given framework/environment. I would like to to move away from an opinionated approach and delegate more and more to the app frameworks as changes in o1js and other tooling allow. We are in a position to start doing this.

In summary, I think we should put together a plan to have a super lean getting started guide that "just works" in a standard NextJS 14 app with minimal config or "weirdness" like imports inside useEffect. The header config in next seems unavoidable, but that's all we should require (hopefully).

I would love for this to happen. We support generating projects with many UI frameworks but Next is the most widely used by o1js developers.

ymekuria commented 3 months ago

Maybe the docs should describe two ways of getting started as equally valid options:

Invoke zk project - suitable for everyone, opinionated

Use o1js in your own project setup - suitable for intermediate TS developers, flexible

here's how to install o1js in your project: npm i o1js

here's the extra config you'll need for XYZ framework: ...

here's the single command to initialize the zk CLI in your project: zk config

here are examples of contracts integrated in XYZ UI framework: ...

@mitschabaude we should seriously consider this approach in our docs.

o1-labs / o1js