Developer onboarding: One line setup

wanderingstan commented 6 years ago

The steps to get started with origin.js are currently long and tedious.

See: https://github.com/OriginProtocol/platform/tree/develop/packages/origin.js#local

We need a friendlier intro for new contributors, and a system that is less error-prone.

Ideas are:

Drop truffle develop and use ganache-cli to run local blockchain. (Using build scripts by @nick as inspiration)
Use env and/or command line args to switch between local chain and network chains. (E.g. Rinkeby, mainnet)
bash script(s) for doing npm link work

Other ideas?

@joshfraser @DanielVF @tyleryasaka

nsimon commented 6 years ago

Looking into this

wanderingstan commented 6 years ago

@nsimon I think the meat of @nick 's magic is here, e.g. starting ganache: https://github.com/OriginProtocol/identity-playground/blob/master/index.js

nsimon commented 6 years ago

Thanks @wanderingstan -- nice starting point

wanderingstan commented 6 years ago

Just learned of @tyleryasaka 's code for adding seed data (Listings) in issue #93 . Ideally we'd love new users to see sample listings (again).

Any thoughts on best way to do this? Here in origin.js or over on demo dapp?

tyleryasaka commented 6 years ago

Not finished yet, but I'm about to fall asleep here and I wanted to share my progress so far.

See my in-progress PR if interested: https://github.com/OriginProtocol/platform/pull/95

wanderingstan commented 6 years ago

Nice!

If we switch from truffle develop to ganache cli, would that let us drop the shell script?

I'm not that concerned about windows users (or people with odd shells) but would rather not introduce another language dependency if possible.

tyleryasaka commented 6 years ago

@wanderingstan Yes if we just used ganache-cli we should be able to remove the bash scripts. In fact, while working on those one-line scripts I definitely did get frustrated with the limitations of Truffle. Almost convinced myself that we should just use ganache-cli directly. 😃

Agreed 100% on preferring not adding bash scripts into the mix. Definitely best not to make assumptions about OS/shell setup. The only way I can justify this approach is reasoning that the bash scripts are just a convenience; everything can be done without them, with more steps.

That PR will just be an example of one approach we can take; I think it's definitely easier to compare options with working examples.

Would be nice if someone (maybe me?) could open a PR that would demonstrate a one-line setup without Truffle.

wanderingstan commented 6 years ago

I believe we can add ganache-cli (and use it instead of truffle develop for test chain) and yet still keep truffle for migrations, contract wrapping, etc. This is similar to our original (December) setup of using test-rpc alongside truffle.

Do you think this would work, @tyleryasaka ? (Setting aside, for now, the question of whether we keep truffle for its other features.)

tyleryasaka commented 6 years ago

@wanderingstan Yeah I think that makes sense. I have thought about that as well.

The concern I have is with migrations. Are you talking about just using the migrations on test nets and main net? And then deploying contracts for local development with other scripts that use ganache-cli?

tyleryasaka commented 6 years ago

So here's the problem I see. If we use Truffle to wrap contracts, then we have to use Truffle for migrations (in all environments including local dev). Because the contract wrappers rely on the ABIs generated by Truffle during migrations. And if we have to use Truffle for migrations, then local development will always involve having to run the migrate command at some point, even if we're using ganache-cli instead of running truffle develop. (Unless there is some way to run Truffle migrations in pure JS, which would be great actually.) If we are required to run that command, we're either going to have to require the developer to type it every time they want to run the dapp locally, or we're going to have to wrap it up in some kind of script that can execute the command. (Could be bash or maybe Javascript, but it's messy either way.)

All of this to say at this point we have lost all of the benefits that we'd gain by not using Truffle. So what would the point be?

It's entirely possible that I'm missing something here, but my thinking right now is that we either use Truffle all the way or we get rid of it. I'm not seeing how the in-between thing would work.

wanderingstan commented 6 years ago

I mean just using ganache-cli as our test chain. Truffle can (I'm 99.9% sure) deploy/migrate contracts to ganache-cli, so we could continue to use truffle for all deployments (local, test, and main).

Or am I missing something where using ganache-cli precludes us from using truffle on it? I never totally dug through Nick's npm script; does it depend on being about to do deployments directly from code? Oh, like maybe you can't run command-line truffle commands from in node? (I may not have thought this through enough!)

tyleryasaka commented 6 years ago

@wanderingstan We're already using ganache-cli as our test chain. truffle develop starts ganache-cli. And if you already have it running, Truffle will actually just connect to the running instance.

So I'm not sure what you're proposing here that's not already being done.

tyleryasaka commented 6 years ago

What Nick is proposing, to the best of my understanding, is just removing the layer of abstraction over ganache-cli that Truffle provides for us, and instead interacting directly with ganache-cli. This requires us to do some more low-level work, but also gives us a lot more freedom.

tyleryasaka commented 6 years ago

And no, I tried to find a way to run node commands that are equivalent to the terminal commands. Didn't see any mention of it in the docs. (For example, again, would be nice for Truffle to provide an API for running migrations in JS. Not saying it's not possible but it's not documented AFAIK.)

wanderingstan commented 6 years ago

@tyleryasaka writes:

@wanderingstan We're already using ganache-cli as our test chain. truffle develop starts ganache-cli. And if you already have it running, Truffle will actually just connect to the running instance.

So I'm not sure what you're proposing here that's not already being done.

Sorry for not being clearer when making this issue. The main goal is simplify our current multi-step instructions for running local test chain down to a single line command. It should:

Compile and deploy contracts
Start a test chain (ideally on port 8545, per metamask)
Build the node module (including building origin.js in dist)

Bonus: (maybe these are enabled by flags?)

Perform npm link
Launch demo-dapp with your latest changes to origin.js or smart contracts. (Would it be crazy to include demo-dapp as an npm devDependency, so it's automatically included?)

Bonus bonus:

Easy way to specify that you also want to run your own IPFS node/gateway, and have origin.js use it instead of gateway.originprotocol.com

tyleryasaka commented 6 years ago

@wanderingstan Good clarification, but I wasn't confused about what your concerns are.

I was responding to this comment:

I mean just using ganache-cli as our test chain. Truffle can (I'm 99.9% sure) deploy/migrate contracts to ganache-cli, so we could continue to use truffle for all deployments (local, test, and main).

I was pointing out that we're already deploying/migrating our contracts to ganache-cli. And I was saying that I don't think this proposal here will work:

I believe we can add ganache-cli (and use it instead of truffle develop for test chain) and yet still keep truffle for migrations, contract wrapping, etc.

There's no such thing as using ganache-cli instead of truffle develop. truffle develop just starts ganache-cli and provides a console for running truffle commands. Of course we can just run the ganache-cli command directly instead of truffle develop, but that's not resolving any of our current issues with truffle.

Is that making sense?

I'm 100% on board with the the goals you listed. I think where we're at right now is that we can either:

accomplish these goals with bash scripts and continue to use Truffle, or
we can ditch Truffle and interact directly with ganache-cli in JS, and then be able to create true one-line JS commands that don't rely on bash scripts.

If someone knows how to set up proper one-line scripts in pure JS while using Truffle, please do share this information. 😃

nick commented 6 years ago

Here was my original script for starting ganache, truffle migrate, a mini bridge server (contract event watcher), local (offline) IPFS node and webpack dev server, all from one node script:

https://gist.github.com/nick/9dacab9203847dd620d2ee5ea05428f3

It basically just uses the node.js spawn command so you don't have to type everything out yourself. Accomplishes the 'one line setup' but in a hacky way :)

tyleryasaka commented 6 years ago

@nick Nice! I hadn't thought of doing it that way. Yeah a bit hacky but not nearly as hacky as the next best approach which is writing bash scripts. If we keep Truffle, I think your approach is the way to go.

For anyone following along the magic is here:

const truffleMigrate = spawn('./node_modules/.bin/truffle', ['migrate'])
    truffleMigrate.stdout.pipe(process.stdout)
    truffleMigrate.stderr.pipe(process.stderr)

    truffleMigrate.on('exit', code => {
      if (code !== 0) {
        return reject()
      }
      console.log('Truffle migrate finished OK.')
      resolve()
    })

Lets you programmatically run truffle's migrate command without a bash script.

tyleryasaka commented 6 years ago

I've got a basic working version of a one-line start script on the platform repo in pure JS. Not finished with it yet but it does work, if anyone wants to take a look. https://github.com/OriginProtocol/platform/pull/95

I'll do some more work on that PR, and then also create a PR to the demo dapp to demonstrate what our final setup could look like.

micahalcorn commented 6 years ago

Resolved by #95 ✅

OriginProtocol / origin-js

Developer onboarding: One line setup #89