Local Workflow

[schickling]

@timsuchanek commented on Sun Oct 01 2017

Local Development

We will enable a local development workflow using docker under the hood. The topic this proposal is shedding light on is how that exactly looks like in the CLI.

Commands

gc local up: Pull & Start a local Graphcool instance gc local start: Start an already intialized local Graphcool instance gc local pull: Pull the latest Graphcool version gc local stop: Stop the currently running local Graphcool instance gc local reset: Reset the local Project

.graphcoolrc

The .graphcoolrc would get a new "kind" of environment, called local:

default: dev
environments:
  prod: cj7zy085306i50193k7ox16z4
  dev: local

~/.graphcool

{
  "token": "...",
  "local_token": "...",
  "local_host": "localhost:60000"
}

So additionaly to token, there will also be local_token and local_host. These values all can be overwritten by environment variables:

export GRAPHCOOL_TOKEN=...
export GRAPHCOOL_LOCAL_TOKEN=...
export GRAPHCOOL_LOCAL_HOST=...

There can be one local environment at the same time. When multiple local environments are needed, that orchestration can be achieved with managing the environment variables.

Update:

Add gc local restart command.

Update 2:

As @kbrandwijk suggested, it makes more sense for the local docker environment information to live in the .graphcoolrc file and not the .graphcool file as .graphcool is global and the local docker env information is project related. With this change the files look as follows:

.graphcoolrc

default: dev
environments:
  prod: cj7zy085306i50193k7ox16z4
  dev:
    token: "..."
    host: "localhost:60000"
    projectId: c27zs08d30ai52683k7mx26a5 # temporary, will be removed soon

~/.graphcool

{
  "token": "..."
}

---

@schickling commented on Sun Oct 01 2017

In general the proposal looks good to me. 👍 Here are a few questions:

gc local reset: Reset the local Project

What does this do exactly (step-by-step)?

export GRAPHCOOL_LOCAL_HOST=...

What does this refer to? What's the purpose of this hostname? What's the underlying network protocol for the intended usage?

---

@kbrandwijk commented on Sun Oct 01 2017

• I feel there might be something counter intuitive about the combination of environment definition and command. If the environment definition is: dev: local then I would expect gc up --env:dev or something etc. Unless the gc local commands get a --env parameter, then it also makes sense.

• Would you be able to create more than one local enviroment?

• Would this local docker environment play a role in automated testing?

• How does deploying to this local environment work? Would you be able to do gc deploy --env dev with dev: local? Would the machine need to be up for that? Or would the docker container contain a reference to the local project folder, so explicitly deploying to your local environment would not be necessary?

• Would you be able to deploy the docker container to a hosting provider, and would that basically be the self-hosted option?

• ~/.graphcool is not project specific. I don't think that these project-specific local environment settings belong there. They should probably go into the project directory.

---

@timsuchanek commented on Mon Oct 02 2017

What does this do exactly (step-by-step)?

We currently have 2 mutations for this:

resetProjectData
resetProjectSchema

They would both be called. We could have 2 more fine-grained versions of the command with --data and --schema.

What does this refer to? What's the purpose of this hostname? What's the underlying network protocol for the intended usage?

This is the deployment endpoint that the CLI needs in order to deploy the project. Currently all endpoints are available on the :60000, we should separate them.

---

@timsuchanek commented on Mon Oct 02 2017

I feel there might be something counter intuitive about the combination of environment definition and command

The idea is, that you can have multiple local environments. So I agree, adding --env to the gc up command is then necessary. We can think about a default here like we do with the current init command.

Would this local docker environment play a role in automated testing?

That's an interesting question, I think it definitely could.

How does deploying to this local environment work?

Yes, the machine needs to be up for that. The CLI should handle it gracefully when deploy is executed without a running docker instance.

Would you be able to deploy the docker container to a hosting provider, and would that basically be the self-hosted option?

Yes, as people can deploy their docker stuff everywhere, this is the first minimal version of the self-hosted option. However, it's built for ease-of-use and not scalability.

~/.graphcool is not project specific. I don't think that these project-specific local environment settings belong there. They should probably go into the project directory.

That's a good point. We have to think about a better solution there.

---

@FredyC commented on Tue Oct 03 2017

I like this, I just want to add here that it's rather frustrating not even knowing the environment where functions are executed and what language features are supported. Especially that obscure "use latest" flag which is not really explained anywhere what it does. Is it a specific NodeJS version that runs that code?

---

@timsuchanek commented on Fri Oct 06 2017

Thanks for bringing that up @FredyC We definitely will make the function workflow much nicer soon :)

---

@timsuchanek commented on Fri Oct 06 2017

Updates

After having the first version implemented, we had some learnings that moved us to the following changes:

Updates for ~/.graphcool

Having the local docker environment information in the .graphcoolrc file mixes two concerns: The concern of configuring docker locally and the concern of where a project is deployed. Because if this, we decided to put this information into the .graphcool file (which is now a YAML file):

token: secure-platform-token
targets:
  dev-1:
    description: This is your local docker dev environment
    host: http://localhost:60000
    token: my-docker-token
  do-2:
    host: https://12.235.235.124:60000/
    token: my-remote-token

That means the .graphcool file will turn into yaml as well. This makes it more consistent with the .graphcoolrc and graphcool.yml.

Renaming for gc deploy

We will rename the args for gc deploy. Instead of providing a project with -p, we will rename it to --target. This makes it easier to point either to the local docker version or to the hosted version of Graphcool. With gc deploy --target dev-1 the project will be deployed to the local docker instance. With gc deploy --target cj8fyu5wl0000jnom2muqv9sl you can point to a normal project id.

Making the gc deploy command interactive

In order to make choosing the deployment target easy, the gc deploy command will add an interactive dialog asking where to deploy. When enough args are provided, this dialog will be bypassed.

Removing of the gc env command

To make things easier, we will remove the gc env command as it is providing more confusion than any value. If you want to edit the local environments setting, just edit the .graphcoolrc directly.

New commands for gc local

The following commands will be added:

• gc local eject: Puts the docker-compose.yml and .envrc in the current folder. From here on you are on your own and can do customizations if needed.
• gc local ps INSTANCE_NAME: List all running containers of an instance
• gc local ls: List all local instances.

---

@schickling commented on Fri Oct 06 2017

Update: ~/.graphcool and .graphcoolrc file will be merged into a global & local .graphcoolrc file.

Root fields

interface Root {
  platformToken?: string
  defaultTarget?: string
  targets: { [name: string]: Target }
}

type Target = ServiceId | DockerTarget | PlatformTarget
type ServiceId = string

interface DockerTarget {
  description?: string
  host: string
  token: string
}

interface PlatformTarget {
  description?: string
  serviceId: ServiceId
}

Default behaviour

gc auth

Writes platformToken to global .graphcoolrc file

gc local up --target/-t

Sets up a new local docker service (or docker-compose ups the existing one) with the target name -t.

Warnings

Warning when override global property and used (e.g. platformToken or concrete target)

Aliasing

A target name (e.g. digitalocean or prod - see below) can reference other already existing targets. The default target has a special role here since it will be used when running gc deploy and specified in the .graphcoolrc file.

Example .graphcoolrc file

Global

platformToken: secure-platform-token # retrieved via `gc auth`

targets:
  dev:
    description: This is your local docker dev environment
    host: http://localhost:60000
    token: my-docker-token
  digitalocean:
    host: https://12.235.235.124:60000/
    token: my-remote-token

Local

targets:
  default: dev # `default` is a magic key that can reference other targets
  prod: cj8g62nxg0000um3t0z64ls08

Usage

# deploy to default target
gc deploy

# deploy to prod
gc deploy --target prod

# deploy to digitalocean via shortcut argument
gc deploy -t digitalocean

---

@marktani commented on Fri Oct 06 2017

What's the difference between platformToken and token?

---

@schickling commented on Fri Oct 06 2017

What's the difference between platformToken and token?

token will be renamed to platformToken to avoid confusion with the token needed for "Docker Targets".

---

@kbrandwijk commented on Sat Oct 07 2017

Regarding removal of the graphcool env command. The only part of that command I find useful is setting the default environment, because it is used in many commands (including graphcool playground and graphcool console). For graphcool deploy, I think the environment/stage should always be explicitly specified, but for other commands, a default environment is still very useful, and there should be an easy way to set it, other than manually editing the environment file.

---

@FredyC commented on Sat Oct 07 2017

I want to get one thing straight here. You are using gc everywhere in here. Does it mean that current cli command graphcool will be replaced by gc or are us using it like alias here for brevity?

---

@kbrandwijk commented on Sat Oct 07 2017

You can already use both graphcool and gc, ever since the first cli version.

---

@FredyC commented on Sat Oct 07 2017

Interesting, where is that actually mentioned? Should be in official beta docs ... https://docs-next.graph.cool/reference/graphcool-cli/commands-aiteerae6l

Although it feels strange to have two identical commands, I guess it's not a big deal. I am wrapping all important tasks in package.json scripts anyway, so I don't care about saving couple of extra characters :)

---

@kbrandwijk commented on Sat Oct 07 2017

A lot of cli commands have a 'shorthand' version, like wt is shorthand for wt-cli for webtasks, serverless has serverless, slss and sls etc.

---

@schickling commented on Fri Oct 13 2017

Yet another update: Given the multi-service cluster format (see #748) we're adjusting the .graphcoolrc file and gc deploy command to work with clusters.

.graphcoolrc root fields

interface Root {
  platformToken?: string
  clusters?: { [name: string]: Cluster }
  // example target: `shared-eu-west-1/cj8g62nxg0000um3t0z64ls08`
  targets?: { [name: string]: string }
}

interface Cluster {
  host: string
  token: string
}

Example .graphcoolrc file

Global

platformToken: secure-platform-token # retrieved via `gc auth`

clusters:
  default: local
  # This is your local docker dev cluster
  local:
    host: http://localhost:60000
    token: my-docker-token
  digitalocean:
    host: https://12.235.235.124:60000/
    token: my-remote-token

Local

targets:
  default: dev # `default` is a magic key that can reference other targets
  dev: dev/cj8g62nxg0000um3t0z64ls08
  prod: shared-eu-west-1/cj8g62nxg0000um3t0z64ls08

Usage

# deploy to default target
gc deploy

# deploy to prod
gc deploy --target prod

gc deploy -t dev/cj8irw8v20000jm3t7fvalhfg
gc deploy -t shared-eu-west-1/cj8irw8v20000jm3t7fvalhfg

gc deploy --new-service my-service name --new-service-cluster shared-eu-west-1

Open questions

• [x] How to automatically/manually migrate existing .graphcoolrc and ~/.graphcool files.
• [ ] How and when is a default cluster selected? Is this based on the ping? @timsuchanek

---

@kbrandwijk commented on Mon Oct 09 2017

I completely lost it with regards to the 'services' and 'clusters' concepts. I've read #748 and #730, but I'm missing a substantial amount of pieces of the puzzle here.

---

@timsuchanek commented on Fri Oct 13 2017

This proposal is now implemented and released with version v0.7.0. You can install it with npm install -g graphcool@next. @kbrandwijk we will release more docs around this in the coming days!

---

@zixia commented on Tue Mar 13 2018

Is gc local reset not supported yet?

$ gc --version
graphcool-framework/0.11.5 (linux-x64) node-v9.7.1

$ gc local reset
 ▸    local reset is not a graphcool-framework command.
 ▸    Perhaps you meant local restart
 ▸    Run graphcool-framework help local for a list of available commands.

Get in touch if you need help: https://www.graph.cool/forum
To get more detailed output, run $ export DEBUG="*"