We will enable a local development workflow using docker under the hood.
The topic this proposal is shedding light on is how that exactly looks like in the CLI.
Commands
gc local up: Pull & Start a local Graphcool instance
gc local start: Start an already intialized local Graphcool instance
gc local pull: Pull the latest Graphcool version
gc local stop: Stop the currently running local Graphcool instance
gc local reset: Reset the local Project
.graphcoolrc
The .graphcoolrc would get a new "kind" of environment, called local:
default: dev
environments:
prod: cj7zy085306i50193k7ox16z4
dev: local
There can be one local environment at the same time. When multiple local environments are needed, that orchestration can be achieved with managing the environment variables.
Update:
Add gc local restart command.
Update 2:
As @kbrandwijk suggested, it makes more sense for the local docker environment information to live in the .graphcoolrc file and not the .graphcool file as .graphcool is global and the local docker env information is project related.
With this change the files look as follows:
.graphcoolrc
default: dev
environments:
prod: cj7zy085306i50193k7ox16z4
dev:
token: "..."
host: "localhost:60000"
projectId: c27zs08d30ai52683k7mx26a5 # temporary, will be removed soon
I feel there might be something counter intuitive about the combination of environment definition and command. If the environment definition is: dev: local then I would expect gc up --env:dev or something etc. Unless the gc local commands get a --env parameter, then it also makes sense.
Would you be able to create more than one local enviroment?
Would this local docker environment play a role in automated testing?
How does deploying to this local environment work? Would you be able to do gc deploy --env dev with dev: local? Would the machine need to be up for that? Or would the docker container contain a reference to the local project folder, so explicitly deploying to your local environment would not be necessary?
Would you be able to deploy the docker container to a hosting provider, and would that basically be the self-hosted option?
~/.graphcool is not project specific. I don't think that these project-specific local environment settings belong there. They should probably go into the project directory.
They would both be called. We could have 2 more fine-grained versions of the command with --data and --schema.
What does this refer to? What's the purpose of this hostname? What's the underlying network protocol for the intended usage?
This is the deployment endpoint that the CLI needs in order to deploy the project. Currently all endpoints are available on the :60000, we should separate them.
I feel there might be something counter intuitive about the combination of environment definition and command
The idea is, that you can have multiple local environments. So I agree, adding --env to the gc up command is then necessary. We can think about a default here like we do with the current init command.
Would this local docker environment play a role in automated testing?
That's an interesting question, I think it definitely could.
How does deploying to this local environment work?
Yes, the machine needs to be up for that. The CLI should handle it gracefully when deploy is executed without a running docker instance.
Would you be able to deploy the docker container to a hosting provider, and would that basically be the self-hosted option?
Yes, as people can deploy their docker stuff everywhere, this is the first minimal version of the self-hosted option. However, it's built for ease-of-use and not scalability.
~/.graphcool is not project specific. I don't think that these project-specific local environment settings belong there. They should probably go into the project directory.
That's a good point. We have to think about a better solution there.
I like this, I just want to add here that it's rather frustrating not even knowing the environment where functions are executed and what language features are supported. Especially that obscure "use latest" flag which is not really explained anywhere what it does. Is it a specific NodeJS version that runs that code?
After having the first version implemented, we had some learnings that moved us to the following changes:
Updates for ~/.graphcool
Having the local docker environment information in the .graphcoolrc file mixes two concerns: The concern of configuring docker locally and the concern of where a project is deployed.
Because if this, we decided to put this information into the .graphcool file (which is now a YAML file):
token: secure-platform-token
targets:
dev-1:
description: This is your local docker dev environment
host: http://localhost:60000
token: my-docker-token
do-2:
host: https://12.235.235.124:60000/
token: my-remote-token
That means the .graphcool file will turn into yaml as well. This makes it more consistent with the .graphcoolrc and graphcool.yml.
Renaming for gc deploy
We will rename the args for gc deploy. Instead of providing a project with -p, we will rename it to --target. This makes it easier to point either to the local docker version or to the hosted version of Graphcool.
With gc deploy --target dev-1 the project will be deployed to the local docker instance.
With gc deploy --target cj8fyu5wl0000jnom2muqv9sl you can point to a normal project id.
Making the gc deploy command interactive
In order to make choosing the deployment target easy, the gc deploy command will add an interactive dialog asking where to deploy. When enough args are provided, this dialog will be bypassed.
Removing of the gc env command
To make things easier, we will remove the gc env command as it is providing more confusion than any value. If you want to edit the local environments setting, just edit the .graphcoolrc directly.
New commands for gc local
The following commands will be added:
gc local eject: Puts the docker-compose.yml and .envrc in the current folder. From here on you are on your own and can do customizations if needed.
gc local ps INSTANCE_NAME: List all running containers of an instance
Sets up a new local docker service (or docker-compose ups the existing one) with the target name -t.
Warnings
Warning when override global property and used (e.g. platformToken or concrete target)
Aliasing
A target name (e.g. digitalocean or prod - see below) can reference other already existing targets. The default target has a special role here since it will be used when running gc deploy and specified in the .graphcoolrc file.
Example .graphcoolrc file
Global
platformToken: secure-platform-token # retrieved via `gc auth`
targets:
dev:
description: This is your local docker dev environment
host: http://localhost:60000
token: my-docker-token
digitalocean:
host: https://12.235.235.124:60000/
token: my-remote-token
Local
targets:
default: dev # `default` is a magic key that can reference other targets
prod: cj8g62nxg0000um3t0z64ls08
Usage
# deploy to default target
gc deploy
# deploy to prod
gc deploy --target prod
# deploy to digitalocean via shortcut argument
gc deploy -t digitalocean
Regarding removal of the graphcool env command. The only part of that command I find useful is setting the default environment, because it is used in many commands (including graphcool playground and graphcool console). For graphcool deploy, I think the environment/stage should always be explicitly specified, but for other commands, a default environment is still very useful, and there should be an easy way to set it, other than manually editing the environment file.
I want to get one thing straight here. You are using gc everywhere in here. Does it mean that current cli command graphcool will be replaced by gc or are us using it like alias here for brevity?
Although it feels strange to have two identical commands, I guess it's not a big deal. I am wrapping all important tasks in package.json scripts anyway, so I don't care about saving couple of extra characters :)
Yet another update: Given the multi-service cluster format (see #748) we're adjusting the .graphcoolrc file and gc deploy command to work with clusters.
platformToken: secure-platform-token # retrieved via `gc auth`
clusters:
default: local
# This is your local docker dev cluster
local:
host: http://localhost:60000
token: my-docker-token
digitalocean:
host: https://12.235.235.124:60000/
token: my-remote-token
Local
targets:
default: dev # `default` is a magic key that can reference other targets
dev: dev/cj8g62nxg0000um3t0z64ls08
prod: shared-eu-west-1/cj8g62nxg0000um3t0z64ls08
Usage
# deploy to default target
gc deploy
# deploy to prod
gc deploy --target prod
gc deploy -t dev/cj8irw8v20000jm3t7fvalhfg
gc deploy -t shared-eu-west-1/cj8irw8v20000jm3t7fvalhfg
gc deploy --new-service my-service name --new-service-cluster shared-eu-west-1
Open questions
[x] How to automatically/manually migrate existing .graphcoolrc and ~/.graphcool files.
[ ] How and when is a default cluster selected? Is this based on the ping? @timsuchanek
I completely lost it with regards to the 'services' and 'clusters' concepts. I've read #748 and #730, but I'm missing a substantial amount of pieces of the puzzle here.
This proposal is now implemented and released with version v0.7.0.
You can install it with npm install -g graphcool@next.
@kbrandwijk we will release more docs around this in the coming days!
$ gc --version
graphcool-framework/0.11.5 (linux-x64) node-v9.7.1
$ gc local reset
▸ local reset is not a graphcool-framework command.
▸ Perhaps you meant local restart
▸ Run graphcool-framework help local for a list of available commands.
Get in touch if you need help: https://www.graph.cool/forum
To get more detailed output, run $ export DEBUG="*"
@timsuchanek commented on Sun Oct 01 2017
Local Development
We will enable a local development workflow using docker under the hood. The topic this proposal is shedding light on is how that exactly looks like in the CLI.
Commands
gc local up
: Pull & Start a local Graphcool instancegc local start
: Start an already intialized local Graphcool instancegc local pull
: Pull the latest Graphcool versiongc local stop
: Stop the currently running local Graphcool instancegc local reset
: Reset the local Project.graphcoolrc
The
.graphcoolrc
would get a new "kind" of environment, calledlocal
:~/.graphcool
So additionaly to
token
, there will also belocal_token
andlocal_host
. These values all can be overwritten by environment variables:There can be one local environment at the same time. When multiple local environments are needed, that orchestration can be achieved with managing the environment variables.
Update:
Add
gc local restart
command.Update 2:
As @kbrandwijk suggested, it makes more sense for the local docker environment information to live in the
.graphcoolrc
file and not the.graphcool
file as.graphcool
is global and the local docker env information is project related. With this change the files look as follows:.graphcoolrc
~/.graphcool
@schickling commented on Sun Oct 01 2017
In general the proposal looks good to me. 👍 Here are a few questions:
What does this do exactly (step-by-step)?
What does this refer to? What's the purpose of this hostname? What's the underlying network protocol for the intended usage?
@kbrandwijk commented on Sun Oct 01 2017
I feel there might be something counter intuitive about the combination of environment definition and command. If the environment definition is:
dev: local
then I would expectgc up --env:dev
or something etc. Unless thegc local
commands get a--env
parameter, then it also makes sense.Would you be able to create more than one local enviroment?
Would this local docker environment play a role in automated testing?
How does deploying to this local environment work? Would you be able to do
gc deploy --env dev
withdev: local
? Would the machine need to be up for that? Or would the docker container contain a reference to the local project folder, so explicitly deploying to your local environment would not be necessary?Would you be able to deploy the docker container to a hosting provider, and would that basically be the self-hosted option?
~/.graphcool is not project specific. I don't think that these project-specific local environment settings belong there. They should probably go into the project directory.
@timsuchanek commented on Mon Oct 02 2017
We currently have 2 mutations for this:
They would both be called. We could have 2 more fine-grained versions of the command with
--data
and--schema
.This is the deployment endpoint that the CLI needs in order to deploy the project. Currently all endpoints are available on the :60000, we should separate them.
@timsuchanek commented on Mon Oct 02 2017
The idea is, that you can have multiple local environments. So I agree, adding
--env
to thegc up
command is then necessary. We can think about a default here like we do with the currentinit
command.That's an interesting question, I think it definitely could.
Yes, the machine needs to be up for that. The CLI should handle it gracefully when
deploy
is executed without a running docker instance.Yes, as people can deploy their docker stuff everywhere, this is the first minimal version of the self-hosted option. However, it's built for ease-of-use and not scalability.
That's a good point. We have to think about a better solution there.
@FredyC commented on Tue Oct 03 2017
I like this, I just want to add here that it's rather frustrating not even knowing the environment where functions are executed and what language features are supported. Especially that obscure "use latest" flag which is not really explained anywhere what it does. Is it a specific NodeJS version that runs that code?
@timsuchanek commented on Fri Oct 06 2017
Thanks for bringing that up @FredyC We definitely will make the function workflow much nicer soon :)
@timsuchanek commented on Fri Oct 06 2017
Updates
After having the first version implemented, we had some learnings that moved us to the following changes:
Updates for
~/.graphcool
Having the local docker environment information in the
.graphcoolrc
file mixes two concerns: The concern of configuring docker locally and the concern of where a project is deployed. Because if this, we decided to put this information into the.graphcool
file (which is now a YAML file):That means the
.graphcool
file will turn into yaml as well. This makes it more consistent with the.graphcoolrc
andgraphcool.yml
.Renaming for
gc deploy
We will rename the args for
gc deploy
. Instead of providing a project with-p
, we will rename it to--target
. This makes it easier to point either to the local docker version or to the hosted version of Graphcool. Withgc deploy --target dev-1
the project will be deployed to the local docker instance. Withgc deploy --target cj8fyu5wl0000jnom2muqv9sl
you can point to a normal project id.Making the
gc deploy
command interactiveIn order to make choosing the deployment target easy, the
gc deploy
command will add an interactive dialog asking where to deploy. When enough args are provided, this dialog will be bypassed.Removing of the
gc env
commandTo make things easier, we will remove the
gc env
command as it is providing more confusion than any value. If you want to edit the local environments setting, just edit the.graphcoolrc
directly.New commands for
gc local
The following commands will be added:
gc local eject
: Puts thedocker-compose.yml
and.envrc
in the current folder. From here on you are on your own and can do customizations if needed.gc local ps INSTANCE_NAME
: List all running containers of an instancegc local ls
: List all local instances.@schickling commented on Fri Oct 06 2017
Update:
~/.graphcool
and.graphcoolrc
file will be merged into a global & local.graphcoolrc
file.Root fields
Default behaviour
gc auth
Writes
platformToken
to global.graphcoolrc
filegc local up --target/-t
Sets up a new local docker service (or
docker-compose up
s the existing one) with the target name-t
.Warnings
Warning when override global property and used (e.g.
platformToken
or concrete target)Aliasing
A target name (e.g.
digitalocean
orprod
- see below) can reference other already existing targets. Thedefault
target has a special role here since it will be used when runninggc deploy
and specified in the.graphcoolrc
file.Example
.graphcoolrc
fileGlobal
Local
Usage
@marktani commented on Fri Oct 06 2017
What's the difference between
platformToken
andtoken
?@schickling commented on Fri Oct 06 2017
token
will be renamed toplatformToken
to avoid confusion with thetoken
needed for "Docker Targets".@kbrandwijk commented on Sat Oct 07 2017
Regarding removal of the
graphcool env
command. The only part of that command I find useful is setting the default environment, because it is used in many commands (includinggraphcool playground
andgraphcool console
). Forgraphcool deploy
, I think the environment/stage should always be explicitly specified, but for other commands, a default environment is still very useful, and there should be an easy way to set it, other than manually editing the environment file.@FredyC commented on Sat Oct 07 2017
I want to get one thing straight here. You are using
gc
everywhere in here. Does it mean that current cli commandgraphcool
will be replaced bygc
or are us using it like alias here for brevity?@kbrandwijk commented on Sat Oct 07 2017
You can already use both
graphcool
andgc
, ever since the first cli version.@FredyC commented on Sat Oct 07 2017
Interesting, where is that actually mentioned? Should be in official beta docs ... https://docs-next.graph.cool/reference/graphcool-cli/commands-aiteerae6l
Although it feels strange to have two identical commands, I guess it's not a big deal. I am wrapping all important tasks in
package.json
scripts anyway, so I don't care about saving couple of extra characters :)@kbrandwijk commented on Sat Oct 07 2017
A lot of cli commands have a 'shorthand' version, like
wt
is shorthand forwt-cli
for webtasks, serverless hasserverless
,slss
andsls
etc.@schickling commented on Fri Oct 13 2017
Yet another update: Given the multi-service cluster format (see #748) we're adjusting the
.graphcoolrc
file andgc deploy
command to work with clusters..graphcoolrc
root fieldsExample
.graphcoolrc
fileGlobal
Local
Usage
Open questions
.graphcoolrc
and~/.graphcool
files.@kbrandwijk commented on Mon Oct 09 2017
I completely lost it with regards to the 'services' and 'clusters' concepts. I've read #748 and #730, but I'm missing a substantial amount of pieces of the puzzle here.
@timsuchanek commented on Fri Oct 13 2017
This proposal is now implemented and released with version
v0.7.0
. You can install it withnpm install -g graphcool@next
. @kbrandwijk we will release more docs around this in the coming days!@zixia commented on Tue Mar 13 2018
Is
gc local reset
not supported yet?