Tooling and documentation on developing functions

[jroper]

Currently, there's no simple way to run the proxy locally, which means there's no way to run a stateful service locally and talk to it using its gRPC interface.

Of course, you can run minikube or something like that locally, but for development, you may want to run your function with a debugger, for example, so you want to run it on your host machine, not in a container.

For a development environment, I think the basic idea would be to run the proxy in docker, and have it call into the function running on the host container. There's a few challenges with this, the biggest one is that there's no standard, cross platform way for a docker container to know the IP address of the host, see:

https://nickjanetakis.com/blog/docker-tip-65-get-your-docker-hosts-ip-address-from-in-a-container

So we'll probably need some tooling to facilitate this. Let's say we had a cloudstate command, I would imagine something like this:

cloudstate run-proxy --http-port=9000 --user-function-port=8080

This would roughly translate to running:

docker run --rm -e USER_FUNCTION_PORT=8080 -e USER_FUNCTION_HOST=<host-ip-address> -p 9000:9000 cloudstateio/cloudstate-proxy-native-dev-mode:latest

Where <host-ip-address> was host.docker.internal on OSX and Windows, and the output of ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+' on Linux. Right now, the USER_FUNCTION_HOST environment variable isn't read, so we'd need to add support for that. 9000/8080 could be the defaults, so if you were just testing one function you could use cloudstate run-proxy, but then if you wanted to run multiple, you'd specify ports.

Having run the above, you could now instantiate a client for your gRPC interface and connect to it on localhost:9000. Easy.

We might also consider whether we want to support bootstrapping a cluster, this would be a lot more involved of course. It may make more sense to simply say if you want to test with a cluster, use minikube.

[viktorklang]

@jroper For running it locally without having the build at hand, would it perhaps make more sense to "just" show the docker command needed, so we wouldn't have to maintain launchers for every platform?

[sleipnir]

Hello @jroper I am implementing in the CLI the cloudstate run command that will do just that.

[sleipnir]

@jroper Actually in my implementation I am considering running the function as a container as well. This does not solve the case you described for debugging purposes. But I'm thinking of a parameter option similar to

cloudstate run --dev-mode --ip-range=192.168.0.0/24

and so the CLI would create a bridge mode docker network. Something like:

docker network create -d bridge --subnet 192.168.0.0/24 --gateway 192.168.0.1 cloudstate-network

and would proxy this network docker

[sleipnir]

@viktorklang Since Cloudstate CLI is just a shell over native commands it could show the exact command on the command line in order to educate the user on native execution if he wants to use it as well. But it could also be included in the documentation.

[viktorklang]

@sleipnir :+1:

[sleipnir]

Just to update, this is the actual syntax of command and add it to the Cloudstate CLI project:

[sleipnir@sleipnir cloudstate]# ./target/debug/cloudstate run --only-proxy --proxy-port=9000 --user-function-port 8080
Running only proxy container
Command: docker run --rm --net=host --name proxy --env HTTP_PORT:9000 -e USER-FUNCTION-PORT:8080 cloudstateio/cloudstate-proxy-native-dev-mode:latest
For stop press ctrl+c
^C

[sleipnir@sleipnir cloudstate]# ./target/debug/cloudstate run --help
cloudstate-run 
Running user function & cloudstate proxy in Docker for tests purposes

USAGE:
    cloudstate run [FLAGS] [OPTIONS]

FLAGS:
    -h, --help          Prints help information
    -o, --only-proxy    Running only Proxy (dev mode). Sharing the host network
    -V, --version       Prints version information

OPTIONS:
    -u, --user-function-port <function-port>    Indicates to the Proxy which port is the user function so that it can communicate
    -p, --proxy-port <proxy-port>               Proxy exposed port. Default is 9000

[SeanCheatham]

@sleipnir was kind enough to direct me to this thread from the Gitter channel.

I recently started exploring the idea of a CloudState IDE. It would not be the traditional IDE where developers maintain direct control over the files in a project. Instead, the IDE would manage the project on behalf of the developer. The developer is responsible for defining Models, Event Sourced Entities, and CRDT Entities in a guided/visual format. (Developers still write the actual function code in a CloudState-supported language though) Simple one-click buttons would be available to Compile, Test, and Deploy. The IDE would integrate with some cloud provider (like GCP or AWS or Azure) to handle all of the operational heavy-lifting. The goal, like with CloudState as a whole, is to further reduce the boilerplate and tediousness of bringing business logic to production.

I'm curious if anything like this (visual tooling) is on the roadmap for CloudState?

What I have so far, as of this comment, is just a rough frontend written in Flutter. It allows a developer to define models, event sourced entities, and CRDTs. Nothing is hooked up to a backend though; nothing currently "works". I'd be happy to share screenshots and a link to the git repo, but I'll await permission as I don't want to violate any sort of advertising guidelines. In the meantime, I'll keep working on it :smile:

[sleipnir]

Hello @SeanCheatham as long as there are no images or mentions there are products and or companies and their brands I believe it is possible to share your links and images without breaking any type of rule

[sleipnir]

@SeanCheatham Thanks for the initiative, I will be happy to contribute if I can

[SeanCheatham]

Okay thank you for clarifying! This is all just independent open source work, so no company branding :smile:

Here are a couple screenshots of what I have so far.

Model Editor:

Event Sourced Entity Editor:

CRDT Entity Editor:

The project is currently located here: https://github.com/SeanCheatham/cloudstation

As of right now, this is frontend only. None of the backend work has been implemented yet. I'm still in the "planning" phase of that.

[sleipnir]

As of right now, this is frontend only. None of the backend work has been implemented yet. I'm still in the "planning" phase of that.

In the model definition stage, it would be interesting to be able to import the protobuf definition files. Wdyt?

[SeanCheatham]

In the model definition stage, it would be interesting to be able to import the protobuf definition files. Wdyt?

That seems like a reasonable thing to support. I imagine for some organizations, it may actually be a requirement. I created a ticket to capture the work. Thank you for the suggestion!

[coreyauger]

This looks rad @SeanCheatham !! Nice work!