Ensemble domains, executors, and supervisory services

[vyzo]

Implements ensemble domains, with supervisory services and the first implementation of the global execution environment.

Work Items

• [x] introduce the concept of ensemble domains, which define hierarchical namespaces.
• [x] ensemble supervisory services
• [x] update httpd for proper ensemble
• [x] debugging
• [x] tooling
• [x] demo
• [ ] update older ensemble commands for the new world order
• [ ] tests
• [ ] documentation:
  • [ ] api
  • [ ] tutorials:
    • [x] update existing ensemble turorial
    • [ ] create advanced ensemble tutorial
    • [ ] update httpd tutorial for gxhttpd

[netlify[bot]]

✅ Deploy Preview for elastic-ritchie-8f47f9 ready!

Name	Link
🔨 Latest commit	e767fad79a28972936b32db93e596b7276e61011
🔍 Latest deploy log	https://app.netlify.com/sites/elastic-ritchie-8f47f9/deploys/66ce499b56e6640008ecd200
😎 Deploy Preview	https://deploy-preview-1263--elastic-ritchie-8f47f9.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[fare]

Looks generally good, but

• tests are required
• not sure how you spawn tasks on other hosts (if you do)

[vyzo]

Of course, this is wip.

I made the server identifiers pairs of server id and ensemble domain.

I will do supervisors, executors, and yes tests, next.

[vyzo]

demo teaser:

$ cat hello.ss
(export main)
(import :std/sugar
        :std/actor
        :std/io
        :std/logger
        :std/config)

(def (main server-id)
  (let (cfg (load-ensemble-server-config (string->server-identifier server-id)))
    (become-ensemble-server! cfg (cut start-hello! cfg))))

(def (start-hello! cfg)
  (spawn/name 'hello hello cfg))

(def (hello cfg)
  (let* ((listen-addr (agetq 'hello (config-get! cfg application:)))
         (server-sock (tcp-listen listen-addr)))
    (spawn say-hello server-sock)
    (let/cc exit
      (while #t
        (<-
         ,(@shutdown
           (infof "shutting down...")
           (using (closer server-sock : Closer)
             (closer.close))
           (exit 'shutdown))
         ,(@ping)
         ,(@unexpected warnf))))))

(def (say-hello sock)
  (using (sock : ServerSocket)
    (while #t
      (using ((cli (sock.accept) : StreamSocket)
              (writer (cli.writer) : Writer))
        (writer.write (string->bytes "hello, world\n"))
        (writer.close)
        (cli.close)))))

$ gxc -O -exe -o hello hello.ss
$ gerbil ensemble control upload --exe hello hello
"418d9ccb77f1ee14f773d20fc29a5bd5f01356056a3fdb72d701599beff68b1a"

$ cat hello.config
config: ensemble-v0
roles:
((hello
  exe: "hello"
  server-config:
  (config: ensemble-server-v0
           application: ((hello . "127.0.0.1:9999")))))

$ gerbil ensemble control update-ensemble-config hello.config
$ gerbil ensemble control start-server -D /test hello hello
((hello . /test) . 910632)

$ gerbil ensemble control get-server-config '(hello . /test)'
(config: ensemble-server-v0 domain: /test identifier: (hello . /test) supervisor: (supervisor . /test) registry: (registry . /test) cookie: "/home/vyzo/gerbil/src/tools/gxensemble-test/dot-gerbil/ensemble/cookie" admin: "/home/vyzo/gerbil/src/tools/gxensemble-test/dot-gerbil/ensemble/admin.pub" role: hello exe: "hello" args: ("(hello . /test)") policy: restart env: "default" log-level: INFO log-file: "/home/vyzo/gerbil/src/tools/gxensemble-test/root/log/test/hello/server.log" log-dir: "/home/vyzo/gerbil/src/tools/gxensemble-test/root/log/test/hello" addresses: ((unix: "dellicious" "/tmp/ensemble/test/hello.sock")) auth: (((supervisor . /test) admin)) known-servers: (((registry . /test) (unix: "dellicious" "/tmp/ensemble/test/registry.sock"))) application: ((hello . "127.0.0.1:9999")))

$ telnet 127.0.0.1 9999
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
hello, world
Connection closed by foreign host.

$ gerbil ensemble control stop-server '(hello . /test)'
(910632)

$ gerbil ensemble control list-servers
(((registry . /test) 910570 running))

$ telnet 127.0.0.1 9999
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused

[vyzo]

Let me fix the remaining broekn commands in gxensemble first.