marieai / marie-ai

Integrate AI-powered Document Analysis Pipelines
MIT License
61 stars 5 forks source link

Integrate service discovery #43

Closed gregbugaj closed 1 year ago

gregbugaj commented 1 year ago

Port existing service discovery that have been implemented via register.py into the new Gateway / Flow framework.

gregbugaj commented 1 year ago

Gateway discovery service can be tested via YAML and code.

Show help via CLI:

marie gateway --help

Under 'Gateway Argument new attributes are present

  --discovery           default: disabled, use "--discovery" to enable it

                        If set, service discovery will be enabled. Default service discovery provider will use Consul

  --discovery-host      type: str; default: None

                        If discovery is enabled, this hostname will be used to configure the discovery agent.

  --discovery-port      type: int; default: None

                        If discovery is enabled, this port will be used to configure the discovery agent.

  --discovery-watchdog-interval 
                        type: int; default: 60

                        Time interval (seconds) between sending system health checks

Start gateway and specify Consul information.

marie gateway --protocol http --discovery --discovery-host 127.0.0.1 --discovery-port 8500 --host 192.168.102.65 --port 5555

Same thing can be done via code :+1:

f = Flow(
    protocol='http',
    port=12345,
    discovery=True,
    discovery_host='127.0.0.1',
    discovery_port=8500,
    discovery_watchdog_interval=60,
).add(uses=FooExecutor)
# f.to_docker_compose_yaml('/tmp/docker-compose.yml')

with f:
    f.block()