trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.36k stars 2.98k forks source link

Improve documentation about network topology in Node scheduler properties #15705

Open llegolas opened 1 year ago

llegolas commented 1 year ago

We are running trino 405 on EKS. Trying to reduce inter-AZ traffic costs we decided to experiment with node-scheduler.network-topology config options. but we are getting the errors below:

Errors:

1) Configuration property 'node-scheduler.network-topology.segments' was not used
2) Configuration property 'node-scheduler.network-topology.subnet.cidr-prefix-lengths' was not used
3) Configuration property 'node-scheduler.network-topology.subnet.ip-address-protocol' was not used
4) Configuration property 'node-scheduler.network-topology.type' was not used

We've tried all the permutations of setting the options on the coordinator, workers or just on one of them but the coordinator and/or workers refuse to start.

The relevant section of the config.properties looks like the below:

config.properties: |
    ......
    node-scheduler.network-topology.segments=vpc,region
    node-scheduler.network-topology.type=subnet
    node-scheduler.network-topology.subnet.ip-address-protocol=IPv4
    node-scheduler.network-topology.subnet.cidr-prefix-lengths=19,22
    .....

What are we missing or/and doing wrong ?

hashhar commented 1 year ago

Do you have node-scheduler.policy=topology set?

llegolas commented 1 year ago

Do you have node-scheduler.policy=topology set?

Actually no but it is not mentioned in the docs as a prerequisite although it certainly sounds like one.