search

moleculerjs / moleculer

:rocket: Progressive microservices framework for Node.js
https://moleculer.services/
MIT License
6.17k stars 587 forks source link

[Proposal/Discussion] Latency-based Balancer #228

Closed zllovesuki closed 6 years ago

zllovesuki commented 6 years ago

My current setup is as follows: DC A: two moleculer services DC B: two moleculer services DC C: one moleculer services Transporter: gnatsd with TLS on all nodes (each moleculer ServiceBroker is connected to its local gnatsd) Persistent data backend: RethinkDB replicated in all DCs

However, DC C is relatively far away (100ms) from A/B (10ms). Currently if I call an action via the web gateway, some of the requests will be routed to DC C even though I'm requesting from DC A.

db-slugsurvival-*.* are running on the RethinkDB hosts for database calls, but the rests are scattered and in different environments to be resilient to failures.

Proposal

Implement some sort of latency-based balancer (like something that Cassandra does). Currently, moleculer already has a CPU-based balancer. So we can probably extend on that one somehow...

Issues

We don't have a reliable way of detecting "latency" if we have a nats transporter between all the nodes... I can see how we can determine the latency if we are running tcp transporter.

If you are interested in my services/actions, here it is...

mol $ actions -i -d
╔════════════════════════════════════════════╤═══════════════════════════════════════════════╤══════════╤════════╤═══════════════════════════════════════════════════╗
║ Action                                     │                                         Nodes │ State    │ Cached │ Params                                            ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ api-gw.rest                                │                                             5 │    OK    │   No   │                                                   ║
║                                            │                             andromeda-2-17594 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18568 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21772 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55806 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3436 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-data.fetch                 │                                             5 │    OK    │   No   │ key                                               ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-data.save                  │                                             5 │    OK    │   No   │ key, value                                        ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.getUser             │                                             5 │    OK    │   No   │ recipient                                         ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.getUsersByCourseNum │                                             5 │    OK    │   No   │ termId, courseNum                                 ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.insertUser          │                                             5 │    OK    │   No   │ recipient, termId, timestamp, code                ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.removeUser          │                                             5 │    OK    │   No   │ recipient                                         ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.setVerified         │                                             5 │    OK    │   No   │ recipient, verified                               ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.updateCourses       │                                             5 │    OK    │   No   │ recipient, courses                                ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ db-slugsurvival-notify.updateUser          │                                             5 │    OK    │   No   │ recipient, termId, timestamp, code, rewriteTermId ║
║                                            │                             andromeda-2-17593 │    OK    │        │                                                   ║
║                                            │                             andromeda-0-18576 │    OK    │        │                                                   ║
║                                            │                             andromeda-1-55814 │    OK    │        │                                                   ║
║                                            │                             andromeda-3-21780 │    OK    │        │                                                   ║
║                                            │                              andromeda-5-3610 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ messaging.auto                             │                                             4 │    OK    │   No   │                                                   ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ messaging.email                            │                                             4 │    OK    │   No   │ to, subject, text                                 ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ messaging.text                             │                                             4 │    OK    │   No   │ to, text                                          ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.getCourses             │                                             4 │    OK    │   No   │ recipient, termId, code                           ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.unsubscribe            │                                             4 │    OK    │   No   │ recipient, termId, code                           ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.updateCourses          │                                             4 │    OK    │   No   │ recipient, termId, code, courses                  ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.verifyNewUser          │                                             4 │    OK    │   No   │ recipient, termId                                 ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.verifyResend           │                                             4 │    OK    │   No   │ recipient, termId                                 ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-notify.verifyUser             │                                             4 │    OK    │   No   │ recipient, termId, code                           ║
║                                            │                    messier-941315164-dbb51-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-rn8x6-22 │    OK    │        │                                                   ║
║                                            │                    messier-941315164-s7lwb-22 │    OK    │        │                                                   ║
║                                            │                                messier-0-2754 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.available            │                                             3 │    OK    │  Yes   │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.compacted            │                                             3 │    OK    │  Yes   │ termId                                            ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.compactedMax         │                                             3 │    OK    │  Yes   │ termId                                            ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.fetch                │                                             3 │    OK    │  Yes   │ termId, courseNum                                 ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.heat                 │                                             3 │    OK    │  Yes   │ termId                                            ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╟────────────────────────────────────────────┼───────────────────────────────────────────────┼──────────┼────────┼───────────────────────────────────────────────────╢
║ slugsurvival-tracking.latestOne            │                                             3 │    OK    │  Yes   │ termId, courseNum                                 ║
║                                            │ andromeda-compute-tracking-400908814-w22x8-22 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-fvl4b-23 │    OK    │        │                                                   ║
║                                            │ andromeda-compute-tracking-400908814-t91qj-22 │    OK    │        │                                                   ║
╚════════════════════════════════════════════╧═══════════════════════════════════════════════╧══════════╧════════╧═══════════════════════════════════════════════════╝

Maybe even a picture: andromeda

icebob commented 6 years ago

The idea is great! You can measure the latency because there is PING/PONG feature in the protocol. It measures the latency between nodes. So we need to create a new Strategy (similar to CpuUsageStrategy), which ping the remote nodes with a timer and collect the current latency times. And in the select method, it selects a node according to the latency (with sampling).

zllovesuki commented 6 years ago

I will have a look into the source more when I find time... Also need to figure out how to measure latency efficiently and effectively.

zllovesuki commented 6 years ago

Again, I would use heartbeat to measure latency passively if I can, but not without changes in the protocol...

EDIT: although as per #230, heartbeat does not work on tcp transporter.

icebob commented 6 years ago

Merged.

icebob commented 6 years ago

@zllovesuki Could you help me to edit "Strategies" page & add Latency strategy with example in the docs?

Page: http://moleculer.services/0.12/docs/service-registry.html Edit: https://github.com/moleculerjs/site/edit/master/source/0.12/docs/service-registry.md

Thanks!