Implement Citus support for pg_auto_failover

[DimCitus]

The support for Citus in pg_auto_failover used to be restricted, and is now being made Open Source.

Some of the needed changes include:

• Adding a “guard” in the keeper FSM to have standalone/coordinator/worker specific transitions
• Adding documentation coverage for Citus specific areas, and updating all the schematics
• Implement make CITUS=1 cluster (at the moment the Citus support for make compose is missing)
• Allow CI testing of the Citus support

Fixes #670. See #416. Fixes #411. Fixes #362. See #50.

[s4ke]

Not a review - just a post of appreciation: This is looking awesome for the future of this project and for my projects hat use PostgreSQL. The whole idea of Citus being fully Open Source removes a lot of my headaches for scaling with PostgreSQL.

[DimCitus]

Docs build preview is available at https://pg-auto-failover.readthedocs.io/en/feature-citus/intro.html#citus-architecture and https://pg-auto-failover.readthedocs.io/en/feature-citus/citus.html ; and then https://pg-auto-failover.readthedocs.io/en/feature-citus/quickstart-cluster.html etc.

[marcocitus]

Would be good to add the coordinator to the metadata automatically. This is required for a lot of features now.

select create_distributed_table_concurrently('test','x');
ERROR:  coordinator is not added to the metadata                                                                                                                                  
HINT:  Use SELECT citus_set_coordinator_host('<hostname>') to configure the coordinator hostname    

And to set wal_level to logical:

select citus_set_coordinator_host('localhost');
select citus_set_node_property(nodename, nodeport, 'shouldhaveshards', true) from pg_dist_node;

select create_distributed_table_concurrently('test','x');
ERROR:  ERROR:  logical decoding requires wal_level >= logical
CONTEXT:  while executing command on localhost:6001

[DimCitus]

Would be good to add the coordinator to the metadata automatically. This is required for a lot of features now.

That a good comment, I think we should act on it on a later PR right after merging that one. In particular because we have then to introduce code behaviour that depends on Citus version (your comment only applies to Citus 11+), and we don't have that infrastructure in the code yet.