search

confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.16k stars 1.1k forks source link

Support port virtualization #487

Open fvlankvelt opened 7 years ago

fvlankvelt commented 7 years ago

When running the schema registry in a Docker container, the port that's exposed on the host is not necessarily the same as the one that the rest service binds to. When running multiple instances of the schema registry, this becomes an issue, as one of them is the master and others should proxy to it.

The 'host.name' config variable already allows the correct hostname to be specified (i.e. so that the docker host is used, rather than the hostname in the container), but a similar variable for the port does not yet exist.

This functionality is similar to what's offered by the advertised.host.name and advertised.port config variables in Kafka.

fvlankvelt commented 7 years ago

For others running into the same problem: a workaround is to configure two listeners:

  listeners=http://0.0.0.0:$PUBLIC_PORT,http://0.0.0.0:$INTERNAL_PORT

the first port will be registered in Zookeeper as the master url, the second one will make sure the RestServer actually receives requests from other (non-master) instances.

michaelgrossopt commented 3 years ago

Thank you for sharing this workaround @fvlankvelt this almost solved my problem :-). My scenario is I'm trying to move the masters into k8s where we're using stateful sets with 1:1 service and ingress mappings to pods so each will have an externally addressable name that we're setting as the host.name but we ran into the same problem with the port (the ingress is listening on port 80 but schema registry is binding on port 8081 inside the pod). Our problem and what caused this workaround to not work in our case is we can't bind to port 80 within the pod as it's a privileged port, so we really do need a separate advertised.port type variable we can use to advertise to clients that isn't bound by the schema registry service.

FWIW this is true regardless of the version of Schema Registry (we've tried as recent as 5.3.3) and whether we're using Zookeeper or Kafka for leader election.

OneCricketeer commented 2 years ago

https://docs.confluent.io/platform/current/schema-registry/installation/config.html#listeners

Schema Registry identities are stored in ZooKeeper and are made up of a hostname and port. If multiple listeners are configured, the first listener’s port is used for its identity.