config_server is limited to use with google.com domain

kunitake commented 8 years ago

Hi,

I think config_server is limited to use with google.com domain.

engine/config/config.go

configServerRE = regexp.MustCompile(`^[\w-\.]+\.google\.com\.?$`)

engine/config/fetcher.go

                if !configServerRE.MatchString(server) {
                        log.Errorf("Invalid config server name: %q", server)
                        continue
                }

Does config_server feature is not supported yet?

Best regards,

baptr commented 8 years ago

Our config server code itself is only useful internally, so we haven't released it. There's no reason something similar couldn't be built and used (and the RE changed) for anyone, however.

kunitake commented 8 years ago

OK, I'll try it.

Thank you!

4a6f656c commented 8 years ago

For the purposes of seesaw, the config server is just a web server - you should be able to point it at any HTTPS capable web server and have it fetch configuration protobufs (how those protobufs get created is a separate problem). As such, the Google specific restriction should be removed.

kunitake commented 8 years ago

Hi,

I tried it.

[cluster]
testcluster

[config_servers]
#primary = https://seesaw-config.localdomain/
primary = seesaw-config.example.com

In this case. The seesaw nodes will access to https://seesaw-config.example.com:10255/config/testcluster

10255 port number is hard coded at engine/config/engine.go

I put testcluster file as cluster.pb.

Finally,

root@testcluster> config source
Config source is server

Thank you for your comments!

I'm happy if seesaw support this featcure with configurable...

Best regards,

kunitake commented 8 years ago

Oops! Config Server's configuration (edited testcluster file) has not been reflected in the Seesaw nodes when I executed 'config reload' and restart seesaw_watchdog.

Hamm...

baptr commented 8 years ago

The engine log should shed some light on what happened with the config fetch.

kunitake commented 8 years ago

Hi @baptr ,

You are right! I have some error messages.

seesaw_engine.INFO

W0510 16:04:04.916328   30504 fetcher.go:187] Fetch failed: fetch failed from https://config.example.com:10255/config/testcluster (192.0.2.1): unexpected Content-Type: ""
E0510 16:04:04.916344   30504 notifier.go:157] Failed to pull configuration: all config server requests failed

Is the configfile (testcluster file) not same file as cluster.pb?

I'll read source code and try again.

Best regards.

baptr commented 8 years ago

I'm not sure what your testcluster file is. There are basically two configuration files for seesaw.

seesaw.cfg is meant to be fairly static, loaded from disk (only) at engine startup, which contains the [ini style] headers and basic node configuration (including the hosts to query as the config server).
cluster.pb is the protobuf formatted vserver configuration file, which is what the engine expects to read from the config server.

kunitake commented 8 years ago

I tested following file as testcluster file.

https://github.com/google/seesaw/blob/master/etc/seesaw/cluster.pb.example

I used ForceType directive on apache, because seesaw check Content-Type of configuration file.

<Location /config/>
   ForceType  application/x-protobuffer
</Location>

And I got new error message.

W0510 17:08:20.361693    6675 fetcher.go:187] Fetch failed: invalid configuration from https://config.example.com:10255/config/testcluster (192.0.2.1): proto: can't skip unknown wire type 7 for config.Cluster
E0510 17:08:20.361730    6675 notifier.go:157] Failed to pull configuration: all config server requests failed

I began to suspect config_server does not send the protobuf formatted file like cluster.pb.

Should I build config_server as application server?

baptr commented 8 years ago

https://github.com/google/seesaw/blob/master/engine/config/fetcher.go#L142 uses proto.Unmarshal, which expects wire (binary) formatted protobufs.

You could switch that to proto.UnmarshalText, or remarshal your text proto to the wire format and serve that.

kunitake commented 8 years ago

Hi,

I changed following codes

engine/config/fetcher.go
engine/config/notifier.go

        // if err := proto.Unmarshal(body, p); err != nil {
        if err := proto.UnmarshalText(string(body), p); err != nil {
                return nil, fmt.Errorf("invalid configuration from %v: %v", source, err)
        }

Finally, it works.

Many thanks to you!

I closed it.

luo964973791 commented 3 years ago

Hi,

I changed following codes

engine/config/fetcher.go

engine/config/notifier.go
        // if err := proto.Unmarshal(body, p); err != nil {
        if err := proto.UnmarshalText(string(body), p); err != nil {
                return nil, fmt.Errorf("invalid configuration from %v: %v", source, err)
        }
Finally, it works.

Many thanks to you!

I closed it. Hello, have you changed the codes of the two files fetcher.go notifier.go? I tested it according to your operation, and it still reported an error Failed to pull configuration: all config server requests failed

google / seesaw

config_server is limited to use with google.com domain #11