SRV for multiple ports

[thockin]

With https://github.com/GoogleCloudPlatform/kubernetes/pull/6182 kubernetes introduces multi-port service IPs.

I'm trying to program SkyDNS to support this, but it seems there is no way to do it. Please correct me if I am wrong? Specifically the msg.Service structure has no place to put the service name (port name for us) or the protocol, and it only supports a single port.

Scanning code I do not see any faculties for using SRV "properly". Please advise?

[smarterclayton]

I think you can create one entry per port you're exposing... at least that's what I saw in that regard.

[thockin]

I tried this and only got one record - the last one.

On Mon, Mar 30, 2015 at 8:54 PM, Clayton Coleman notifications@github.com wrote:

I think you can create one entry per port you're exposing... at least that's what I saw in that regard.

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-87926025 .

[miekg]

[ Quoting notifications@github.com in "[skydns] SRV for multiple ports (#1..." ]

With https://github.com/GoogleCloudPlatform/kubernetes/pull/6182 kubernetes introduces multi-port service IPs.

I'm trying to program SkyDNS to support this, but it seems there is no way to do it. Please correct me if I am wrong? Specifically the msg.Service structure has no place to put the service name (port name for us) or the protocol, and it only supports a single port.

Scanning code I do not see any faculties for using SRV "properly". Please advise?

What is it what you exactly need? Reading this, I think the answer is no.

You need an extra field in msg.Service so you can differentiate the different ports? So that skydns returns multiple SRVs for one name?

/Miek

Miek Gieben

[thockin]

Since the SkyDNS record has a field for "port" I am trying to figure out what to do now that I have multiple ports on a single IP.

full disclosure: I barely know anything about SRV, so I am going from wikipedia :)

On Mon, Mar 30, 2015 at 11:56 PM, Miek Gieben notifications@github.com wrote:

[ Quoting notifications@github.com in "[skydns] SRV for multiple ports (#1..." ]

With https://github.com/GoogleCloudPlatform/kubernetes/pull/6182 kubernetes introduces multi-port service IPs.

I'm trying to program SkyDNS to support this, but it seems there is no way to do it. Please correct me if I am wrong? Specifically the msg.Service structure has no place to put the service name (port name for us) or the protocol, and it only supports a single port.

Scanning code I do not see any faculties for using SRV "properly". Please advise?

What is it what you exactly need? Reading this, I think the answer is no.

You need an extra field in msg.Service so you can differentiate the different ports? So that skydns returns multiple SRVs for one name?

/Miek

Miek Gieben

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-87967537 .

[miekg]

You'll need multiple SRV records in that case. So, this works already today. When you need 2 ports you'll have to register 2 names:

port80.something.skydns.local and port81.something.skydns.local. The record that you then want to use is something.skydns.local which will return both port80 and port81 SRV record. You cannot encode multiple ports inside a single SRV record.

[thockin]

I'm not sure how to do that with SkyDNS msg.Service structs. Given a service "foo.bar" at IP 1.2.3.4 with 2 ports, "http" and "https" - Is it like this?

etcd key = "foo.bar.skydns.local"

Service = {

Host: "1.2.3.4",

Port: 0  // FIXME: is this right?

}

etcd key = "http.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 80

}

etcd key = "https.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 443

}

On Apr 1, 2015 2:26 AM, "Miek Gieben" notifications@github.com wrote:

You'll need multiple SRV records in that case. So, this works already today. When you need 2 ports you'll have to register 2 names:

port80.something.skydns.local and port81.something.skydns.local. The record that you then want to use is something.skydns.local which will return both port80 and port81 SRV record. You cannot encode multiple ports inside a single SRV record.

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88411221 .

[miekg]

Yes. Does that work? You don't need the port: 0 I think. It should be optionally.

This whole thing might be worth documenting as well. On 1 Apr 2015 5:02 pm, "Tim Hockin" notifications@github.com wrote:

I'm not sure how to do that with SkyDNS msg.Service structs. Given a service "foo.bar" at IP 1.2.3.4 with 2 ports, "http" and "https" - Is it like this?

etcd key = "foo.bar.skydns.local"

Service = {

Host: "1.2.3.4",

Port: 0 // FIXME: is this right?

}

etcd key = "http.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 80

}

etcd key = "https.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 443

}

On Apr 1, 2015 2:26 AM, "Miek Gieben" notifications@github.com wrote:

You'll need multiple SRV records in that case. So, this works already today. When you need 2 ports you'll have to register 2 names:

port80.something.skydns.local and port81.something.skydns.local. The record that you then want to use is something.skydns.local which will return both port80 and port81 SRV record. You cannot encode multiple ports inside a single SRV record.

— Reply to this email directly or view it on GitHub < https://github.com/skynetservices/skydns/issues/144#issuecomment-88411221> .

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88533788 .

[thockin]

This can't be right - the etcd key gets converted to a Path, and then I get:

2015/04/01 16:17:30 Setting DNS record: hostnames.default.kubernetes.local. -> 10.0.245.85:0 2015/04/01 16:17:30 Setting DNS record: frobber.hostnames.default.kubernetes.local. -> hostnames.default.kubernetes.local.:80 2015/04/01 16:17:30 Failed to mutate etcd using mutator: 0x403fe0 due to: 104: Not a directory (/skydns/local/kubernetes/default/hostnames) [4731]. Will retry in: 50ms

Please advise

On Wed, Apr 1, 2015 at 9:01 AM, Tim Hockin thockin@google.com wrote:

I'm not sure how to do that with SkyDNS msg.Service structs. Given a service "foo.bar" at IP 1.2.3.4 with 2 ports, "http" and "https" - Is it like this?

etcd key = "foo.bar.skydns.local"

Service = {

Host: "1.2.3.4",

Port: 0  // FIXME: is this right?

}

etcd key = "http.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 80

}

etcd key = "https.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 443

}

On Apr 1, 2015 2:26 AM, "Miek Gieben" notifications@github.com wrote:

You'll need multiple SRV records in that case. So, this works already today. When you need 2 ports you'll have to register 2 names:

port80.something.skydns.local and port81.something.skydns.local. The record that you then want to use is something.skydns.local which will return both port80 and port81 SRV record. You cannot encode multiple ports inside a single SRV record.

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88411221 .

[miekg]

Ah yes. I can see what's happening here. The actual hostname and thus the IP needs to live somewhere else. I will give it it try later. On 1 Apr 2015 5:57 pm, "Tim Hockin" notifications@github.com wrote:

This can't be right - the etcd key gets converted to a Path, and then I get:

2015/04/01 16:17:30 Setting DNS record: hostnames.default.kubernetes.local. -> 10.0.245.85:0 2015/04/01 16:17:30 Setting DNS record: frobber.hostnames.default.kubernetes.local. -> hostnames.default.kubernetes.local.:80 2015/04/01 16:17:30 Failed to mutate etcd using mutator: 0x403fe0 due to: 104: Not a directory (/skydns/local/kubernetes/default/hostnames) [4731]. Will retry in: 50ms

Please advise

On Wed, Apr 1, 2015 at 9:01 AM, Tim Hockin thockin@google.com wrote:

I'm not sure how to do that with SkyDNS msg.Service structs. Given a service "foo.bar" at IP 1.2.3.4 with 2 ports, "http" and "https" - Is it like this?

etcd key = "foo.bar.skydns.local"

Service = {

Host: "1.2.3.4",

Port: 0 // FIXME: is this right?

}

etcd key = "http.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 80

}

etcd key = "https.foo.bar.skydns.local"

Service = {

Host: "foo.bar.skydns.local",

Port: 443

}

On Apr 1, 2015 2:26 AM, "Miek Gieben" notifications@github.com wrote:

You'll need multiple SRV records in that case. So, this works already today. When you need 2 ports you'll have to register 2 names:

port80.something.skydns.local and port81.something.skydns.local. The record that you then want to use is something.skydns.local which will return both port80 and port81 SRV record. You cannot encode multiple ports inside a single SRV record.

— Reply to this email directly or view it on GitHub < https://github.com/skynetservices/skydns/issues/144#issuecomment-88411221> .

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88541656 .

[miekg]

(strapped for time) I think this should work:

etcd key = "host.foo.bar.skydns.local" 
Service = { Host: "1.2.3.4" } 

etcd key = "http.foo.bar.skydns.local" 
Service = { Host: "host.foo.bar.skydns.local", Port: 80 } 

etcd key = "https.foo.bar.skydns.local" 
Service = { Host: "foo.bar.skydns.local", Port: 443 }

[miekg]

So, isnt just this what you want:

 ETCD="127.0.0.1:4001/v2/keys/skydns"
 curl -XPUT $ETCD/local/skydns/bar/foo/addr \
        -d value='{"host":"192.168.0.1", "port":80}'

 curl -XPUT $ETCD/local/skydns/bar/foo/443 \
        -d value='{"host": "addr.foo.bar.skydns.local", "port":443}'

dig @localhost SRV foo.bar.skydns.local, gives:

;; ANSWER SECTION:
foo.bar.skydns.local.   3600    IN  SRV 10 50 80 addr.foo.bar.skydns.local.
foo.bar.skydns.local.   3600    IN  SRV 10 50 443 addr.foo.bar.skydns.local.

;; ADDITIONAL SECTION:
addr.foo.bar.skydns.local. 3600 IN  A   192.168.0.1

So two foo.bar's with SRV records to a diferent port with the same ip address.

?

[thockin]

But I want the A record to be "foo.bar..." not "addr.foo.bar..."

On Wed, Apr 1, 2015 at 12:19 PM, Miek Gieben notifications@github.com wrote:

So, isnt just this what you want:

ETCD="127.0.0.1:4001/v2/keys/skydns" curl -XPUT $ETCD/local/skydns/bar/foo/addr \ -d value='{"host":"192.168.0.1", "port":80}'

curl -XPUT $ETCD/local/skydns/bar/foo/443 \ -d value='{"host": "addr.foo.bar.skydns.local", "port":443}'

dig @localhost SRV foo.bar.skydns.local, gives:

;; ANSWER SECTION: foo.bar.skydns.local. 3600 IN SRV 10 50 80 addr.foo.bar.skydns.local. foo.bar.skydns.local. 3600 IN SRV 10 50 443 addr.foo.bar.skydns.local.

;; ADDITIONAL SECTION: addr.foo.bar.skydns.local. 3600 IN A 192.168.0.1

So two foo.bar's with SRV records to a diferent port with the same ip address.

?

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88599683 .

[miekg]

Ah, ok. Finally nailed to actually bug/issue :-) No, this can't be done. Now I get your original feature request for multiple ports (which could work).

Why must the A record be for foo.bar and not for addr.foo.bar, i.e. what breaks at your side?

[miekg]

Another way of saying this is: we want to be able to store multiple msg.Service under a key.

Which leads to a syntax in which the name implies "same key", "different record".

/local/skydns/bar/foo@1 and /local/skydns/bar/foo@2 or something like that. Have to think through the implication, but I think this can "just" work.

[thockin]

All clients are doing DNS lookups for foo.bar already. I want to offer SRV for the ports they define.

I tried it, and it seems that skyDNS is setting up an A record for both.

docker exec -ti 8aba9187~f740 /etcdctl set

/skydns/local/kubernetes/myns2/mysvc/host '{"host":"1.2.3.4"}'

nslookup mysvc.myns2.kubernetes.local 10.0.0.10

Server: 10.0.0.10 Address: 10.0.0.10#53

Name: mysvc.myns2.kubernetes.local Address: 1.2.3.4

nslookup host.mysvc.myns2.kubernetes.local 10.0.0.10

Server: 10.0.0.10 Address: 10.0.0.10#53

Name: host.mysvc.myns2.kubernetes.local Address: 1.2.3.4

I am not sure how to ask dig for "all records in this domain".

This breaks down if I add multiple records under that key though, as you suggested above.

I just want to add a default address for a subdomain, I guess, but I can't see how as subdomains are represented as valueless directories. Maybe some magical "out-of-band" sub-record like "_default"?

On Wed, Apr 1, 2015 at 1:34 PM, Miek Gieben notifications@github.com wrote:

Ah, ok. Finally nailed to actually bug/issue :-) No, this can't be done. Now I get your original feature request for multiple ports (which could work).

Why must the A record be for foo.bar and not for addr.foo.bar, i.e. what breaks at your side?

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88623812 .

[thockin]

e.g.

# docker exec -ti 8aba9187f740 /etcdctl set
/skydns/local/kubernetes/myns4/mysvc/_default '{"host":"1.2.3.4"}'

# docker exec -ti 8aba9187f740 /etcdctl set
/skydns/local/kubernetes/myns4/mysvc/http
'{"host":"mysvc.myns4.kubernetes.local", "port":80}'

# docker exec -ti 8aba9187f740 /etcdctl set
/skydns/local/kubernetes/myns4/mysvc/https
'{"host":"mysvc.myns4.kubernetes.local", "port":443}'

# dig @10.0.0.10 SRV mysvc.myns4.kubernetes.local

; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> @10.0.0.10 SRV
mysvc.myns4.kubernetes.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59496
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;mysvc.myns4.kubernetes.local. IN SRV

;; ANSWER SECTION:
mysvc.myns4.kubernetes.local. 3600 IN SRV 10 50 80
mysvc.myns4.kubernetes.local.
mysvc.myns4.kubernetes.local. 3600 IN SRV 10 50 443
mysvc.myns4.kubernetes.local.

;; Query time: 2 msec
;; SERVER: 10.0.0.10#53(10.0.0.10)
;; WHEN: Wed Apr  1 21:05:30 2015
;; MSG SIZE  rcvd: 142

All that DOESN'T work is that mysvc.myns4.kubernetes.local does not resolve. I wonder if SkyDNS's "service oriented" API is just a bad fit for what I want to do? Alternatives?

On Wed, Apr 1, 2015 at 1:59 PM, Tim Hockin thockin@google.com wrote:

All clients are doing DNS lookups for foo.bar already. I want to offer SRV for the ports they define.

I tried it, and it seems that skyDNS is setting up an A record for both.

docker exec -ti 8aba9187~f740 /etcdctl set

/skydns/local/kubernetes/myns2/mysvc/host '{"host":"1.2.3.4"}'

nslookup mysvc.myns2.kubernetes.local 10.0.0.10

Server: 10.0.0.10 Address: 10.0.0.10#53

Name: mysvc.myns2.kubernetes.local Address: 1.2.3.4

nslookup host.mysvc.myns2.kubernetes.local 10.0.0.10

Server: 10.0.0.10 Address: 10.0.0.10#53

Name: host.mysvc.myns2.kubernetes.local Address: 1.2.3.4

I am not sure how to ask dig for "all records in this domain".

This breaks down if I add multiple records under that key though, as you suggested above.

I just want to add a default address for a subdomain, I guess, but I can't see how as subdomains are represented as valueless directories. Maybe some magical "out-of-band" sub-record like "_default"?

On Wed, Apr 1, 2015 at 1:34 PM, Miek Gieben notifications@github.com wrote:

Ah, ok. Finally nailed to actually bug/issue :-) No, this can't be done. Now I get your original feature request for multiple ports (which could work).

Why must the A record be for foo.bar and not for addr.foo.bar, i.e. what breaks at your side?

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88623812 .

[miekg]

Yes, I see. The thing is that SRV records first class citizens and A/AAAA records are not. What could be done is extending msg.Service with an IP array which specifies the IP addresses (v4 or v6). If not found skydns will just use the current mechanism for finding an IP.

I'm just wondering if this needs to more generic, i.e. do we want a TXT array as well. (We just might, as we support that as well).

[thockin]

with this IP array, how do I specify that I want an IP (A record) assigned to the directory?

On Wed, Apr 1, 2015 at 2:16 PM, Miek Gieben notifications@github.com wrote:

Yes, I see. The thing is that SRV records first class citizens and A/AAAA records are not. What could be done is extending msg.Service with an IP array which specifies the IP address (v4 or v6). If not found skydns will just use the current mechanism for finding an IP.

I'm just wondering if this needs to more generic, i.e. do we want a TXT array as well. (We just might, as we support that as well).

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88633982 .

[miekg]

hmm, yeah. That would still break, how about the syntax from above: local/skydns/bar/foo@1? That could still work I think.

[thockin]

The assumption being that the record "local/skydns/bar/foo" is the default A record and "local/skydns/bar/foo@1" means the same as "local/skydns/bar/foo/1" ?

If that is what you mean, I think it would work, though it feels a bit more complex to me than a "local/skydns/bar/foo/@default" or similar.

On Wed, Apr 1, 2015 at 2:28 PM, Miek Gieben notifications@github.com wrote:

hmm, yeah. That would still break, how about the syntax from above: local/skydns/bar/foo@1? That could still work I think.

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88636311 .

[miekg]

No. "local/skydns/bar/foo@1 is just how it is stored in etcd, for the DNS answer everything after (and including @) is dropped. So @default would work as well.

Why I like it more is because it is more general, you can just put more Service entries in a directory (as you should)

[thockin]

oh, so the .../bar/foo@1 key holds the Service object for foo.bar.skydns.local while .../bar/foo/other key holds the Service object for other.foo.bar... ?

That could work. What happens if I provide foo@1, foo@2, foo@3 all at the same time? Multiple SRV records?

On Wed, Apr 1, 2015 at 2:37 PM, Miek Gieben notifications@github.com wrote:

No. "local/skydns/bar/foo@1 is just how it is stored in etcd, for the DNS answer everything after (and including @) is dropped. So @default would work as well.

Why I like it more is because it is more general, you can just put more Service entries in a directory (as you should)

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88638711 .

[miekg]

oh, so the .../bar/foo@1 key holds the Service object for foo.bar.skydns.local while .../bar/foo/other key holds the Service object for other.foo.bar... ?

yes, something like that, except that foo@1 would be stored as foo/@1

That could work. What happens if I provide foo@1, foo@2, foo@3 all at the same time? Multiple SRV records?

Yes, or multiple A/AAAA records, depends on what is specified in the Service.

But I need to code this and see if it would actually works and makes sense. (Looks like it does though)

[thockin]

If you code it up I am happy to test.

On Wed, Apr 1, 2015 at 2:51 PM, Miek Gieben notifications@github.com wrote:

oh, so the .../bar/foo@1 key holds the Service object for foo.bar.skydns.local while .../bar/foo/other key holds the Service object for other.foo.bar... ?

yes, something like that, except that foo@1 would be stored as foo/@1

That could work. What happens if I provide foo@1, foo@2, foo@3 all at the same time? Multiple SRV records?

Yes, or multiple A/AAAA records, depends on what is specified in the Service.

But I need to code this and see if it would actually works and makes sense. (Looks like it does though)

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-88643281 .

[miekg]

So this almost does what you need:

 curl -XPUT $ETCD/local/skydns/bar/@1 -d value='{"host":"192.168.0.1", "port":80}'
 curl -XPUT $ETCD/local/skydns/bar/f1 -d value='{"host": "bar.skydns.local", "port":443}'
 curl -XPUT $ETCD/local/skydns/bar/f2 -d value='{"host": "bar.skydns.local", "port":4443}'

almost does what you need:

;; ANSWER SECTION:
bar.skydns.local.   3600    IN  SRV 10 33 80 \@1.bar.skydns.local.
bar.skydns.local.   3600    IN  SRV 10 33 443 bar.skydns.local.
bar.skydns.local.   3600    IN  SRV 10 33 4443 bar.skydns.local.

;; ADDITIONAL SECTION:
\@1.bar.skydns.local.   3600    IN  A   192.168.0.1

Bascially what this @x syntax should do, is to say, make this Service for the directory name instead of the leave node. Have to think about all these magic chars, also see pull req #145 an #106.

[miekg]

Had a long tube ride and time to think. I think this might actually be a bug. If we synthesize a hostname for "host":"192.168.0.1", "port":80 we use the key as a base. What we should do (I think) is use the name from the question section. Also the @foo syntax gets messy fast.

The above answer would then be:

;; ANSWER SECTION:
bar.skydns.local.   3600    IN  SRV 10 33 80 bar.skydns.local.
bar.skydns.local.   3600    IN  SRV 10 33 443 bar.skydns.local.
bar.skydns.local.   3600    IN  SRV 10 33 4443 bar.skydns.local.

;; ADDITIONAL SECTION:
bar.skydns.local.   3600    IN  A   192.168.0.1

This seems be a very small change, so I will try this out somewhere today/tonight.

@bketelsen Do you some spare cycles to look through this thread?

[miekg]

See the commits on the issue-144 branch. I've added a test case that highlights the issue. I think the fundamental issue here is that you cannot control how the target of a srv record is generated. Does it a) use the key path (current behavior) or b) use the qname from the query.

That skydns does (a) is useful as well, but (b) is also nice. I think what we need a msg.Service field that allows the user to pick between the two, .i.e. something like SynFromQname that is false by default.

[miekg]

Named it TargetFromQuery. See https://github.com/skynetservices/skydns/commit/239488f745caa58282333a183b20c9b061775ae6

Still not 100% convinced some smarter is needed. Like chopping of a label from the key name... i.e. add an integer that says how many path elements should be subtracted when generating a Target for the srv record. Maybe that an even better idea...

[smarterclayton]

That would be helpful - custom backends might want to finely craft SRV records and having that flexibility is nice. Maybe the key value on msg.Service should be changed into something specific, like "ResponseName" and the etcd backend would do the key name conversion.

On Apr 3, 2015, at 8:54 AM, Miek Gieben notifications@github.com wrote:

See the commits on the issue-144 branch. I've added a test case that highlights the issue. I think the fundamental issue here is that you cannot control how the target of a srv record is generated. Does it a) use the key path (current behavior) or b) use the qname from the query.

That skydns does (a) is useful as well, but (b) is also nice. I think what we need a msg.Service field that allows the user to pick between the two, .i.e. something like SynFromQname that is false by default.

— Reply to this email directly or view it on GitHub.

[miekg]

Yes, that's the most generic case. Basically you want to register something and then tell skydns how to mangle the synthesized name you get back. Ideally you want this on a per service basis, but I'm not seeing how that would work.

[miekg]

Note completely sure we need that much flexibility (you could envision that you actually do a dns lookup to fill in the Target field of a SRV record). Anyway https://github.com/skynetservices/skydns/commit/f4f47733e63edcf8478c9f9e064671fdef563566 implements TargetStrip which strips labels from left.

[miekg]

@thockin Do you still have comments, our should I merge the TargetStrip stuff?

[thockin]

TargetStrip will make cleaner results, so I am in favor of it.

I think I have finally figured out what you guys were doing when you built it this way, and I think I get how to get what I want.

etcdctl set /skydns/local/kubernetes/th1/srvtst2/_tcp/_http/pod1

'{"host":"1.1.1.1", "port":80}' {"host":"1.1.1.1", "port":80}

etcdctl set /skydns/local/kubernetes/th1/srvtst2/_tcp/_http/pod2

'{"host":"1.1.1.2", "port":80}' {"host":"1.1.1.2", "port":80}

etcdctl set /skydns/local/kubernetes/th1/srvtst2/_tcp/_https/pod1

'{"host":"1.1.1.1", "port":443}' {"host":"1.1.1.1", "port":443}

etcdctl set /skydns/local/kubernetes/th1/srvtst2/_tcp/_https/pod2

'{"host":"1.1.1.2", "port":443}' {"host":"1.1.1.2", "port":443}

dig @10.0.0.10 +noall +answer +additional

_http._tcp.srvtst2.th1.kubernetes.local SRV _http._tcp.srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 80 pod1._http._tcp.srvtst2.th1.kubernetes.local. _http._tcp.srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 80 pod2._http._tcp.srvtst2.th1.kubernetes.local. pod1._http._tcp.srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.1 pod2._http._tcp.srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.2

dig @10.0.0.10 +noall +answer +additional

_https._tcp.srvtst2.th1.kubernetes.local SRV _https._tcp.srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 443 pod2._https._tcp.srvtst2.th1.kubernetes.local. _https._tcp.srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 443 pod1._https._tcp.srvtst2.th1.kubernetes.local. pod2._https._tcp.srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.2 pod1._https._tcp.srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.1

dig @10.0.0.10 +noall +answer +additional srvtst2.th1.kubernetes.local A

# etcdctl set /skydns/local/kubernetes/th1/srvtst2/pod1 '{"host":"1.1.1.1", "port":80}' {"host":"1.1.1.1", "port":80} # etcdctl set /skydns/local/kubernetes/th1/srvtst2/pod2 '{"host":"1.1.1.2", "port":80}' {"host":"1.1.1.2", "port":80} # dig @10.0.0.10 +noall +answer +additional srvtst2.th1.kubernetes.local A srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.1 srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.2 # dig @10.0.0.10 +noall +answer +additional srvtst2.th1.kubernetes.local SRV srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 80 pod1.srvtst2.th1.kubernetes.local. srvtst2.th1.kubernetes.local. 3600 IN SRV 10 50 80 pod2.srvtst2.th1.kubernetes.local. pod1.srvtst2.th1.kubernetes.local. 3600 IN A 1.1.1.1 On Wed, Apr 8, 2015 at 8:18 AM, Miek Gieben notifications@github.com wrote: > @thockin https://github.com/thockin Do you still have comments, our > should I merge the TargetStrip stuff? > > — > Reply to this email directly or view it on GitHub > https://github.com/skynetservices/skydns/issues/144#issuecomment-90947669 > .

[miekg]

[ Quoting notifications@github.com in "Re: [skydns] SRV for multiple ports..." ]

TargetStrip will make cleaner results, so I am in favor of it.

Ack.

I think I have finally figured out what you guys were doing when you built it this way, and I think I get how to get what I want.

See my reply to issue something something about underscored in etcd names which skews your examples here.

[SpComb]

See my comment under #150 for the multi-port host/service schema that I'm using; that example is probably even more relevant here. Your example above is similar, except I would omit the port field entirely on the {pod1,pod2}.srvtst2.th1.kubernetes.local records, and use the aforementioned hostnames for the host field in the actual _http._tcp/_https._tcp service records instead of the IP address for compatibility with glibc.

Now, trying to fit both the _http._tcp/_https._tcp service records and the host records under the same srvtst2.th1.kubernetes.local name (as opposed to a separate tree for hostnames in my example) gets more interesting... with the current etcd _hidden behavior this accidentially works since skydns ignores the _prefixed service records when doing an srvtst2.th1.kubernetes.local A query, but responds correctly to the _https._tcp.srvtst2.th1.kubernetes.local SRV query.

With PR152 fixing this behaviour, I presume that this example here would return a total of 3x2 duplicated A records in the A reply. I'm not sure what skydns would do with the indirect host service records - return a mixture of A and CNAME records in the reply?

[SpComb]

I haven't had a chance to try the TargetFromQuery/TargetStrip features, but they sound like they would help to avoid the compatibility issues with _prefix'd names, while still sticking to direct IP addresses in the SRV records. Still needs something to avoid duplicate A records when querying a service name that has multiple ports, though? And I haven't even tried to figure out what happens if you throw additional IPv6 AAAA records into the mix? :)

IMO I see the host/service split as a better approach than generating A records for ip:port service records, since I can also set sensible PTR records for the hostnames (helps to debug ss/iftop/etc output when docker containers connect to eachother) at the same time, even if a given host has multiple services.

[thockin]

Yeah #152 seems to totally break my scheme.

Simulating #152 by replacing "_" with "xx"

etcdctl rm --recursive /skydns/local/kubernetes/ns

etcdctl set /skydns/local/kubernetes/ns/svc/pod1/http '{"host":"1.1.1.1",
"port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/pod1/https '{"host":"1.1.1.1",
"port":443}'
etcdctl set /skydns/local/kubernetes/ns/svc/pod2/http '{"host":"2.2.2.2",
"port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/pod2/https '{"host":"2.2.2.2",
"port":443}'
etcdctl set /skydns/local/kubernetes/ns/svc/pod3/http '{"host":"3.3.3.3",
"port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/pod3/https '{"host":"3.3.3.3",
"port":443}'

etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttp/pod1
'{"host":"pod1.svc.ns.kubernetes.local", "port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttps/pod1
'{"host":"pod1.svc.ns.kubernetes.local", "port":443}'
etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttp/pod2
'{"host":"pod2.svc.ns.kubernetes.local", "port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttps/pod2
'{"host":"pod2.svc.ns.kubernetes.local", "port":443}'
etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttp/pod3
'{"host":"pod3.svc.ns.kubernetes.local", "port":80}'
etcdctl set /skydns/local/kubernetes/ns/svc/xxtcp/xxhttps/pod3
'{"host":"pod3.svc.ns.kubernetes.local", "port":443}'

dig @10.0.0.10 +noall +answer +additional +ignore svc.ns.kubernetes.local A
svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
svc.ns.kubernetes.local. 3600 IN CNAME pod3.svc.ns.kubernetes.local.
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
svc.ns.kubernetes.local. 3600 IN CNAME pod1.svc.ns.kubernetes.local.
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
svc.ns.kubernetes.local. 3600 IN CNAME pod2.svc.ns.kubernetes.local.
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
svc.ns.kubernetes.local. 3600 IN CNAME pod1.svc.ns.kubernetes.local.
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
svc.ns.kubernetes.local. 3600 IN CNAME pod2.svc.ns.kubernetes.local.
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
svc.ns.kubernetes.local. 3600 IN CNAME pod3.svc.ns.kubernetes.local.
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
svc.ns.kubernetes.local. 3600 IN A 2.2.2.2

dig @10.0.0.10 +noall +answer +additional +ignore
pod1.svc.ns.kubernetes.local A
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1
pod1.svc.ns.kubernetes.local. 3600 IN A 1.1.1.1

dig @10.0.0.10 +noall +answer +additional +ignore
pod2.svc.ns.kubernetes.local A
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2
pod2.svc.ns.kubernetes.local. 3600 IN A 2.2.2.2

dig @10.0.0.10 +noall +answer +additional +ignore
pod3.svc.ns.kubernetes.local A
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3
pod3.svc.ns.kubernetes.local. 3600 IN A 3.3.3.3

dig @10.0.0.10 +noall +answer +additional +ignore
xxhttp.xxtcp.svc.ns.kubernetes.local SRV
xxhttp.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 80
pod1.svc.ns.kubernetes.local.
xxhttp.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 80
pod2.svc.ns.kubernetes.local.
xxhttp.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 80
pod3.svc.ns.kubernetes.local.

dig @10.0.0.10 +noall +answer +additional +ignore
xxhttps.xxtcp.svc.ns.kubernetes.local SRV
xxhttps.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 443
pod2.svc.ns.kubernetes.local.
xxhttps.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 443
pod3.svc.ns.kubernetes.local.
xxhttps.xxtcp.svc.ns.kubernetes.local. 3600 IN SRV 10 33 443
pod1.svc.ns.kubernetes.local.

Everything works EXCEPT the first A record

On Thu, Apr 9, 2015 at 10:01 AM, Tero Marttila notifications@github.com wrote:

See my comment https://github.com/skynetservices/skydns/issues/150#issuecomment-91271940 under #150 https://github.com/skynetservices/skydns/issues/150 for the multi-port host/service schema that I'm using; that example is probably even more relevant here. Your example above is similar, except I would omit the port field entirely on the {pod1,pod2}.srvtst2.th1.kubernetes.local records, and use the aforementioned hostnames for the host field in the actual _http._tcp/_https._tcp service records instead of the IP address for compatibility with glibc.

Now, trying to fit both the _http._tcp/_https._tcp service records and the host records under the same srvtst2.th1.kubernetes.local name (as opposed to a separate tree for hostnames in my example) gets more interesting... with the current etcd _hidden behavior this accidentially works since skydns ignores the _prefixed service records when doing an srvtst2.th1.kubernetes.local A query, but responds correctly to the _https._tcp.srvtst2.th1.kubernetes.local SRV query.

With PR152 https://github.com/skynetservices/skydns/pull/152 fixing this behaviour, I presume that this example here would return a total of 3x2 duplicated A records in the A reply. I'm not sure what skydns would do with the indirect host service records - return a mixture of A and CNAME records in the reply?

— Reply to this email directly or view it on GitHub https://github.com/skynetservices/skydns/issues/144#issuecomment-91291457 .

[miekg]

[ Quoting notifications@github.com in "Re: [skydns] SRV for multiple ports..." ]

Yeah #152 seems to totally break my scheme.

Simulating #152 by replacing "_" with "xx"

Selecting the CNAME from that response:

svc.ns.kubernetes.local. 3600 IN CNAME pod3.svc.ns.kubernetes.local.
svc.ns.kubernetes.local. 3600 IN CNAME pod1.svc.ns.kubernetes.local.
svc.ns.kubernetes.local. 3600 IN CNAME pod2.svc.ns.kubernetes.local.
svc.ns.kubernetes.local. 3600 IN CNAME pod3.svc.ns.kubernetes.local.

This is another CNAME bug, because there can only be 1 CNAME per name. But there isn't a check for this. hmmm.. Maybe I should not use CNAMEs in the first place for these chains...

But indeed maybe we're saved here by Etcd handling of names starting with underscores.

Everything works EXCEPT the first A record

/Miek

Miek Gieben

[thockin]

As long as #152 does not go in, this seems to be a non-issue. Not sure if you want to fix it anyway...