Closed RRunner1337 closed 2 years ago
networkop
commented
2 years ago yep, I was expecting domain names to be the same as metadata labels or name: must consist of lower case alphanumeric characters
What's your use case? would it be ok if we just lowercase the value from annotation before saving it in the cache?
RRunner1337
commented
2 years ago No use case, just testing what we can and should not input.
It should be noted that there is no limitation on annotation values (https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set) as opposed to the other two types (name and namespace) which hint in documentation that they should reflect a part o domain name and hence follow the FQDN limitations.
I am fairly sure I can put in the annotation value "http://here:you@myfaivourite.value.com" :) will test it out to see what will happen. I am also sure I can configure and deploy this annotation value "abcd0123456789012345678901234567890123456789012345678901234567890" :) - I am also wandering what will happen with this one.
There is some log output for the magical one "http://here:you@myfaivourite.value.com":
[DEBUG] plugin/k8s_gateway: Adding index http://here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index http://here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index http://here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Computed Index Keys [http://here:you\@myfaivourite.value.com.k8tst.my.lan http://here:you\@myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:17286 - 263 "A IN http://here:you\@myfaivourite.value.com.k8tst.my.lan. udp 73 false 512" NXDOMAIN qr,rd 226 0.000216396sSimilar happens if you change the value to "here:you@myfaivourite.value.com"
[DEBUG] plugin/k8s_gateway: Adding index here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you@myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Computed Index Keys [here:you\@myfaivourite.value.com.k8tst.my.lan here:you\@myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:37189 - 25570 "A IN here:you\@myfaivourite.value.com.k8tst.my.lan. udp 66 false 512" NXDOMAIN qr,rd 219 0.000197397sObviously it will not work since the value is invalid. But I am surprised nslookup even allowed my to lookup this values.
This is interesting "here:you_myfaivourite.value.com"
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Computed Index Keys [here:you_myfaivourite.value.com.k8tst.my.lan here:you_myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 1 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses [10.18.12.108]
[INFO] 172.16.224.0:38946 - 20480 "A IN here:you_myfaivourite.value.com.k8tst.my.lan. udp 66 false 512" NOERROR qr,aa,rd 130 0.000232597s
[DEBUG] plugin/k8s_gateway: Computed Index Keys [here:you_myfaivourite.value.com.k8tst.my.lan here:you_myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 1 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses [10.18.12.108]
[INFO] 172.16.224.0:35150 - 57449 "AAAA IN here:you_myfaivourite.value.com.k8tst.my.lan. udp 66 false 512" NOERROR qr,rd 219 0.000139498sactually resolves :) - not sure that ":" is a valid per DNS name spec.
This is something I would think of "myfaivourite.value.com":
[DEBUG] plugin/k8s_gateway: Adding index myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Computed Index Keys [myfaivourite.value.com.k8tst.my.lan myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 1 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses [10.18.12.108]
[INFO] 172.16.224.0:6499 - 3831 "A IN myfaivourite.value.com.k8tst.my.lan. udp 57 false 512" NOERROR qr,aa,rd 112 0.000177197s
[DEBUG] plugin/k8s_gateway: Computed Index Keys [myfaivourite.value.com.k8tst.my.lan myfaivourite.value.com]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 1 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses [10.18.12.108]
[INFO] 172.16.224.0:53314 - 623 "AAAA IN myfaivourite.value.com.k8tst.my.lan. udp 57 false 512" NOERROR qr,rd 210 0.000187697sIt actually resolves. Probably this is desired behavior.
The out of spec one "abcd0123456789012345678901234567890123456789012345678901234567890" (64 bytes of first DNS label - limited to 63)
[DEBUG] plugin/k8s_gateway: Adding index abcd0123456789012345678901234567890123456789012345678901234567890 for service www5
[DEBUG] plugin/k8s_gateway: Adding index abcd0123456789012345678901234567890123456789012345678901234567890 for service www5
[DEBUG] plugin/k8s_gateway: Adding index abcd0123456789012345678901234567890123456789012345678901234567890 for service www5Depending on the tool - but same information I would say Linux (cygwin):
$ nslookup abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan 10.18.12.101
nslookup: 'abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan' is not a legal IDN name (domain label longer than 63 characters), use +noidnin
$ dig abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan @10.18.12.101
dig: 'abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan' is not a legal IDN name (domain label longer than 63 characters), use +noidninnothing happens on the gateway which is ok since the tool does not send the request.
Windows:
C:\Users\myUser>nslookup abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan 10.18.12.101
Server: UnKnown
Address: 10.18.12.101
*** UnKnown can't find abcd0123456789012345678901234567890123456789012345678901234567890.k8tst.my.lan: Unspecified errorand log from k8s_gateway:
[INFO] 172.16.224.0:17588 - 1 "PTR IN 101.12.18.10.in-addr.arpa. udp 43 false 512" - - 0 0.000074699s
[ERROR] plugin/errors: 2 101.12.18.10.in-addr.arpa. PTR: plugin/loop: no next plugin foundNot sure what is happening here, but seems to be in range of acceptable.
And the non ASCII characters :) "ôóčćšž12345"?
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Computed Index Keys [xn--12345-ztag7m3b58cej.k8tst.my.lan xn--12345-ztag7m3b58cej]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:25624 - 5601 "A IN xn--12345-ztag7m3b58cej.k8tst.my.lan. udp 58 false 512" NXDOMAIN qr,rd 211 0.000241796s
[DEBUG] plugin/k8s_gateway: Computed Index Keys [xn--12345-ztag7m3b58cej.k8tst.my.lan xn--12345-ztag7m3b58cej]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:34765 - 64092 "A IN xn--12345-ztag7m3b58cej.k8tst.my.lan. udp 81 false 4096" NXDOMAIN qr,rd 211 0.000191996sDoes not resolve, but it is queried :)
I would suggest a validation of the annotation value (RFC 1035 - section 2.3.1 and RFC 2181 - section 11).
I would also suggest to validate the length of the fully setup DNS record - probably for all types you support (should not be greater than 253 characters if I understand everything correctly - first DNS label should not be larger than 63 bytes I believe).
As per character case treatment I do expect that DNS names are treated in all lower case. Weather that means a failure to deploy or fixing the business logic to check everything in lower case is up to you :)
networkop
commented
2 years ago I've done a few changes in 0.3.2 that should address most of the serious issues. It should be checking the length of the DNS label and treat upper/lower case similarly now. let me know when you test it.
RRunner1337
commented
2 years ago Upper case input seems to be fixed. I don't see to invalid DNS names to be present in the index and fallback to www5.k8s-dns-test is a perfect solution. Maybe a WARN output in log that this was done would be efficient (invalid domain name in annotation :
[DEBUG] plugin/k8s_gateway: Invalid domain name in annotation: abcd0123456789012345678901234567890123456789012345678901234567890
[DEBUG] plugin/k8s_gateway: Adding index www5.k8s-dns-test for service www5
[DEBUG] plugin/k8s_gateway: Invalid domain name in annotation: abcd0123456789012345678901234567890123456789012345678901234567890
[DEBUG] plugin/k8s_gateway: Adding index www5.k8s-dns-test for service www5
[DEBUG] plugin/k8s_gateway: Invalid domain name in annotation: abcd0123456789012345678901234567890123456789012345678901234567890
[DEBUG] plugin/k8s_gateway: Adding index www5.k8s-dns-test for service www5
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7
[DEBUG] plugin/k8s_gateway: Adding index here:you_myfaivourite.value.com for service www7Still I do see "here:you_myfaivourite.value.com" and "ôóčćšž12345" which IMHO would better fallback to www6 and www7.
networkop
commented
2 years ago I think falling back here would be a bad idea. If someone specified a hostname in annotation, that means they want to use that value, falling back could be an unexpected side effect. Regarding "here:you_myfaivourite.value.com" and "ôóčćšž12345", how do you know they are not valid domain names? based on https://www.rfc-editor.org/rfc/rfc2181#section-11, the only restriction is the length, not the value:
The DNS itself places only one restriction on the particular labels
that can be used to identify resource records. That one restriction
relates to the length of the label and the full name. The length of
any one label is limited to between 1 and 63 octets. A full domain
name is limited to 255 octets (including the separators). The zero
length full name is defined as representing the root of the DNS tree,
and is typically written and displayed as ".". Those restrictions
aside, any binary string whatever can be used as the label of any
resource record. Similarly, any binary string can serve as the value
of any record that includes a domain name as some or all of its value
(SOA, NS, MX, PTR, CNAME, and any others that may be added).
Implementations of the DNS protocols must not place any restrictions
on the labels that can be used. In particular, DNS servers must not
refuse to serve a zone because it contains labels that might not be
acceptable to some DNS client programs. A DNS server may be
configurable to issue warnings when loading, or even to refuse to
load, a primary zone containing labels that might be considered
questionable, however this should not happen by default.Granted that is true ... Although I must warn you that RFC2181 was superseded by RFC4343. As I recall RFC4343 refers to some other RFC's that deal with the internationalization of domain names. If this is a hole you want to drop into then there will need to be a consensus how to find the "ôóčćšž12345.k8tst.my.lan" host's IP on the g8s_gateway ... I believe there is a UNICODE mapping that should be defined and implemented for internationalized domain names. Nslookup, dig, browsers implement it the same way and I am guessing that you can implement the translation (or call a go function - should be related to "stringprep" or something) of the DNS labels of "ôóčćšž12345" to "xn--12345-ztag7m3b58cej" when you discover them in annotation based host names.
So this issue;
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Adding index ôóčćšž12345 for service www6
[DEBUG] plugin/k8s_gateway: Computed Index Keys [xn--12345-ztag7m3b58cej.k8tst.my.lan xn--12345-ztag7m3b58cej]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:25624 - 5601 "A IN xn--12345-ztag7m3b58cej.k8tst.my.lan. udp 58 false 512" NXDOMAIN qr,rd 211 0.000241796s
[DEBUG] plugin/k8s_gateway: Computed Index Keys [xn--12345-ztag7m3b58cej.k8tst.my.lan xn--12345-ztag7m3b58cej]
[DEBUG] plugin/k8s_gateway: Found 0 matching Ingress objects
[DEBUG] plugin/k8s_gateway: Found 0 matching Service objects
[DEBUG] plugin/k8s_gateway: Computed response addresses []
[INFO] 172.16.224.0:34765 - 64092 "A IN xn--12345-ztag7m3b58cej.k8tst.my.lan. udp 81 false 4096" NXDOMAIN qr,rd 211 0.000191996swill be resolved.
The issue I still have is that people can really type in anything - there is no limitation on annotation in Kubernetes so - "username:password@mylabel" will probably fail though and full URI's like "ftp://username:password@mylabel2" will probably fail too - so fallback to name.namespace should be warned in those cases. Granted it is a user error and IMHO if it is possible I would fail creation of the resource on cluster :)
Although I will still point out the updated words of wisdom (RFC1035 - DOMAIN NAMES - IMPLEMENTATION AND SPECIFICATION - 2.3.1. Preferred name syntax)
The following syntax will result in fewer problems with many
applications that use domain names (e.g., mail, TELNET).
<domain> ::= <subdomain> | " "
<subdomain> ::= <label> | <subdomain> "." <label>
<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]
<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>
<let-dig-hyp> ::= <let-dig> | "-"
<let-dig> ::= <letter> | <digit>
<letter> ::= any one of the 52 alphabetic characters A through Z in
upper case and a through z in lower case
<digit> ::= any one of the ten digits 0 through 9
Note that while upper and lower case letters are allowed in domain
names, no significance is attached to the case. That is, two names with
the same spelling but different case are to be treated as if identical.and then the same RFC section 3-1:
[3.1](https://www.rfc-editor.org/rfc/rfc1035#section-3.1). Name space definitions
Domain names in messages are expressed in terms of a sequence of labels.
Each label is represented as a one octet length field followed by that
number of octets. Since every domain name ends with the null label of
the root, a domain name is terminated by a length byte of zero. The
high order two bits of every length octet must be zero, and the
remaining six bits of the length field limit the label to 63 octets or
less.
To simplify implementations, the total length of a domain name (i.e.,
label octets and label length octets) is restricted to 255 octets or
less.
Although labels can contain any 8 bit values in octets that make up a
label, it is strongly recommended that labels follow the preferred
syntax described elsewhere in this memo, which is compatible with
existing host naming conventions. Name servers and resolvers must
compare labels in a case-insensitive manner (i.e., A=a), assuming ASCII
with zero parity. Non-alphabetic codes must match exactly.I still believe that this is mostly true and should be reflected and RFC dealing with the update should take this specification into consideration (since the following RFC's update this RFC and not obsolete it). Based on this specification and understanding that this part of RFC is still in effect (in case it was not updated somewhere) and DNS labels should follow the "preferred syntax" (section 2.3.1) which can't contain ":" character and should be limited to [a-zA-z0-9-]. Maybe this was updated somewhere. And I do understand that you MUST take a "should follow" under consideration and not follow it - and there is where RFC 2181 - Section 11 really spills the beans and empowers implementation to do the right thing regarding "questionable" labels. Now the philosophical question that offers itself is: "Which labels are "questionable"?". Maybe RFC4343 offers some answers.
networkop
commented
2 years ago ok, so I definitely don't want to implement anything complicated for this, rather niche, feature. Since annotations are less strict than labels, there's not much I can do to prevent user error. I can't fail the creation of a resource (I don't manage any of them), but I can log that I failed to create a DNS entry for it.
WRT the hostnames, what I can do is borrow the label checking logic from the kubernetes code and apply it to k8s_gateway annotations. That means, I would restrict the hostname to the following regex ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$. This should be fairly simple.
RRunner1337
commented
2 years ago Should work ... and should fix all the above problems :)
networkop
commented
2 years ago should be fixed in the #144. @RRunner1337 are you able to test?
RRunner1337
commented
2 years ago should be fixed in the #144. @RRunner1337 are you able to test?
I would love to but I don't have a "go build" environment :/ could you build and release it. I checked the code that was changed and I believe it should do the trick.
networkop
commented
2 years ago The 070f8b7 tag now includes the fix. Let me know once you've tested it and I'll cut a new release.
I have setup k8s_gateway on my Kubernetes server instance and have deployed this configuration:
When redeploying this manifest I get the following DEBUG output in the k8s_gateway:
So far so good ... then I do dig queries directed to the externally available server and I get the following results:
Dig query
dig scBRB-www.k8tst.my.lan @hostordig scbrb-www.k8tst.my.lan @hostfails with 0 Service objects.
Interesting enough changing upper case letters to lower case letters in annotation seems to solve the issue. Any chance to fix this?