Closed eyalb181 closed 1 year ago
This would be great to have. We have an external API we use, and we have to allow list specific IPs. So we've allowlisted our k8s cluster. I want to be able to call that API, but the code will also write to the db, and I want to write to my local db not the remote one
My usecase is https://www.prisma.io/ which starts an "engine" that listens on a random port (it seems to me). Which makes it impossible to use, as the "in cluster pod" might not be listening on exactly the same port 😦
I have implemented https://github.com/metalbear-co/mirrord/pull/1154 until we get this done fully. Next version you'll be able to control if outbound sockets to localhost would go through the pod or locally, same for listening sockets.
We need to decide whether we leave this config after we add the full filtering functionality or let users use the new one.
This issue doesn't cover Incoming
so we should consider that too.
Another usacese we have is:
We are running mirrord in docker container (A). We are running a database in a different docker container (B). We want A to be able to connect to B.
This is currently not possible. As A would route all traffic via K8S, and K8S can't access B, as B is on the developers laptop not in the K8S.
To allow us to have mixed remote and local resources that we can connect to.
Part of this was implemented in #1607!
Named hosts are coming in #1648.
I think part of DNS resolving should be done in the DNS feature itself, we could expand it to also be filter-like, and have dns: ["cluster-service". "foo.com"]
resolve remotely, while everything else resolves locally.
Doing so would help in the case where the user has (outgoing) local: ["localhost"]
, and the app tries to connect("localhost")
, but if they have DNS enabled, it resolves to some cluster address, thus breaking the request/traffic.
Makes sense to me. Should we still let the user configure a hostname filter under outgoing
, though? Shouldn't it be moved entirely to the dns
configuration field?
and have
dns: ["cluster-service". "foo.com"]
resolve remotely
I don't think we need an extra list, I think we can use the outgoing filter. DNS only matters for outgoing traffic, so all names you are connecting to remotely should be resolved remotely and all names you are connecting to locally should be resolved locally, right?
Yup, my mistake, you're right
Putting this in the outgoing filter feels confusing to me, for example:
dns = on
local = ["service-a", "service-b"]
// service-a becomes 1.2.3.4:0
// service-b becomes 1.2.3.4:0
// service-c (which we want remote) becomes 1.2.3.4:0
dns = off
local = ["service-a", "service-b"]
// service-a becomes 127.0.0.1:0
// service-b becomes 127.0.0.1:0
// service-c (which we want remote) becomes 127.0.0.1:0
This is kinda what I expect, when I enable DNS resolving, I want things to be resolved on the agent. So changing the DNS behavior on the filter config like this:
dns = on
local = ["service-a", "service-b"]
// service-a becomes 127.0.01:0
// service-b becomes 127.0.0.1:0
// service-c (which we want remote) becomes 1.2.3.4:0
Feels a bit like the outgoing filter is going over its boundaries? Like, the 2 features enter in a bit of a contradiction (subverts expectations)? What do you guys think?
In my opinion, we should put DNS handling stuff in the DNS feature, the user would need something like:
dns.local = ["service-a", "service-b"]
local = ["service-a", "service-b"]
Is explicit better than implicit here?
I think we definitely don't want the user to have to specify two lists.
dns
field is going to be omitted entirely, so users aren't going to be like "but I asked for DNS to be done remotely".What I'd do is show a warning if the user specified DNS on/off explicitly, and also specified a hostname in the filter.
tal's suggestion link:
On every outgoing connection - we send out DNS queries (probably via the agent) for all the names in the outgoing filter. I think alternatively - we could avoid making any extra DNS queries and just check in the getaddrinfo hook if the queried name is in the filter - and resolve that IP in the filter if it is. That would mean the feature is slightly different - we would only filter out connections by name if the application looked them up by name, but this makes sense in my opinion.
Agree with the last line, @aviramha wdyt? So getaddrinfo would read the filter and use it to decide whether to resolve locally or remotely?
Sounds good
The filter should now be supporting almost everything listed in the issue, except for custom resolvers:
Go/Custom DNS resolving (using UDP instead of just libc's getaddrinfo etc) - harder to cover, need to do some sort of MITM for the resolving. Can be deferred for a follow up issue.
A little bit of stateful DNS was added, to make the filter behave nicely with a mix of DNS and other ip/port configs, not the full thing mentioned in the issue though (there is no socket<->dns state).
Closing this in favor of #1826
We should let users enable outgoing traffic, but then specify a list of IPs for which outgoing traffic would remain local.
Motivation
We want to let users mix and match resources to be used, locally and remote.
Implementation
outgoing
config should support the fieldremote
andlocal
, each overriding the default behavior of the general setting. The value for these fields will be a list of items with the following format:[PROTOCOL://]IP/[subnet]:PORT
HOST:PORT
:PORT
HOST
For example:
will lead to the following behavior:
Details
This issue should solve the following use cases:
Warnings
This feature has some sharp edges that might hurt, those edges come to mind:
google.com
we would want to passthrough connections to it but we need to maintain some state dns <-> sockets that doesn't exist currently -> can be deferred to another issue