Closed aviramha closed 1 week ago
Possible solutions:
SOCKET/FILES
fd struct as environment variable for fds that aren't CLOEXECSOCKET/FILES
(dunno about this..)listen
with missing fd (this happens on uvicorn + reloader case) use getsockname to resolve addr and then decide if to add the socket and backfill the information. This is pretty good approach but has some caveats:
Please let us know if this is impacting anyone so we can know if we need to prioritize it.
This would probably affect me if I were not already blocked by this issue
I'm not blocked by this, but it does affect my productivity. MacOS M1 Pro
I'm not blocked by this, but it does affect my productivity.
MacOS M1 Pro
Hey, does it still happen to you on latest version?
I'm running 3.75.0
and I'm not getting traffic with --reload
and I do get traffic without it
Also doesn't work when using Uvicorn with more than one worker process.
Happening to me on 3.84.1. I am not using the operator. Not sure if that matters.
Ran into the same issue. Might be worth flagging this in the docs somewhere until its fixed
I've been exploring this, and here are a few solutions I can think of:
The main issue we have is that our sockfd
s are stored in a per-layer global SOCKETS
, which are not being shared between forks. So the proposal here is to just move the SOCKETS
global to the intproxy
, and change every operation that calls SOCKETS.[get|take](&sockfd)
to use make_proxy_request_with_response
(or something similar).
A problem with this solution is that now every sockfd
becomes shared between exec
s and fork
s, which is not ideal. Another (minor) problem is that we must manually remember to call insert
after take
, as some operations call SOCKETS.take
to modify the socket, meh.
Instead of moving the whole SOCKETS
to intproxy
, whenever we detect a fork
(new layer connection?), we send a copy of the SOCKETS
minus the ones with CLOEXEC
to the intproxy
, which in turn sets the new layer connection's SOCKETS
to this curated copy.
Delegate more work to the intproxy
, by moving the socket/ops
there, and creating some sort of MirrordSocket
API that keeps an association between a sockfd
and the pid
that created it, so we can avoid leaking sockfd
s.
That's it, so far. I'm leaning more towards the :donkey: solution, as :earth_americas: is a bug waiting to happen, and :peach: is too big of a pointless refactor.
🫏 is what would make most sense and is how it works in terms of OS level too.
We can actually pass all SOCKETS
downstream and iterate each fd and see if it's still open (so we'll just follow OS logic if cloexec or not)
Fixed in #2610
Looks like the problem is still there on macOS + vscode.
Bug Description
When a parent process binds a socket, then children sockets listen/accept on that socket mirrord doesn't intercept those calls because the fd isn't managed (doesn't exist in our fork). Using this issue to document use cases where this happens for now:
--reload
flag.Steps to Reproduce
run with
mirrord exec --target pod/pod uvicorn -- --reload --port 80 main:app
Backtrace
No response
Relevant Logs
No response
Your operating system and version
macOS
Local process
python
Local process version
No response
Additional Info
No response