search

metalbear-co / mirrord

Connect your local process and your cloud environment, and run local code in cloud conditions.
https://mirrord.dev
MIT License
3.76k stars 103 forks source link

Golang 1.23 mirrord steal stopped working #2680

Closed drehelis closed 1 month ago

drehelis commented 1 month ago

Bug Description

After upgrading our dev containers to go1.23, mirrord is is unable to steal the workload. Replacing the go binary with 1.22.6 works.

Steps to Reproduce

  1. In devcontainer, with go1.23 in place, run debug and fails.
  2. Downgrade go to 1.22 latest and it works.

Relevant Logs

Starting: /go/bin/dlv dap --listen=127.0.0.1:39361 --log-dest=3 from /workspaces/cerberus
DAP server listening at: 127.0.0.1:39361
Type 'dlv help' for list of commands.
2024-08-19T10:59:33.618899Z TRACE ThreadId(01) mirrord_layer::load: Loading into process: __debug_bin3609130236 invoked as __debug_bin3609130236.
2024-08-19T10:59:33.622467Z TRACE ThreadId(01) mirrord_layer: hooked "close"
2024-08-19T10:59:33.622589Z TRACE ThreadId(01) mirrord_layer: hooked "__close_nocancel"
2024-08-19T10:59:33.622594Z TRACE ThreadId(01) mirrord_layer::hooks: found "__close" in "libc.so.6", hooking
2024-08-19T10:59:33.622596Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__close" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.622599Z TRACE ThreadId(01) mirrord_layer::hooks: found "__close" in "libpthread.so.0", hooking
2024-08-19T10:59:33.622600Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__close" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.622602Z TRACE ThreadId(01) mirrord_layer::hooks: found "__close" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.622603Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__close" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.622605Z TRACE ThreadId(01) mirrord_layer::hooks: found "__close" in "libm.so.6", hooking
2024-08-19T10:59:33.622606Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__close" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.622608Z TRACE ThreadId(01) mirrord_layer::hooks: found "__close" in "libdl.so.2", hooking
2024-08-19T10:59:33.622609Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__close" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.623011Z TRACE ThreadId(01) mirrord_layer: hooked "fork"
2024-08-19T10:59:33.623142Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "socket"
2024-08-19T10:59:33.623333Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "recvfrom"
2024-08-19T10:59:33.623525Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "sendto"
2024-08-19T10:59:33.623708Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "recvmsg"
2024-08-19T10:59:33.623906Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "sendmsg"
2024-08-19T10:59:33.624001Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "bind"
2024-08-19T10:59:33.624093Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "listen"
2024-08-19T10:59:33.624290Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "connect"
2024-08-19T10:59:33.624553Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "fcntl"
2024-08-19T10:59:33.624668Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "dup"
2024-08-19T10:59:33.624781Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "dup2"
2024-08-19T10:59:33.624879Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "getpeername"
2024-08-19T10:59:33.624993Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "getsockname"
2024-08-19T10:59:33.625234Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "gethostname"
2024-08-19T10:59:33.625425Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "accept4"
2024-08-19T10:59:33.625527Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "dup3"
2024-08-19T10:59:33.625856Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "accept"
2024-08-19T10:59:33.626454Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "gethostbyname"
2024-08-19T10:59:33.627007Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "getaddrinfo"
2024-08-19T10:59:33.627230Z TRACE ThreadId(01) mirrord_layer::socket::hooks: hooked "freeaddrinfo"
2024-08-19T10:59:33.627335Z TRACE ThreadId(01) mirrord_layer::exec_hooks::hooks: hooked "execv"
2024-08-19T10:59:33.627447Z TRACE ThreadId(01) mirrord_layer::exec_hooks::hooks: hooked "execve"
2024-08-19T10:59:33.627997Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "open"
2024-08-19T10:59:33.628024Z TRACE ThreadId(01) mirrord_layer::hooks: found "open64" in "libc.so.6", hooking
2024-08-19T10:59:33.628027Z TRACE ThreadId(01) mirrord_layer::hooks: hook "open64" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.628029Z TRACE ThreadId(01) mirrord_layer::hooks: found "open64" in "libpthread.so.0", hooking
2024-08-19T10:59:33.628030Z TRACE ThreadId(01) mirrord_layer::hooks: hook "open64" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.628032Z TRACE ThreadId(01) mirrord_layer::hooks: found "open64" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.628033Z TRACE ThreadId(01) mirrord_layer::hooks: hook "open64" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.628035Z TRACE ThreadId(01) mirrord_layer::hooks: found "open64" in "libm.so.6", hooking
2024-08-19T10:59:33.628036Z TRACE ThreadId(01) mirrord_layer::hooks: hook "open64" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.628037Z TRACE ThreadId(01) mirrord_layer::hooks: found "open64" in "libdl.so.2", hooking
2024-08-19T10:59:33.628039Z TRACE ThreadId(01) mirrord_layer::hooks: hook "open64" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.628699Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "openat"
2024-08-19T10:59:33.628716Z TRACE ThreadId(01) mirrord_layer::hooks: found "openat64" in "libc.so.6", hooking
2024-08-19T10:59:33.628718Z TRACE ThreadId(01) mirrord_layer::hooks: hook "openat64" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.628720Z TRACE ThreadId(01) mirrord_layer::hooks: found "openat64" in "libpthread.so.0", hooking
2024-08-19T10:59:33.628721Z TRACE ThreadId(01) mirrord_layer::hooks: hook "openat64" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.628722Z TRACE ThreadId(01) mirrord_layer::hooks: found "openat64" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.628724Z TRACE ThreadId(01) mirrord_layer::hooks: hook "openat64" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.628725Z TRACE ThreadId(01) mirrord_layer::hooks: found "openat64" in "libm.so.6", hooking
2024-08-19T10:59:33.628726Z TRACE ThreadId(01) mirrord_layer::hooks: hook "openat64" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.628728Z TRACE ThreadId(01) mirrord_layer::hooks: found "openat64" in "libdl.so.2", hooking
2024-08-19T10:59:33.628729Z TRACE ThreadId(01) mirrord_layer::hooks: hook "openat64" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.628948Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "read"
2024-08-19T10:59:33.629222Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "closedir"
2024-08-19T10:59:33.629290Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "dirfd"
2024-08-19T10:59:33.629468Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "pread"
2024-08-19T10:59:33.629640Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "readv"
2024-08-19T10:59:33.629957Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "preadv"
2024-08-19T10:59:33.630120Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "readlink"
2024-08-19T10:59:33.630229Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "lseek"
2024-08-19T10:59:33.630398Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "write"
2024-08-19T10:59:33.630603Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "pwrite"
2024-08-19T10:59:33.630715Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "access"
2024-08-19T10:59:33.631402Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "faccessat"
2024-08-19T10:59:33.631563Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fsync"
2024-08-19T10:59:33.631712Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fdatasync"
2024-08-19T10:59:33.632181Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "realpath"
2024-08-19T10:59:33.632782Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "statx"
2024-08-19T10:59:33.632909Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "__xstat"
2024-08-19T10:59:33.632925Z TRACE ThreadId(01) mirrord_layer::hooks: found "__xstat64" in "libc.so.6", hooking
2024-08-19T10:59:33.632927Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__xstat64" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.632929Z TRACE ThreadId(01) mirrord_layer::hooks: found "__xstat64" in "libpthread.so.0", hooking
2024-08-19T10:59:33.632930Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__xstat64" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.632932Z TRACE ThreadId(01) mirrord_layer::hooks: found "__xstat64" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.632933Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__xstat64" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.632934Z TRACE ThreadId(01) mirrord_layer::hooks: found "__xstat64" in "libm.so.6", hooking
2024-08-19T10:59:33.632935Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__xstat64" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.632936Z TRACE ThreadId(01) mirrord_layer::hooks: found "__xstat64" in "libdl.so.2", hooking
2024-08-19T10:59:33.632938Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__xstat64" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.633021Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "__lxstat"
2024-08-19T10:59:33.633024Z TRACE ThreadId(01) mirrord_layer::hooks: found "__lxstat64" in "libc.so.6", hooking
2024-08-19T10:59:33.633025Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__lxstat64" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.633027Z TRACE ThreadId(01) mirrord_layer::hooks: found "__lxstat64" in "libpthread.so.0", hooking
2024-08-19T10:59:33.633028Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__lxstat64" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.633029Z TRACE ThreadId(01) mirrord_layer::hooks: found "__lxstat64" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.633030Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__lxstat64" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.633031Z TRACE ThreadId(01) mirrord_layer::hooks: found "__lxstat64" in "libm.so.6", hooking
2024-08-19T10:59:33.633032Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__lxstat64" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.633034Z TRACE ThreadId(01) mirrord_layer::hooks: found "__lxstat64" in "libdl.so.2", hooking
2024-08-19T10:59:33.633035Z TRACE ThreadId(01) mirrord_layer::hooks: hook "__lxstat64" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.633198Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "lstat"
2024-08-19T10:59:33.633361Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fstat"
2024-08-19T10:59:33.633513Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "stat"
2024-08-19T10:59:33.633606Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fstatat"
2024-08-19T10:59:33.633699Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fstatfs"
2024-08-19T10:59:33.634202Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "fdopendir"
2024-08-19T10:59:33.634634Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "readdir_r"
2024-08-19T10:59:33.634645Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir64_r" in "libc.so.6", hooking
2024-08-19T10:59:33.634647Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir64_r" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.634649Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir64_r" in "libpthread.so.0", hooking
2024-08-19T10:59:33.634650Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir64_r" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.634651Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir64_r" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.634653Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir64_r" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.634654Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir64_r" in "libm.so.6", hooking
2024-08-19T10:59:33.634655Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir64_r" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.634656Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir64_r" in "libdl.so.2", hooking
2024-08-19T10:59:33.634658Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir64_r" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.634898Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "readdir64"
2024-08-19T10:59:33.634911Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir" in "libc.so.6", hooking
2024-08-19T10:59:33.634913Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir" in "libc.so.6" failed with err Function already replaced
2024-08-19T10:59:33.634914Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir" in "libpthread.so.0", hooking
2024-08-19T10:59:33.634915Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir" in "libpthread.so.0" failed with err Function already replaced
2024-08-19T10:59:33.634916Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir" in "libgcc_s.so.1", hooking
2024-08-19T10:59:33.634917Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir" in "libgcc_s.so.1" failed with err Function already replaced
2024-08-19T10:59:33.634919Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir" in "libm.so.6", hooking
2024-08-19T10:59:33.634920Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir" in "libm.so.6" failed with err Function already replaced
2024-08-19T10:59:33.634921Z TRACE ThreadId(01) mirrord_layer::hooks: found "readdir" in "libdl.so.2", hooking
2024-08-19T10:59:33.634922Z TRACE ThreadId(01) mirrord_layer::hooks: hook "readdir" in "libdl.so.2" failed with err Function already replaced
2024-08-19T10:59:33.635178Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "opendir"
2024-08-19T10:59:33.641886Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: found version >= 1.23
2024-08-19T10:59:33.648427Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: hook "internal/runtime/syscall.Syscall6" in main module failed with err NoSymbolName("internal/runtime/syscall.Syscall6")
2024-08-19T10:59:33.648647Z  INFO ThreadId(01) mirrord_layer: Initializing mirrord-layer!
2024-08-19T10:59:33.648701Z DEBUG ThreadId(01) mirrord_layer: Loaded into executable ...
...
...

Your operating system and version

MacOS - devcontainer

Local process

internal binary

Local process version

$(find ~ -executable -type f -name mirrord) --version
mirrord 3.114.1

Additional Info

No response

drehelis commented 1 month ago

Tried mirrord-3.113.1 -

2024-08-19T11:18:34.347018Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "opendir"
2024-08-19T11:18:34.353332Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: found version >= 1.23
2024-08-19T11:18:34.359899Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: hook "internal/runtime/syscall.Syscall6" in main module failed with err NoSymbolName("internal/runtime/syscall.Syscall6")
2024-08-19T11:18:34.360109Z  INFO ThreadId(01) mirrord_layer: Initializing mirrord-layer!

and also mirrord-3.113.0 -

2024-08-19T11:20:57.417440Z TRACE ThreadId(01) mirrord_layer::file::hooks: hooked "opendir"
2024-08-19T11:20:57.424759Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: found version >= 1.19
2024-08-19T11:20:57.431486Z TRACE ThreadId(01) mirrord_layer::go::go_hooks: hook "runtime/internal/syscall.Syscall6.abi0" in main module failed with err NoSymbolName("runtime/internal/syscall.Syscall6.abi0")
2024-08-19T11:20:57.431724Z  INFO ThreadId(01) mirrord_layer: Initializing mirrord-layer!