search

rr-debugger / rr

Record and Replay Framework
http://rr-project.org/
Other
9.21k stars 586 forks source link

Chromium network service crashes when using headless=new with rr #3786

Open Manouchehri opened 4 months ago

Manouchehri commented 4 months ago

This only seems to happen with the new headless mode, not sure what's going on. The crash does not happen when running without rr.

[2222385:2222385:0716/184629.478804:ERROR:network_service_instance_impl.cc(600)] Network service crashed, restarting service.
rr record ./out/linux-x64-release/chrome --user-data-dir=/tmp/.chrome-user-1 --disable-crash-reporter --no-report-upload --js-flags="--expose-gc --allow-natives-syntax" --allow-running-insecure-content --allow-insecure-localhost --ignore-certificate-errors --disable-breakpad --enable-logging=stderr --v=0 --allow-sandbox-debugging --disable-gpu-sandbox --disable-seccomp-filter-sandbox --disable-setuid-sandbox --disable-namespace-sandbox --no-sandbox --disable-gpu-process-crash-limit --headless=new --timeout=5000 --dump-dom --disable-hang-monitor --vmodule=render_frame_impl=1 "https://localhost/hello_world.html"

Backtrace:

#0  content::GetNetworkService() () at ../../content/browser/network_service_instance_impl.cc:600
#1  0x0000569d147c7284 in IntranetRedirectDetector::SetupDnsConfigClient() (this=0x30d4000414a0) at ../../chrome/browser/intranet_redirect_detector.cc:249
#2  0x0000569d0def7b60 in base::OnceCallback<void ()>::Run() && (this=<optimized out>) at ../../base/functional/callback.h:156
#3  0x0000569d1546c80b in mojo::InterfaceEndpointClient::NotifyError(std::__Cr::optional<mojo::DisconnectReason> const&) (this=<optimized out>, reason=...)
    at ../../mojo/public/cpp/bindings/lib/interface_endpoint_client.cc:759
#4  0x0000569d1547a7c9 in mojo::internal::MultiplexRouter::ProcessNotifyErrorTask(mojo::internal::MultiplexRouter::Task*, mojo::internal::MultiplexRouter::ClientCallBehavior, base::SequencedTaskRunner*)
    (this=this@entry=0x30d400b6e000, task=task@entry=0x30d400365130, client_call_behavior=client_call_behavior@entry=mojo::internal::MultiplexRouter::ALLOW_DIRECT_CLIENT_CALLS, current_task_runner=current_task_runner@entry=0x1d300035ca80) at ../../mojo/public/cpp/bindings/lib/multiplex_router.cc:1034
#5  0x0000569d15476f7e in mojo::internal::MultiplexRouter::ProcessTasks(mojo::internal::MultiplexRouter::ClientCallBehavior, base::SequencedTaskRunner*)
    (this=this@entry=0x30d400b6e000, client_call_behavior=mojo::internal::MultiplexRouter::ALLOW_DIRECT_CLIENT_CALLS, current_task_runner=0x1d300035ca80)
    at ../../mojo/public/cpp/bindings/lib/multiplex_router.cc:947
#6  0x0000569d1547593f in mojo::internal::MultiplexRouter::OnPipeConnectionError(bool) (this=0x30d400b6e000, force_async_dispatch=false)
    at ../../mojo/public/cpp/bindings/lib/multiplex_router.cc:856
#7  0x0000569d0def7b60 in base::OnceCallback<void ()>::Run() && (this=<optimized out>) at ../../base/functional/callback.h:156
#8  0x0000569d15466543 in mojo::Connector::HandleError(bool, bool) (this=<optimized out>, force_pipe_reset=<optimized out>, force_async_handler=<optimized out>)
#0  content::GetNetworkService() () at ../../content/browser/network_service_instance_impl.cc:600
#1  0x0000569d147c7284 in IntranetRedirectDetector::SetupDnsConfigClient() (this=0x30d4000414a0) at ../../chrome/browser/intranet_redirect_detector.cc:249
#2  0x0000569d0def7b60 in base::OnceCallback<void ()>::Run() && (this=<optimized out>) at ../../base/functional/callback.h:156
#3  0x0000569d1546c80b in mojo::InterfaceEndpointClient::NotifyError(std::__Cr::optional<mojo::DisconnectReason> const&) (this=<optimized out>, reason=...)
    at ../../mojo/public/cpp/bindings/lib/interface_endpoint_client.cc:759
#4  0x0000569d1547a7c9 in mojo::internal::MultiplexRouter::ProcessNotifyErrorTask(mojo::internal::MultiplexRouter::Task*, mojo::internal::MultiplexRouter::ClientCallBehavior, base::SequencedTaskRunner*)
    (this=this@entry=0x30d400b6e000, task=task@entry=0x30d400365130, client_call_behavior=client_call_behavior@entry=mojo::internal::MultiplexRouter::ALLOW_DIRECT_CLIENT_CALLS, current_task_runner=current_task_runner@entry=0x1d300035ca80) at ../../mojo/public/cpp/bindings/lib/multiplex_router.cc:1034
...
cbiesinger commented 4 months ago

See https://pernos.co/debug/iJ2K_nP-o4rNlL2wvCu0tQ/index.html#f{m[Ytc,VA_,t[RQ,IzA3_,f{e[YtA,Al0_,s{aVWtk7kAA,bBQ,ud6GplQ,od6IFjw___/

khuey commented 4 months ago

Does your workload work with record -n?

cbiesinger commented 4 months ago

My workload is different from @Manouchehri but for me that does fix it. (it also slows things down enough that they time out, but that's a separate issue..)

rocallahan commented 3 months ago

I tried to reproduce this but it worked for me. Chromium revision 36986813ab32b467a9f041f65bc3cd889fe8beab, args.gn:

use_remoteexec=true
enable_nacl = false

Basically the same command line as https://github.com/rr-debugger/rr/issues/3786#issue-2411864904:

rr record ./out/Default/chrome --user-data-dir=/tmp/.chrome-user-1 --disable-crash-reporter --no-report-upload --js-flags="--expose-gc --allow-natives-syntax" --allow-running-insecure-content --allow-insecure-localhost --ignore-certificate-errors --disable-breakpad --enable-logging=stderr --v=0 --allow-sandbox-debugging --disable-gpu-sandbox --disable-seccomp-filter-sandbox --disable-setuid-sandbox --disable-namespace-sandbox --no-sandbox --disable-gpu-process-crash-limit --headless=new --timeout=5000 --dump-dom --disable-hang-monitor --vmodule=render_frame_impl=1 "https://example.com"

rr revision 172b24cb88e0a1a65238406f5174c323da8e9635.