Open aarroisi opened 4 months ago
Thanks for the report! I'll get a chance to look at this later this week.
Having a repro case provided makes my job about 1000x easier. Thanks for that!
I can repro this on your instance, but I'm unable to repro this locally or via a new fly instance.
In the cases where I can't repro, this seems to be happening:
When it goes wrong, what seems to happen is that Safari doesn't get a response back when trying to create that new websocket connection, falls back to long polling, which also fails. At this point, signs would point to something funny with your fly config, or perhaps fly itself. As I'm not able to repro this myself, I wonder if you have any special fly config in that respect, or if you can provide both your fly logs and screenshots of your safari inspector (along with the 'preview' panes for any of the websocket connections displaying the back and forth websocket messages with timestamps).
Any progress on getting the above debug info @aarroisi ?
@mtrudel Hi, so sorry for the late reply. Here is the log from fly. It started with me opening https://example-aarroisi.fly.dev/posts, put it in background for about a minute, then opened Safari again. It showed error reconnecting message, then eventually refreshed the whole page.
2024-06-08T01:44:09.175 app[e82de11b774628] sin [info] 01:44:09.174 request_id=F9bkDEFCzjzEQjUAAAHB [info] GET /posts
2024-06-08T01:44:09.178 app[e82de11b774628] sin [info] 01:44:09.177 request_id=F9bkDEFCzjzEQjUAAAHB [info] Sent 200 in 2ms
2024-06-08T01:44:09.841 app[e82de11b774628] sin [info] 01:44:09.840 [info] CONNECTED TO Phoenix.LiveView.Socket in 22µs
2024-06-08T01:44:09.841 app[e82de11b774628] sin [info] Transport: :websocket
2024-06-08T01:44:09.841 app[e82de11b774628] sin [info] Serializer: Phoenix.Socket.V2.JSONSerializer
2024-06-08T01:44:09.841 app[e82de11b774628] sin [info] Parameters: %{"_csrf_token" => "bTBzIQ4nAR8ZOCA5ZippAk45MgETKiB52dAYlAhfAMqx0EDAvjsuuaZ5", "_live_referer" => "undefined", "_mounts" => "0", "_track_static" => %{"0" => "https://example-aarroisi.fly.dev/assets/app-a5fe870326318b359c192ce86b220d13.css?vsn=d", "1" => "https://example-aarroisi.fly.dev/assets/app-a7b7a1bb2570182b93179aee6863d1bb.js?vsn=d"}, "vsn" => "2.0.0"}
2024-06-08T01:45:24.299 app[e82de11b774628] sin [info] 01:45:24.298 [info] CONNECTED TO Phoenix.LiveView.Socket in 20µs
2024-06-08T01:45:24.299 app[e82de11b774628] sin [info] Transport: :websocket
2024-06-08T01:45:24.299 app[e82de11b774628] sin [info] Serializer: Phoenix.Socket.V2.JSONSerializer
2024-06-08T01:45:24.299 app[e82de11b774628] sin [info] Parameters: %{"_csrf_token" => "bTBzIQ4nAR8ZOCA5ZippAk45MgETKiB52dAYlAhfAMqx0EDAvjsuuaZ5", "_live_referer" => "undefined", "_mounts" => "0", "_track_static" => %{"0" => "https://example-aarroisi.fly.dev/assets/app-a5fe870326318b359c192ce86b220d13.css?vsn=d", "1" => "https://example-aarroisi.fly.dev/assets/app-a7b7a1bb2570182b93179aee6863d1bb.js?vsn=d"}, "vsn" => "2.0.0"}
2024-06-08T01:45:27.297 app[e82de11b774628] sin [info] 01:45:27.296 [info] CONNECTED TO Phoenix.LiveView.Socket in 43µs
2024-06-08T01:45:27.297 app[e82de11b774628] sin [info] Transport: :longpoll
2024-06-08T01:45:27.297 app[e82de11b774628] sin [info] Serializer: Phoenix.Socket.V2.JSONSerializer
2024-06-08T01:45:27.297 app[e82de11b774628] sin [info] Parameters: %{"_csrf_token" => "bTBzIQ4nAR8ZOCA5ZippAk45MgETKiB52dAYlAhfAMqx0EDAvjsuuaZ5", "_live_referer" => "undefined", "_mounts" => "1", "_track_static" => %{"0" => "https://example-aarroisi.fly.dev/assets/app-a5fe870326318b359c192ce86b220d13.css?vsn=d", "1" => "https://example-aarroisi.fly.dev/assets/app-a7b7a1bb2570182b93179aee6863d1bb.js?vsn=d"}, "vsn" => "2.0.0"}
2024-06-08T01:45:44.570 app[e82de11b774628] sin [info] 01:45:44.569 request_id=F9bkIndFAjzbCJMAAAHx [info] GET /posts
2024-06-08T01:45:44.572 app[e82de11b774628] sin [info] 01:45:44.571 request_id=F9bkIndFAjzbCJMAAAHx [info] Sent 200 in 1ms
2024-06-08T01:45:45.520 app[e82de11b774628] sin [info] 01:45:45.519 [info] CONNECTED TO Phoenix.LiveView.Socket in 20µs
2024-06-08T01:45:45.520 app[e82de11b774628] sin [info] Transport: :websocket
2024-06-08T01:45:45.520 app[e82de11b774628] sin [info] Serializer: Phoenix.Socket.V2.JSONSerializer
2024-06-08T01:45:45.520 app[e82de11b774628] sin [info] Parameters: %{"_csrf_token" => "B2BBIS5LAlQsQhAqZAZ_K2A0OSUADAMAX4sYL-k-t7Ak2iRhXgxQfGyL", "_live_referer" => "undefined", "_mounts" => "0", "_track_static" => %{"0" => "https://example-aarroisi.fly.dev/assets/app-a5fe870326318b359c192ce86b220d13.css?vsn=d", "1" => "https://example-aarroisi.fly.dev/assets/app-a7b7a1bb2570182b93179aee6863d1bb.js?vsn=d"}, "vsn" => "2.0.0"}
And here is the fly config file:
# fly.toml app configuration file generated for example-aarroisi on 2024-05-11T07:54:18+07:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'example-aarroisi'
primary_region = 'sin'
kill_signal = 'SIGTERM'
[build]
[deploy]
release_command = '/app/bin/migrate'
[env]
PHX_HOST = 'example-aarroisi.fly.dev'
PORT = '8080'
[http_service]
internal_port = 8080
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ['app']
[http_service.concurrency]
type = 'connections'
hard_limit = 1000
soft_limit = 1000
[[vm]]
size = 'shared-cpu-1x'
I don't think it has any special configuration since it's what generated by fly automatically. Probably related to sin
region?
If it's related to fly, I'll try to replicate this with a DigitalOcean droplet (vm) later today.
Fly.io is known to have aggressive TCP timeout settings that break websockets, which is what I expect is happening here if this isn't reproducible elsewhere
https://community.fly.io/t/long-lived-tcp-connections-are-dropped/4074
Fly.io is known to have aggressive TCP timeout settings that break websockets, which is what I expect is happening here if this isn't reproducible elsewhere
https://community.fly.io/t/long-lived-tcp-connections-are-dropped/4074
Hi @feld, thanks for the info. However, @mtrudel said that it doesn't happen on his freshly made Fly app on his account. So, it's probably not a general Fly behavior.
Environment
Elixir version: 1.16.2 Phoenix version: 1.7.12 Phoenix LiveView version: 0.20.14 Bandit version: 1.5.2 Thousand Island version: 1.3.5 Operating system: iOS 17.0.3 Browser: Safari
Actual behavior
I open Safari, viewing a LiveView page that's connected via websocket. Everything works normal. Then I return to home screen, wait for about a minute, then open Safari again. I saw the client-error with the usual "We can't find the internet" for about 10 seconds, then server-error with the usual "Something went wrong" for about 5 seconds, then the page refreshes automatically.
This error shows in the Safari console (I open it using Developer mode, connected to a Mac):
WebSocket connection to 'wss://example-aarroisi.fly.dev/live/websocket?_csrf_token=SihGLVAoM2hsVUo-PDwLHgcbJnU1LxItzB0GfBt93x8HyOjPVA_AQFDW&_track_static%5B0%5D=https%3A%2F%2Fexample-aarroisi.fly.dev%2Fassets%2Fapp-a5fe870326318b359c192ce86b220d13.css%3Fvsn%3Dd&_track_static%5B1%5D=https%3A%2F%2Fexample-aarroisi.fly.dev%2Fassets%2Fapp-a7b7a1bb2570182b93179aee6863d1bb.js%3Fvsn%3Dd&_mounts=0&_live_referer=undefined&vsn=2.0.0' failed: The operation couldn’t be completed. Operation timed out
I also saw the app is trying to connect using longpoll, but eventually failed and refreshed the page.
You can view the issue at play here: https://youtu.be/MxfFKilcA_k.
It's a very vanilla project generated with
mix phx.new
command, and using generated live view from genericmix phx.gen.live
command. The source code is here: https://github.com/aarroisi/example-aarroisi. You can try it yourself at https://example-aarroisi.fly.dev/posts.After I switched it from Bandit to Cowboy, the problem vanishes completely and working as expected. So I guess the problem is at Bandit on iOS Safari.
Expected behavior
When I try it in Chrome iOS, this problem doesn't show up. In Chrome, after waking up from sleep, the client error message only show up for about half a second, then it reconnects without refresh. I expect this should also happen in Safari.