Darkness4 / fc2-live-dl-go

Automatically download FC2 livestream. Fast and light.
MIT License
9 stars 0 forks source link

100% CPU consumption #72

Open SmileLikeGod opened 1 week ago

SmileLikeGod commented 1 week ago

Good evening, when recording a broadcast that switches to paid streaming during the live session, fc2-live-dl-go starts consuming 100% of the CPU capacity. Here is the log:

9:32PM ERR github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:448 > ws listen failed error="login required" channelID=58968002
9:32PM WRN github.com/Darkness4/fc2-live-dl-go/fc2/fc2_websocket.go:313 > canceled awaiting for response error="context canceled"
9:32PM WRN github.com/Darkness4/fc2-live-dl-go/utils/try/try.go:161 > canceled all tries
9:32PM ERR github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:499 > failed to fetch playlist error="context canceled" channelID=58968002
9:32PM INF github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:576 > cancelling goroutine group... channelID=58968002
9:32PM INF github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:427 > healthcheck canceled channelID=58968002

and a screenshot https://imgur.com/a/B67W1Ue Here are the settings:

docker run -it --rm \
  -u 1000:1000 \
  -v /usr/videos/:/out \
  ghcr.io/darkness4/fc2-live-dl-go:latest download \
  --wait-for-quality-max-tries 5 \
  --no-remux \
  --poll-interval 360s\
  --format "/out/{{ .Date }} {{ .Time}}.{{ .Ext }}" 58968002

I tried waiting for 5 minutes, but it just kept using 100% of the CPU.

Darkness4 commented 1 day ago

Hello, it seems that FC2 has changed the login API, ~which no longer allows you to connect~ (actually you can still connect, but the logs doesn't say anything anymore).

As for why there is 100% CPU, I'm looking into it... but it will be quite difficult to find. I will probably refactor the whole codebase to try to add new test cases. This will also permits me to easily add contract testing, which will permits me to avoid drifting from future API changes.

Darkness4 commented 13 hours ago

The deadlock has been found and an infinite for loop is causing 100% CPU:

https://github.com/Darkness4/fc2-live-dl-go/blob/52bc0b133e588d4dbb857577534eeba170fdd3a8/fc2/fc2.go#L615-L624

If the channel playlists is closed, the program will immediatly enter the <- playlists case with ok=false. Usual behavior is that the hls.Downloader.Read returns the err to the errChan (few lines below, in the same function), which move the program to the <-errChan case and avoid the <-playlists case. Because no err is in the errChan, the loop continues.

The continue increase the CPU cycle, causing 100% CPU. The fix is being worked on at the same time with the refactor.