Issues with the old buffer in v1.6 being fixed with a brand new buffer in v1.7

[Renkyz]

I've seen this happen on both v1.5.0 stable and the v1.6.0 beta. Sometimes my channel provider has issues where certain channels just refuse to load. My friend who watches channels from me just keeps trying to load them until they sometimes eventually load. Today I checked logs and saw something odd. There were 2 channels that seemed to think it was loaded correctly and someone was watching, so it kept the buffer open (multiple instances of FFMpeg was open in task manager) and the logs just filled up with the same channel being watched and adding clients to the existing buffer. Yet the buffer was not actually connected to anything if that make sense. Each time my friend kept trying to load the same channel, it just continually added clients to it, thinking that it was already live. But he was unable to watch at all. It would just load until it eventually tells him connection timed out or there was an error with the server. My guess is that this only happens when a client is repeatedly trying to watch the channel that isn't loading and trying to force it to work, but due to the fast requests, the buffer doesn't have time to close so it remains open and just starts adding clients to a "fake" buffer or something like this.

[Renkyz]

This might have something to do with my settings. We use a 1MB buffer size, 500 milisecond client connection timeout and the buffer is stored in RAM instead of on disk. I'll try different settings but we haven't had much luck with other settings in the past.

[marcelGoerentz]

Yeah that is possible. As I said, I have to rework the buffer, but this will take some time. I will inform you when I’m done.

[Renkyz]

Oh sorry, I didn't realise you were already aware of this. My bad haha.

[marcelGoerentz]

Please check if latest beta fixed the issue.

[Renkyz]

Checking it now, will report back once my remotely located friend has a chance to test it with me :)

[marcelGoerentz]

So coming back to the temporarily stored stream files that will not be deleted. This seems to be an Windows specific problem. I'm not sure why, but the files are not getting closed correctly. I'm not sure if I can find a workaround for this. I think I will disable the option for those hosts. Since this error only occurs when the files are getting stored on disk, I will disable this option for windows hosts.

[Renkyz]

Oh please don't disable the option. For whatever reason, it seems to work better than storing the buffer in RAM for me on some occasions. Maybe just add a warning about the issue with the Windows version in a text field? I wonder if the version of Windows matters? I'm using Windows Server 2022 21H2. I think it affected my Windows 10 back when I hosted Threadfin on that OS, but I've not tried the most recent version of 10 or even any version of Windows 11 yet. There is also Server 2025 now too. The error message it shows is odd as well. It claims the file is "being used by another process". Maybe some sort of force delete is necessary. I remember using SuperAntiSpyware in the past for it's SUPER Delete option. It always worked for me whenever a standard right click delete would not. Perhaps something similar can be done here? All I know from my testing is if I closed Threadfin entirely, the files could then be manually deleted. Or simply restarting the computer dumps the temp files as well of course but that solution isn't great for me with the servers I run. I remember you mentioning it was related to the current poor buffer implementation. The choice is completely yours of course, but I'd be happy to wait until a new buffer solution is made to see what difference that makes. That reminds me, I only ever tested with FFMpeg buffer as well, not VLC. I used to switch between them in the past to try rule out issues but I think the Australian free to air IPTV channels made some changes in the way they stream and now only FFMpeg works for me, VLC buffer option causes my channels to stop loading.

[Renkyz]

I'm happy to help test on the Windows systems I have available to me. I have the latest version of 10 (22H2 iirc), Windows 11 24H2 (on an i7 860 rig that's not meant to run 11 lmao) and of course I have Windows Server 2022 21H2. I might look into switching to Server 2025, need to research if it's worth the hassle for me to update to it yet or not.

[marcelGoerentz]

Should be fixed in latest beta now!

[marcelGoerentz]

Just as a note, the buffer has been rewritten by me. I extracted the core idea of the implementation and setup new data structures as well as logics around it. I hope this will make the code more robust and easier to understand as well as easier to maintain.

Right now there should be no memory leaks and also the error where the temporarily saved files couldn’t be deleted should be fixed.

Next thing to do will be to simplify the rest of the code part by part.

[Renkyz]

I am testing it right now as we speak actually. So far, it seems to be working great but my friend who streams remotely from me via my Plex server is currently not available, but hopefully he can help test very soon as well.

So far, I only noticed one thing that seemed odd, but it didn't cause a problem it seems, at least not yet. 2024/11/19 12:40:06 [Threadfin] FFMPEG log: [mpegts @ 000001cb85c9a6c0] AAC bitstream not in ADTS format and extradata missing That line appeared in my log for 1 of the streams, 2 streams were happening at the same time but I only ever saw that log just once so far. I will keep testing of course and see if I can find any issues that cause a problem. But like I said, so far there is no issues with the stream itself. I will update here with anything important that myself and my friend discover that will be helpful bug testing information for you :)

[Renkyz]

Hmm, it seems like I spoke too soon sadly. I have a client stacking issue again. And it happened right after the second time I've seen that ADTS format issue.

2024-11-19 13:16:57 [Threadfin] FFMPEG log:             [mpegts @ 000001d75dc69640] AAC bitstream not in ADTS format and extradata missing
2024-11-19 13:18:41 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 13:18:41 [Threadfin] Buffer Size:            2048 KB
2024-11-19 13:18:41 [Threadfin] Channel Name:           7 Mate
2024-11-19 13:18:41 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 13:18:41 [Threadfin] Streaming:              Client joined 58b8c379b610d02b84275eafe5c87869, total: 3
2024-11-19 13:19:31 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 13:19:31 [Threadfin] Buffer Size:            2048 KB
2024-11-19 13:19:31 [Threadfin] Channel Name:           7 Mate
2024-11-19 13:19:31 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 13:19:31 [Threadfin] Streaming:              Client joined 58b8c379b610d02b84275eafe5c87869, total: 4

I have 2 "clients" watching. Both Windows PC's running my Threadfin m3u8 playlist as a network stream inside of VLC, that's my current testing environment.

[Renkyz]

Stopping both of my clients hasn't resolved the issue. Threadfin has kept the channel open and it still thinks there are 4 clients connected. For whatever reason, I am unable to watch the channel again, it just tries to load endlessly. It seems the only solution is to restart Threadfin to fix this, or force close the ffmpeg instance in task manager, which will crashed Threadfin anyway so it's the same thing for me really haha. EDIT: Just tried to kill the ffmpeg and it didn't crash this version of Threadfin, but it also didn't fix the issue so I have to restart it anyway)

[Renkyz]

If there is any debug logging I can turn on to collect data for you, let me know and I will be happy to do so :)

[marcelGoerentz]

You can start the program with

threadfin.exe -debug=3

this should start the program with the highest debug level.

I've tested it also with multiple clients on the localhost ... I will check what is going on there.

[mpegts @ 000001d75dc69640] AAC bitstream not in ADTS format and extradata missing => This can happen, and is not a problem

I will check regarding the open staying ffmpeg instance and the misinformation about the clients.

[Renkyz]

I've reproduced the error twice so far. About to test again for a 3rd time and I will try it with the debug option enabled. So far the same thing happened on both occasions. Both clients were watching the stream until the stream started freezing and buffering, the client try to reconnect and that's when Threadfin seems to think that a 3rd and 4th client are trying to connect to the stream, rather than seeing it as what is actually happening, which is that there was an unkown issue with the stream in the first place and both the original clients 1 and 2 are trying to reconnect. It see's those reconnect attempts as fresh connections as far as I can tell. We'll see what further info we can find with debug level 3 turned on.

[Renkyz]

Well, I tried with the logging on but unfortunately, the 3rd test ended with both clients streaming the channel for 3 minutes and 43 seconds until Threadfin just completely crashed on my server PC, so there is no log to read I'm afraid haha.

[Renkyz]

Just to be clear, I am testing with 2 different computers at the same time on my local network. A Windows 10 PC with VLC and a Windows 11 PC with VLC. Just in case that matters seeing as you mentioned you tested with ,ultiple clients on the localhost. My current setup is a system running Windows Server 2022, that's running my Plex and Threadfin etc, and my clients are seperate computers but on the same local network as the server system.

[marcelGoerentz]

No worries, I hop I found a fix for this mess 😆 Please try with next beta.

[Renkyz]

ok, I will try it now. Just to report my last results with v1.7 (0-beta) - Everything has been working great for 45 minutes when I have both of my clients watching seperate channels. The issue only seems to occur if both clients try to watch the same channel, and the issue usually starts within a few minutes after both clients are watching the same channel. It's like the buffer just gives up I guess haha. Lets see how it goes with the latest beta.

[Renkyz]

Well, slightly better this time, but still crashed Threadfin completely after 10 minutes of both clients watching the same channel. At least that's better than 3 and a half minutes I guess haha. Nothing abnormal in the debug logs either.

[Renkyz]

I wonder if this has something to do with my FFmpeg options inside Threadfin settings. Let me take a screenshot and see what you think.

[Renkyz]

Let me know any changes you'd like me to try there.

[marcelGoerentz]

Nah, seems to be correct. I will check if there is something else wrong when multiple stream are watching the same channel. Maybe I need to improve there. I made a change so the files will be transferred more efficient. Maybe I have to adapt there something 👍🏼

[Renkyz]

Ok no problem then. Just to be sure, I'm going to run the same test with having the buffer write to thie disk instead of to RAM. Maybe it has a different impact then, especially with the changes you made to the deleting of temp files.

[Renkyz]

Already I have an interesting result!

2024-11-19 15:59:00 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-19 15:59:00 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 0
2024-11-19 15:59:00 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 15:59:00 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 15:59:00 [Threadfin] Buffer Size:            2048 KB
2024-11-19 15:59:00 [Threadfin] Channel Name:           7 Mate
2024-11-19 15:59:00 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 15:59:00 [DEBUG] Created folder on disk:  C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-19 15:59:00 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-19 15:59:00 [DEBUG] FFMPEG:                 C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe [-user_agent Threadfin -hide_banner -loglevel error -i http://192.168.0.9:8080 -c copy -f mpegts pipe:1]
2024-11-19 15:59:00 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 0
2024-11-19 15:59:00 [Threadfin] [ERROR] read |0: file already closed () - EC: 0
2024-11-19 15:59:00 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 15:59:00 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 15:59:00 [Threadfin] Buffer Size:            2048 KB
2024-11-19 15:59:00 [Threadfin] Channel Name:           7 Mate
2024-11-19 15:59:00 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 15:59:00 [DEBUG] Created folder on disk:  C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-19 15:59:00 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-19 15:59:00 [DEBUG] FFMPEG:                 C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe [-user_agent Threadfin -hide_banner -loglevel error -i http://192.168.0.9:8080 -c copy -f mpegts pipe:1]
2024-11-19 15:59:00 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [Threadfin] DEBUG:                  5
2024-11-19 15:59:00 [Threadfin] [ERROR] read |0: file already closed () - EC: 0
2024-11-19 15:59:00 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 15:59:00 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 15:59:00 [Threadfin] Buffer Size:            2048 KB
2024-11-19 15:59:00 [Threadfin] Channel Name:           7 Mate
2024-11-19 15:59:00 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 15:59:00 [DEBUG] Created folder on disk:  C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 15:59:00 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-19 15:59:00 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-19 15:59:00 [DEBUG] FFMPEG:                 C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe [-user_agent Threadfin -hide_banner -loglevel error -i http://192.168.0.9:8080 -c copy -f mpegts pipe:1]
2024-11-19 15:59:00 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [Threadfin] DEBUG:                  5
2024-11-19 15:59:00 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 15:59:00 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 15:59:00 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\

The key difference here is this time, Threadfin did not crash when the issue started, so we actually get some logs here explaining what happens, at least at the surface level. As you can see, for some reason it kicked both of my clients and stopped streaming immediately before it deleted the temp files. Then for some reason after that when my clients tried auto reconnecting, it just didn't want to stream again to me. I waited a few minutes before trying to reconnect 1 of my clients, and this is what I saw in the log next.

2024-11-19 16:05:20 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 16:05:20 [Threadfin] Buffer Size:            2048 KB
2024-11-19 16:05:20 [Threadfin] Channel Name:           7 Mate
2024-11-19 16:05:20 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 16:05:20 [DEBUG] Created folder on disk:  C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 16:05:20 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-19 16:05:20 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-19 16:05:20 [DEBUG] FFMPEG:                 C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe [-user_agent Threadfin -hide_banner -loglevel error -i http://192.168.0.9:8080 -c copy -f mpegts pipe:1]
2024-11-19 16:05:20 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 16:05:20 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 0
2024-11-19 16:05:20 [Threadfin] [ERROR] read |0: file already closed () - EC: 0
2024-11-19 16:05:20 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-19 16:05:20 [Threadfin] Buffer Size:            2048 KB
2024-11-19 16:05:20 [Threadfin] Channel Name:           7 Mate
2024-11-19 16:05:20 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-19 16:05:20 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 16:05:20 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 16:05:20 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 16:05:20 [DEBUG] Created folder on disk:  C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\
2024-11-19 16:05:20 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-19 16:05:20 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-19 16:05:20 [DEBUG] FFMPEG:                 C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe [-user_agent Threadfin -hide_banner -loglevel error -i http://192.168.0.9:8080 -c copy -f mpegts pipe:1]
2024-11-19 16:05:20 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 16:05:20 [Threadfin] DEBUG:                  5
2024-11-19 16:05:20 [Threadfin] Streaming:              Stopped streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-19 16:05:20 [DEBUG] Streaming Status:       Remove temporary files (C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\)
2024-11-19 16:05:20 [DEBUG] Remove tmp folder:      C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\M5OF8R24JHFCECE3R21K\58b8c379b610d02b84275eafe5c87869\

Hopefully these logs are helpful for you to diagnose what is going on.

[marcelGoerentz]

It seems like the buffer is not able to read the file saved on the disk 😢 I would recommend to use the RAM to be honest 😄 Nevertheless, I will adapt the amount of files stored before they got deleted. I also need to add more debug messages 🤣 This should do the magic.

[Renkyz]

Yeah I will stick with RAM, but thought it would be useful to try the write to disk just to see if any difference in behaviour would give more clues to what exactly is going wrong. I do use a 1TB M.2 SSD (Samsung 970 Pro) so for my system personally, it's maybe not such a huge difference between RAM and disk, but of course, RAM is always better haha. I did manage to upgrade it like I said I was going to a while ago. It now has 128GB DDR4 3600Mhz G.Skill Trident Z Neo. So hopefully it is ideal scenario for me now haha. I'll be sure to test the exact same way as before once you have a new version ready, it's obviously important to keep my testing methodology as consistent as possible. Fingers crossed that the changes you have planned are the last change necessary to make it stable. If not, then at least it is probably 1 step closer haha.

[Renkyz]

Just passed the 3 hour mark of 1 client successfully streaming 1 channel without any issues. So the current buffer implementation seems to work perfect for that. But I know for sure that if I connect to the same channel with a 2nd client, within 2-3 minutes it will 100% crash Threadfin. I've been thinking for a while and I have no idea why it only fails if there are multiple clients watching the same channel.

[marcelGoerentz]

Thanks for the confirmation! I’m sharing the resources of the buffer with the clients but for some reason they are not synced up yet. I think I need to adapt the data structure there.

[Renkyz]

Ah I see. I suppose if you can find the right balance with that method, it must be the most resource efficient way of doing it. The clients musn't be too far out of sync, when I was testing with 2 of them, they seemed to be very close together in terms of the image being displayed and it wasn't obvious as if one of the clients was several seconds or more ahead of the other. If that is the issue though then surely adapting the amount of file stored before deletion as you suggested earlier, should solve that problem. For some reason, I don't think that is the issue though, I could be wrong but it's my gut instinct I guess haha. I have to rely on that when I don't understand the code myself haha.

[marcelGoerentz]

Okay, now there should be a new version. Hopefully everything is fixed now. I used 3 local clients to test it.

[Renkyz]

Will test it right now :)

[Renkyz]

Testing it right now with 5 clients all on the same channel. Media PC using VLC Main PC using VLC as well as Plex via Chrome browser Chromecast with Google TV running Plex Android App Google TV Streamer 4K running the Plex Android App

So far, all 3 Plex streams are having issues with errors occuring during playback, but I never tried testing like this on the old buffer implementation so for all I know, this issue is completely to do with Plex. This is most likely, because the 2 VLC clients have been going for over 20 minutes without any issue now except for right at the beginning where it seemed to be a little laggy and suffered a few micro freezes, but eventually it smoothed itself out and is playing normally. I can't rule out that initial problem being caused by the Live TV channel itself, that is a high possibility too.

I'll stop the Plex instances for now and leave just the 2 computers with VLC running while I make myself something to eat for dinner. I'll report back with a progress update after that :)

[Renkyz]

Still working perfectly after more than an hour. Seems like whatever you changed has solved the problem. I have found another slight issue though. I decided to use Plex for Windows to open up a different channel before I went AFK. I just stopped the channel being viewed in Plex and then decided to use that API call to see if the active streams were being displayed correctly still. Here is what I found:

C:\Users\Renkyz\Desktop>curl -X POST -H "Content-Type: application/json" -d "{\"cmd\":\"getCurrentlyUsedChannels\"}" http://192.168.0.116:34400/api/
{
  "activeStreams": {
    "playlists": {
      "M5OF8R24JHFCECE3R21K": {
        "playlistName": "PerthTV,
        "activeChannels": [
          "7 Mate"
        ],
        "clienConnections": 2
      },
      "MM9LF6M11YDZ3JHYLEHV": {
        "playlistName": "Sports",
        "activeChannels": [],
        "clienConnections": 0
      }
    }
  }
}
C:\Users\Renkyz\Desktop>pause
Press any key to continue . . .

Normally, it wouldn't display the 2nd playlist entry there after I stopped any channel that was being loaded from a 2nd playlist.

[marcelGoerentz]

Ok I see, this can only happen when the list doesn’t get deleted after all streams has been closed. I will try to reproduce the issue and fix it.

[marcelGoerentz]

Okay is fixed now!

[Renkyz]

I checked my logs this morning and saw my friend had tried streaming some channels from me but seemed to be unsuccessful. I'm guessing the buffer isn't waiting long enough for hime to actually connect, it takes him a while seeing as he watches over Plex from a remote location not close to me. I'll try give him a phone call and find out more info of what exactly is happening for him on his end. I've tried trimming down the log so it isn't so huge, I'll paste it here:

2024/11/20 04:46:42 [Threadfin] Buffer:                 true [ffmpeg]
2024/11/20 04:46:42 [Threadfin] Buffer Size:            2048 KB
2024/11/20 04:46:42 [Threadfin] Channel Name:            9Rush
2024/11/20 04:46:42 [Threadfin] Client User-Agent:      Lavf/LIBAVFORMAT_VERSION
2024/11/20 04:46:42 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024/11/20 04:46:42 [Threadfin] Streaming URL:          https://i.mjh.nz/.r/rush-wa.m3u8
2024/11/20 04:46:42 [Threadfin] Streaming:              Started streaming for 76d17bae299c52f1f87274eba1776d8e
2024/11/20 04:46:47 [Threadfin] Buffer:                 true [ffmpeg]
2024/11/20 04:46:47 [Threadfin] Buffer Size:            2048 KB
2024/11/20 04:46:47 [Threadfin] Channel Name:            Channel 9
2024/11/20 04:46:47 [Threadfin] Client User-Agent:      Lavf/LIBAVFORMAT_VERSION
2024/11/20 04:46:47 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024/11/20 04:46:47 [Threadfin] Streaming URL:          https://i.mjh.nz/.r/channel-9-wa.m3u8
2024/11/20 04:46:47 [Threadfin] Streaming:              Started streaming for 18fae985c5ca4353b8dab93cd5b41f58
2024/11/20 04:47:14 [Threadfin] Streaming:              Client left 76d17bae299c52f1f87274eba1776d8e, total: 0
2024/11/20 04:47:14 [Threadfin] DEBUG:                   Reached EOF!
2024/11/20 04:47:14 [Threadfin] Streaming:              Stopped streaming for 76d17bae299c52f1f87274eba1776d8e
2024/11/20 04:47:15 [Threadfin] Buffer:                 true [ffmpeg]
2024/11/20 04:47:15 [Threadfin] Buffer Size:            2048 KB
2024/11/20 04:47:15 [Threadfin] Channel Name:            Channel 9
2024/11/20 04:47:15 [Threadfin] Client User-Agent:      Lavf/LIBAVFORMAT_VERSION
2024/11/20 04:47:15 [Threadfin] Streaming:              Client joined 18fae985c5ca4353b8dab93cd5b41f58, total: 2
2024/11/20 04:47:48 [Threadfin] Streaming:              Client left 18fae985c5ca4353b8dab93cd5b41f58, total: 1
2024/11/20 04:47:48 [Threadfin] Streaming:              Client left 18fae985c5ca4353b8dab93cd5b41f58, total: 0
2024/11/20 04:47:48 [Threadfin] DEBUG:                   Reached EOF!
2024/11/20 04:47:48 [Threadfin] Streaming:              Stopped streaming for 18fae985c5ca4353b8dab93cd5b41f58
2024/11/20 04:48:06 [Threadfin] Buffer:                 true [ffmpeg]
2024/11/20 04:48:06 [Threadfin] Buffer Size:            2048 KB
2024/11/20 04:48:06 [Threadfin] Channel Name:            Channel 9
2024/11/20 04:48:06 [Threadfin] Client User-Agent:      Lavf/LIBAVFORMAT_VERSION
2024/11/20 04:48:06 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024/11/20 04:48:06 [Threadfin] Streaming URL:          https://i.mjh.nz/.r/channel-9-wa.m3u8
2024/11/20 04:48:06 [Threadfin] Streaming:              Started streaming for 18fae985c5ca4353b8dab93cd5b41f58
2024/11/20 04:48:38 [Threadfin] Buffer:                 true [ffmpeg]
2024/11/20 04:48:38 [Threadfin] Buffer Size:            2048 KB
2024/11/20 04:48:38 [Threadfin] Channel Name:            Channel 9
2024/11/20 04:48:38 [Threadfin] Client User-Agent:      Lavf/LIBAVFORMAT_VERSION
2024/11/20 04:48:38 [Threadfin] Streaming:              Client joined 18fae985c5ca4353b8dab93cd5b41f58, total: 2
2024/11/20 04:48:59 [Threadfin] Streaming:              Client left 18fae985c5ca4353b8dab93cd5b41f58, total: 1
2024/11/20 06:04:02 [Threadfin] FFMPEG log:             [https @ 0000016f139193c0] Stream ends prematurely at 2099376, should be 5865976
2024/11/20 06:04:03 [Threadfin] FFMPEG log:             [NULL @ 0000016f13a97d40] illegal reordering_of_pic_nums_idc 32
2024/11/20 07:05:16 [Threadfin] FFMPEG log:             [tls @ 0000016f1531ce00] Error in the pull function.
2024/11/20 07:05:16 [Threadfin] FFMPEG log:             [tls @ 0000016f1531ce00] IO error: Error number -10054 occurred
2024/11/20 07:05:21 [Threadfin] FFMPEG log:             [tls @ 0000016f1531ce00] The specified session has been invalidated for some reason.
2024/11/20 07:05:23 [Threadfin] FFMPEG log:             [NULL @ 0000016f13a97d40] illegal reordering_of_pic_nums_idc 32

[Renkyz]

I've just updated to the 1.7.3 beta. All my future tests will be done on this version now.

[marcelGoerentz]

When I’m interpreting the log correctly, then there was an issue with the original stream itself! DEBUG: EOF reached, means the stream that ffmpeg is buffering has ended/was terminated.

I will add more debug messages and better Error messages for the buffer in the next version.

Right now I’m pretty happy with the buffer as it is.

[Renkyz]

Hmmm, we had this same issue with the old buffer implementation too. As far as we could tell, it's because his client takes so long to connect that the buffer thought he had ended the connection, like it doesn't wait long enough for him. We found that if he just kept trying to load the channel that eventually it would work after 2, 3 or 4 tries etc. Keep in mind though that my particular setup is perhaps an extreme case haha. I import the channels from Threadfin into my Plex server and then my friend who lives a little over 3 hours away in a rural town, connects to my Plex server and streams the channels from me. These particular channels should be robust, they are the free to air TV channels in Australia, the same we get on aerial TV except digitally streamed on the channels websites as well. I'll test them myself and see if I have any issue connecting to it locally, that will confirm my suspicion about his issue.

[Renkyz]

Ok so Channel 9 on my list does work for me, but it took a loooong time to load. I've tried a few times and the longest it took was 56 seconds before a picture appeared in my VLC. According to my VLC media statistics, the stream is 1080p50 @ 8Mbps so that might be why it took so long to load. When it comes to Plex, Plex doesn't like to wait that long. After maybe 15 seconds you'll get a pop up message simply stating "an error occured during playback, please try again" or something like that. I've previously had my friend on the phone and he would tell me what was going on with his Plex while I looked at the Threadfin logs to see if the issue was on my end and it always seemed like his client was giving up before Threadfin even had a chance to pass the stream to him, if that makes sense.

[Renkyz]

Channel 9 are probably the industry leaders here when it comes to their streaming system and trying to prevent people watching it in any way other than from their own web player. So I don't really think this buffer is the issue. I just had this pop up in my log for my own stream via VLC for example:

2024-11-20 13:34:35 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-20 13:34:35 [Threadfin] Buffer Size:            2048 KB
2024-11-20 13:34:35 [Threadfin] Channel Name:            Channel 9
2024-11-20 13:34:35 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-20 13:34:35 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-20 13:34:35 [Threadfin] Streaming URL:          https://i.mjh.nz/.r/channel-9-wa.m3u8
2024-11-20 13:34:35 [Threadfin] Streaming:              Started streaming for 18fae985c5ca4353b8dab93cd5b41f58
2024-11-20 13:39:01 [Threadfin] DEBUG:                   Could not open file!
2024-11-20 13:39:01 [Threadfin] [ERROR] open C:\Users\Administrator\AppData\Local\Temp\threadfin\2023-11-9MFQ-X9T1CJ\MGFA2YO23CIBAH9F6GIL\18fae985c5ca4353b8dab93cd5b41f58\129.ts: The system cannot find the file specified. () - EC: 0

I'll mess around with my buffer settings inside Threadfin to see if I can improve it, but the issue could just be Channel 9 itself not liking the way we are s treaming it haha. I'll also get my friend to test the Channel 7 network and see if those all work fine (each network here has 5 channels each, Government rules etc). Will keep you updated with any other issues I think might be relevant if/when we find them.

[Renkyz]

So I went back and tested beta v1.6.2 with the old buffer system and noticed that it loads a lot faster than the new buffer system in beta v1.7.x. Is this anything that could be made user configurable at all? Hopefully without it being really difficult to implement?

[Renkyz]

I should add that I updated from 1.7.3 to 1.7.4 and the buffer time on my test channel that I host locally using VLC to output a HTTP stream, decreased from ~15 seconds on 1.7.3 to now being closer to 10 seconds loading time on 1.7.4.

[marcelGoerentz]

This is simple to explain, the old buffer didn’t care about the buffer size setting at all. And save the files with very few bytes. The new buffer is respecting the setting and wait for 2 files to be saved befit sending them to the client. So when you reduce the buffer size it will automatically take less time for the stream to start.

[Renkyz]

Ahhhhh thanks for that explanation, I just always assumed the old buffer was doing it's job properly as I could see a noticeable difference between the 2MB vs 8MB buffer size options. Glad to hear the new buffer is actually using that setting properly. Maybe I should decrease the setting from 2MB to 1MB and see if that helps my friend, seeing as the buffer size doesn't actually impact me much at all given I'm able to stream it directly on my LAN network and don't have to put up with Plex timing out the connection because it's taking too long thanks to streaming over the internet from a long way from here haha.

[Renkyz]

So I've been testing beta 1.7.6 and so far the only issue I've noticed is the buffer seems to think my client keeps leaving the channel it's watching, but when I check in VLC to see if the channel is still playing, it's working jus fine. I'm not sure exactly how long it takes for this error to occur, the timing seems to vary between 5-10 minutes or so. I've reproduced it 3 times now with the same channel, but haven't had any luck making it happen on any other channel yet. Here's a short example of what I see repeated in the logs:

2024-11-21 19:58:15 [Threadfin] Buffer:                 true [ffmpeg]
2024-11-21 19:58:15 [Threadfin] Buffer Size:            2048 KB
2024-11-21 19:58:15 [Threadfin] Channel Name:           7 Mate
2024-11-21 19:58:15 [Threadfin] Client User-Agent:      VLC/3.0.20 LibVLC/3.0.20
2024-11-21 19:58:15 [Threadfin] FFMPEG path:            C:\Users\Administrator\Desktop\ffmpeg-7.0.2-full_build\bin\ffmpeg.exe
2024-11-21 19:58:15 [Threadfin] Streaming URL:          http://192.168.0.9:8080
2024-11-21 19:58:15 [Threadfin] Streaming:              Started streaming for 58b8c379b610d02b84275eafe5c87869
2024-11-21 20:11:47 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:11:55 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:04 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:13 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:21 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:32 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:43 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:12:54 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:13:08 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:13:18 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:13:31 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1
2024-11-21 20:13:45 [Threadfin] Streaming:              Client left 58b8c379b610d02b84275eafe5c87869, total: 1

I didn't experience this on beta 1.7.4 or lower (I had test streams going for 12+ hours with no errors on 1.7.4 for example) so it has to be a really recent change that caused it.

[Renkyz]

I'll switch back to beta 1.7.4 and do the exact same thing again to be 100% sure it isn't an issue introduced by any change from my channel provider.

[Renkyz]

Tested for an hour or so now and the issue hasn't occured on 1.7.4