search

AronHetLam / ATEM_tally_light_with_ESP8266

Wireless tally light for use with ATEM switchers, which connects over WiFi, using an ESP WiFi module.
GNU General Public License v3.0
159 stars 40 forks source link

Freeze ATEM SDI PRO ISO, after 2 or 3 hour use #119

Open jmvb-cl opened 8 months ago

jmvb-cl commented 8 months ago

A few weeks ago, I was doing and outdoor job, 3 cameras and 3 tallys. The ATEM was also direct recording to an USB SSD disk.

As It was and outdoor environment, I guess that was also a busy wifi network environment.

After a few hours, the tallys stop to work, and the ATEM panel lights turn freeze, but still switching, the recording stoped. The only solution was power cycle the ATEM. After a while, freeze again.

Why the problem back again after a while?, I guess that the fast turn on-off was not enough to erase the connections cache.

The problem gone if the tally's become disconnected.

I did some test in home, not busy network environment, the system works perfect, 6 hours recording.

I know that Atem Switchers only allow a limited number of network connections. My guessing: If a connection drops and reconnects it will be "counted" as a new connection. This can result in exceeding the connection limit although the simultaneous connections are still lower than the limit. Exceeding the limit can result in the ATEM Switcher becoming unresponsive.

That could be override using the server mode?

PDT: I'm using a very simple and cheap TL-WR741ND TP.LINK router, If I use the isolated AP mode could hep?

AronHetLam commented 8 months ago

It could be an issue with the ATEM switcher being sensitive to packet loss, if the tally lights are unable to correctly reply to all messages, due to weak WiFi signal. This will however be hard to test/solve. Having a single tally close to the access point that acts as server for the rest of the tallys could potentially help. Else an Arduino or Raspberry PI with ethernet might be even better as a server. However, I'm not sure what it requires to get up and running on a PI.

jmvb-cl commented 8 months ago

Aron: You mean, and example, arduino one with your tally sketch as a server? The sketch can be uploaded to an arduino one?

a powerful AP could help?

jmvb-cl commented 3 months ago

I can't test the ethernet tally server in real life yet, (because i made a rookie mistake, a tin ball snuck in during a home test and burn the wt32)

My setup ATEM SDI pro ISO, 9.6 firmware Main resolution 1060P60 (or 59.94, no sure at this moment) 3 720P60 cameras 1 1080P60 computer

Ubiquity AP-LR, connected directly from ATEM ethernet port

1 Tally as server, camera 1 3 Tally in cameras, 2, 3 y 4

During the pre-show (2 hours), all good, everything Tally related work good.

But, after 4 o 5 minutes into the event, the tallys begin to lag.

Additional to that, my macbook pro M1, had several problems to connect the ATEM WIFI network, so my prime suspicious was RF overwhelming because 200 or 300 smartphones trying to find or share WIFI

I tried several solutions, change the AP wifi channel, moving away the server from it, restarting the server, restarting the AP, restarting th ATEM, no joy.

Then I tried to downgrade the ATEM firmware to 9.4, but the problem persist, BUT, the ATEM turn irresponsive when it happens, so, I discarded WIFI and look to the ATEM...

I overlooked a very important detail: when the event begin, also does the event record, from ATEM to a USB-C SSD...

Overheating in now the suspect.

I raised the ATEM, and now the lag appears ONLY when I pass for camera 1, the only camera which is in a different resolution.

My guess is, adding to not enough cooling yet, pass for a different resolution camera, spike an extra work load and the network is affected.

The network intermittencies would explain the tallys behavior and the difficulty to connect the macbook pro to AP through WIFI when the issue happen

I'm I right or wrong?

jmvb-cl commented 1 week ago

Just FYI, I did 2 upgrades to my system, and, IDK if both together or one over other was the solution. 1.- USB server 2.- I printer a pair od 3d stands to raise the ATEM and improve the cooling.

No more hangs so far

productiongrowlifechurch commented 1 week ago

@jmvb-cl Could you explain what you mean by USB server? I just installed two of these tally lights and have the same exact freezing issue you described.

jmvb-cl commented 1 week ago

@jmvb-cl Could you explain what you mean by USB server? I just installed two of these tally lights and have the same exact freezing issue you described.

I meant Ethernet server (instead usb) https://github.com/AronHetLam/ATEM_tally_light_with_ESP8266/issues/122