Stutter & freezing when adding wifi control.

[ReeceKetley]

Hi i'm having an issue on wifi where it works fine then it freezes it also seems to set the wrong pixels too. Ive attached a video and the code i've used.

I've adapted ArtNet to use these matrix panels.

#include <ArtnetWifi.h>
#include <Arduino.h>
#include <ESP32-HUB75-MatrixPanel-I2S-DMA.h>
MatrixPanel_I2S_DMA matrix;

// Wifi settings
const char* ssid = "Reece-PC";
const char* password = "";
const int numLeds = 2048; // CHANGE FOR YOUR SETUP
const int numberOfChannels = numLeds * 3; // Total number of channels you want to receive (1 led = 3 channels)
// Art-Net settings
ArtnetWifi artnet;
const int startUniverse = 0; // CHANGE FOR YOUR SETUP most software this is 1, some software send out artnet first universe as 0.
const int maxUniverses = numberOfChannels / 512 + ((numberOfChannels % 512) ? 1 : 0);
bool universesReceived[maxUniverses];
bool sendFrame = 1;
int previousDataLength = 0;

int matrixWidth = 64;
int matrixHeight = 32;
boolean ConnectWifi(void)
{
  boolean state = true;
  int i = 0;

  WiFi.begin(ssid, password);
  Serial.println("");
  Serial.println("Connecting to WiFi");

  // Wait for connection
  Serial.print("Connecting");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    if (i > 20){
      state = false;
      break;
    }
    i++;
  }
  if (state){
    Serial.println("");
    Serial.print("Connected to ");
    Serial.println(ssid);
    Serial.print("IP address: ");
    Serial.println(WiFi.localIP());
  } else {
    Serial.println("");
    Serial.println("Connection failed.");
  }

  return state;
}

int* computeLedPos(int ledNo)
{
  int x = 0;
  int y = 0;
  for (int i = 0; i < ledNo; i++)
  {
    if(y = 32) {
      ++x;
      y = 0;
    }
    ++y;
  }
  static int cords[2];
  cords[0] = x;
  cords[1] = y;
  return cords;
}

void initTest()
{

}

void onDmxFrame(uint16_t universe, uint16_t length, uint8_t sequence, uint8_t* data)
{
  sendFrame = 1;
  // set brightness of the whole strip
  if (universe == 15)
  {
    //FastLED.setBrightness(data[0]);
    //FastLED.show();
  }

  // Store which universe has got in
  if ((universe - startUniverse) < maxUniverses) {
    universesReceived[universe - startUniverse] = 1;
  }

  for (int i = 0 ; i < maxUniverses ; i++)
  {
    if (universesReceived[i] == 0)
    {
      //Serial.println("Broke");
      sendFrame = 0;
      break;
    }
  }

  // read universe and put into the right part of the display buffer
  for (int i = 0; i < length / 3; i++)
  {
    int led = i + (universe - startUniverse) * (previousDataLength / 3);
    if (led < numLeds){
      int x = led % matrixWidth;
      int y = led / matrixWidth;
      //Serial.print("X: ");
      //Serial.print(x);
      //Serial.print("Y: ");
      //Serial.print(y);
      //Serial.print("R: ");
      //Serial.print(data[i * 3]);
      //Serial.print("G: ");
      //Serial.print( data[i * 3 + 1]);
      //Serial.print("B: ");
      //Serial.print( data[i * 3 + 2]);
      //Serial.println();
      matrix.drawPixelRGB888(x, y, data[i * 3], data[i * 3 + 1], data[i * 3 + 2]);
      //leds[led] = CRGB(data[i * 3], data[i * 3 + 1], data[i * 3 + 2]);
    }
  }
  previousDataLength = length;

  if (sendFrame)
  {
    matrix.fillScreen(0);
    // Reset universeReceived to 0
    memset(universesReceived, 0, maxUniverses);
  }
}

void setup()
{
  Serial.begin(115200);
  ConnectWifi();
  matrix.begin();
  matrix.fillScreen(0);
  artnet.begin();
  initTest();
  artnet.setArtDmxCallback(onDmxFrame);
}

void loop()
{
  // we call the read function inside the loop
  artnet.read();
} 

Only a few pixels at the top are being set too that maybe me doing the wrong calculation however as the orignal example used FastLED which sets leds using:

leds[led] = CRGB(data[i * 3], data[i * 3 + 1], data[i * 3 + 2]);

And i do:

int x = led % matrixWidth; int y = led / matrixWidth; matrix.drawPixelRGB888(x, y, data[i * 3], data[i * 3 + 1], data[i * 3 + 2]);

Any help would be greatly appreicated.

https://user-images.githubusercontent.com/4342846/102909174-7a12a400-4470-11eb-9eca-b227002c78ab.mp4

[vortigont]

I think it would be better to start with one of the examples provided, like PatterPlasma to make sure the matrix works fine. Than you can add WiFi code, etc..

[ReeceKetley]

I think it would be better to start with one of the examples provided, like PatterPlasma to make sure the matrix works fine. Than you can add WiFi code, etc..

The matrix works fine with opther examples, Also it works fine if i remove the WifI code and set each LED manually but for some reason the second i add WiFi iit stutters and freezes

[mrcodetastic]

Unfortunately this is likely a result of the ESP32 hardware and/or RTOS possibly diverting CPU/DMA time to the WiFi portion of the chip. There's nothing this library can do about it I'm afriad.

[ReeceKetley]

Unfortunately this is likely a result of the ESP32 hardware and/or RTOS possibly diverting CPU/DMA time to the WiFi portion of the chip. There's nothing this library can do about it I'm afriad.

Ahh isee, Off topic but could i offload the wifi portion to another esp and connect them directly together using i2c or something? or would i still face the same issue?

[mrcodetastic]

What software are you using to transmit?

[ReeceKetley]

Jinx! or Glediator for now but im already in the process of writting my own software for my use case. I just need to be able to get around 30fps transmited from PC -> ESP -> Matrix to be honest it dosen't need to be WiFi but Serail is just too slow even at the maximum baud rate.

[mrcodetastic]

You might be interested to know I reverse enginerred Glediator and attempted to hack it for this purpose: https://github.com/mrfaptastic/Glediator3

Perhaps you can contribute if you're good at Java? I think it's time for a simpler protocol to be honest. I attempted to implement a hack protocol in Glediator which was based on 16bit color only. 33% reduction in bandwidth and corresponding increase in speed, but colors wern't as good. Serial was hopelessly slow as well.

[ReeceKetley]

Ahhh yes i've used your version to be honest it worked right out the box! but as you say Serial is hopelessly slow if WiFi is an issue i may need to completely rethink this and take a new approach. I work in java from time to time but mostly code in netcore or C++/C#

[vortigont]

tried to reproduce the issue but I was unable to even send some data to the controller. Jinx! just sends empty packets - with all correct headers but all zeros in data. Glediator doesn't work via UDP -sigh! So, @ReeceKetley how do you send Art-Net data to the controller? 30 minutes of searching gave me 0 results on any other free/open software able to do render some Art-Net data via network.

[ReeceKetley]

tried to reproduce the issue but I was unable to even send some data to the controller. Jinx! just sends empty packets - with all correct headers but all zeros in data. Glediator doesn't work via UDP -sigh! So, @ReeceKetley how do you send Art-Net data to the controller? 30 minutes of searching gave me 0 results on any other free/open software able to do render some Art-Net data via network.

You have to patch the pixels in output patch using fast patch (fyi its a bit buggy fast patch is a bit weird you have to try weird values to get it to patch all of the pixels at once also if you see the video i posted i couldn't get it to fully show all the pixels no idea why)

[mrcodetastic]

I'm in the process of hacking my decompiled version of Glediator3 to use TPM2_NET as the transmission protocol from the PC to the ESP32 (via WiFi using UDP). Are you forced to use ArtNet, or do you not care?

https://github.com/mrfaptastic/Glediator3

[vortigont]

"Weird" is an understatement :) I was able to light about 150 pixels poking some random values. Colours and animation matches, although it is very slow. Well, it works nothing more to say. But this weird software is a no-go.

[mrcodetastic]

Merry Christmas by the way you two.

[ReeceKetley]

I'm in the process of hacking my decompiled version of Glediator3 to use TPM2_NET as the transmission protocol from the PC to the ESP32. Are you forced to use ArtNet, or do you not care?

https://github.com/mrfaptastic/Glediator3

I really hold no love for ArtNet or Jinx! i find it very confusing to be honest. TPM2_NET and Glediator would be fine. Do you think TPM2_Net would have better luck with the panels? I did make my own protocol system to use but as i compressed the data it made things even slower to be honest. @vortigont yeah it's really not ideal. I think as a comunity we could develop something thats better. Im doing some more poking around to see what i can do.

@mrfaptastic Merry Christmas to you too dude happy hacking!

[ReeceKetley]

https://user-images.githubusercontent.com/4342846/103111496-b3503d00-4645-11eb-846d-673eab387fd3.mp4

I can play doom on it now.

[mrcodetastic]

Is that for real? If so that's the best thing to come out of 2020.

[vortigont]

That's awesome! :) @ReeceKetley pls. pls more details! :)))

[ReeceKetley]

So thats using @mrfaptastic Glediator TPM2 sketch and some c# code that i wrote to get the screen buffer of an emulator then quickly send it va TPM2 Serial colours are reduced to gain more speed but if you stand like 10 feet away its playable decent FPS

[mrcodetastic]

Closing this as I created an example that uses WiFi. I had to hack the TPM2 protocol and split a frame over multiple packets (UDP payload limit is about 1400 bytes) and re-assemble.

Easily got 30 fps over WiFi. Silky smooth.

https://github.com/mrfaptastic/Glediator3/tree/master/esp32-arduino

[mrcodetastic]

I have updated the Glediator3 jar to make it much easier to use with the Arduino ESP32 TPM2_NET sketch. Literally three steps now.

https://github.com/mrfaptastic/Glediator3

[yusufsulaiman]

hi. I've tried the sketch above, and it worked. but the problem is the delay / scanning of the led. I've tried in ELM software, Resolume and jinx, in ELM software I also tried lowering the fps, but it's still the same, after I observe it turns out that in universe 6 to 12 there is a delay, universe 1-5 is ok. I'm attaching the video

https://user-images.githubusercontent.com/6624207/165770198-d660f858-2698-4d2a-8c12-feddc9b66707.mp4

[yusufsulaiman]

elm video

https://user-images.githubusercontent.com/6624207/165771323-3540ea64-2034-47a7-b4b2-4803ff4b98a5.mp4