search

lathoub / Arduino-AppleMIDI-Library

Send and receive MIDI messages over Ethernet (rtpMIDI or AppleMIDI)
Other
306 stars 66 forks source link

Received packets dropped occasionally when using on an AVR-NET-IO board #100

Closed bjoernbau closed 3 years ago

bjoernbau commented 3 years ago

First of all, thanks for this great library. Basically, I could get it running quite easily. I am using an AVR-NET-IO board with an ATmega664P (16MHz) and the EthernetENC library for driving the ENC28J60 network controller. The board is connected to a Windows 10 computer running the rtpMIDI driver from Tobias Erichsen. Unfortunately I am faced with the problem that received packets get dropped occasionally (I handle the exceptionCallback and get the ReceivedPacketsDropped exception). This happens e.g. when playing a short midi score: Some midi note commands are received, some not while all of them are on the ethernet wire (checked with wireshark). I'm not sure if the problem is caused by some bottleneck in the EthernetENC library, by the AppleMIDI library or the MIDI library. Is there a possibility to speed up the code of the AppleMIDI library? Is it possible to remove or deactivate the MIDI library used by the AppleMIDI library? I do not need the decoding of midi messages at all as I send the raw midi data directly to the serial port. I tried to comment out the MIDI.read() in the main loop but then the rtpMIDI driver cannot connect to the board at all. Thanks for your help and best regards, Björn

lathoub commented 3 years ago

This is the first release with checks on packet drops - and i'm not sure if the reporting of actual dropped packets is correct (so it might be reporting that packets are dropped, but that they actually did arrive) . If you see them on Wireshark, they should be visible on your device.

Can you count the midi message from your midi score, and compare with the amount that arrive?

I can't speed things up, but there are some events that you can use to get earlier access to the incoming MIDI messages (see the callbackexample: setHandleStartReceivedMidi, setHandleReceivedMidiand setHandleEndReceivedMidi)

lathoub commented 3 years ago

Alternatively, do you have another device (eg ESP32) to test on the same network setup?

lathoub commented 3 years ago

Also, in case of a suspected error, make sure you test against the latest source code here on GitHub and not against the released version available via the Arduino Library manager.

bjoernbau commented 3 years ago

Thanks for your reply. I now had a closer look to the wireshark output and the incoming midi messages. The score has eight notes (thus creating eight note on and eight note off messages when played). They can be seen in wireshark:

No. Time Source Destination Protocol Length Info 209 56.717146 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=E2, v=95) 210 57.430087 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=E2, v=95) 211 57.430279 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=G2, v=95) 213 58.144811 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=G2, v=95) 214 58.145128 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=A2, v=95) 215 58.849437 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=A2, v=95) 216 58.849626 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=D3, v=95) 218 59.561362 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=D3, v=95) 219 59.561542 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=E3, v=95) 221 60.260123 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=E3, v=95) 222 60.260301 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=G3, v=95) 225 60.973536 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=G3, v=95) 226 60.973739 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=A3, v=95) 227 61.671559 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=A3, v=95) 228 61.671762 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note On (c=1, n=E3, v=95) 231 62.386917 192.168.1.1 192.168.1.2 RTP-MIDI 58 Note Off (c=1, n=E3, v=95)

I already use the HandleReceivedMidi callback to get the raw bytes. I get the following output on my serial debug console (the hex values are the bytes the HandleReceivedMidi callback gives me):

ReceivedPacketsDropped 65511 0x80 0x28 0x5F ReceivedPacketsDropped 65535 0x80 0x2B 0x5F 0x90 0x2D 0x5F 0x80 0x2D 0x5F ReceivedPacketsDropped 65535 0x80 0x32 0x5F ReceivedPacketsDropped 65535 0x80 0x34 0x5F ReceivedPacketsDropped 65535 0x80 0x37 0x5F ReceivedPacketsDropped 65535 0x80 0x39 0x5F *** ReceivedPacketsDropped 65535 0x80 0x34 0x5F

As can be seen, all midi off messages are delivered, but only one midi on message. When there should be a midi on message I get the ReceivedPacketsDropped exception instead.

Next, I will check the behaviour using a ESP01 module and give a feedback soon.

bjoernbau commented 3 years ago

Also, in case of a suspected error, make sure you test against the latest source code here on GitHub and not against the released version available via the Arduino Library manager.

I checked the versions. The version from the Arduino library manager is equal to the latest version on GitHub.

lathoub commented 3 years ago

Also, in case of a suspected error, make sure you test against the latest source code here on GitHub and not against the released version available via the Arduino Library manager.

I checked the versions. The version from the Arduino library manager is equal to the latest version on GitHub.

What i meant was: don't use the release version, use a clone or copy of the latest source code (i have made modifications to the source code after the release)

lathoub commented 3 years ago

Something is off, the fact that its always the note On is dropped (dropping packets should be totally random, due to network conditions - in a home network (?) packets are rarely/never dropped).

Also, the sequenceNris 65535 or 0xFFFF and looks like the sequenceNr uint16_t does not roll over.

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/3e2e2a698fb75a13cbd629b8d3063785e8678708/src/AppleMIDI.hpp#L1015-L1020

lathoub commented 3 years ago

Something is off, the fact that its always the note On is dropped (dropping packets should be totally random, due to network conditions - in a home network (?) packets are rarely/never dropped).

0x90 0x2D 0x5F There is 1 noteOn in the series, so not always

Your computer has ip 192.168.1.1 and your Arduino 192.168.1.2 - what does your network topology look like? Do you use a router with DHCP? (192.168.1.1 is typically used by a router, not a computer).

lathoub commented 3 years ago

Can you add to your sketch

#define SerialMon Serial
#define APPLEMIDI_DEBUG SerialMon

in AppleMIDI.hpp:

DBG("pParticipant->receiveSequenceNr", pParticipant->receiveSequenceNr);
DBG("rtp.sequenceNr", rtp.sequenceNr);

before

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/3e2e2a698fb75a13cbd629b8d3063785e8678708/src/AppleMIDI.hpp#L1015

and replace

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/3e2e2a698fb75a13cbd629b8d3063785e8678708/src/AppleMIDI.hpp#L1015

with 

     if ((uint16_t)(pParticipant->receiveSequenceNr + 1)) != rtp.sequenceNr) {
bjoernbau commented 3 years ago

Also, in case of a suspected error, make sure you test against the latest source code here on GitHub and not against the released version available via the Arduino Library manager.

I checked the versions. The version from the Arduino library manager is equal to the latest version on GitHub.

What i meant was: don't use the release version, use a clone or copy of the latest source code (i have made modifications to the source code after the release)

I compared the code from the main git branch to the release in Arduino (v3.0.0), they are equal. There are only minimal differences in the library.properties and README.md files.

bjoernbau commented 3 years ago

Something is off, the fact that its always the note On is dropped (dropping packets should be totally random, due to network conditions - in a home network (?) packets are rarely/never dropped).

Also, the sequenceNris 65535 or 0xFFFF and looks like the sequenceNr uint16_t does not roll over.

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/3e2e2a698fb75a13cbd629b8d3063785e8678708/src/AppleMIDI.hpp#L1015-L1020

I think the calculation of the dropped packets needs to be reversed to: _exceptionCallback(ssrc, ReceivedPacketsDropped, rtp.sequenceNr - pParticipant->receiveSequenceNr - 1); With this modification, I get the correct number of dropped packets.

bjoernbau commented 3 years ago

Your computer has ip 192.168.1.1 and your Arduino 192.168.1.2 - what does your network topology look like?

Correct, I use a direct ethernet cable between the computer and the board. Maybe the EthernetENC library for driving the ENC28J60 requires so much computational power so it cannot handle all packets, but this would be quite weird.

bjoernbau commented 3 years ago

Can you add to your sketch

I did it and get the following result (with the reversed calculation of the dropped packets). There are several midi commands before and after the notes. This is caused by the score software. And again, there is just one note on command comming through.

Booting Ethernet cable is not connected. OK, now make sure you an rtpMIDI session that is Enabled Add device named Arduino with Host 192.168.1.2 Port 5004 (Name AppleMIDI-Arduino ) Then press the Connect button Then open a MIDI listener and monitor incoming notes Connected to session HP-ProBook-450-G2 pParticipant->receiveSequenceNr 255 rtp.sequenceNr 304 ReceivedPacketsDropped 48 0xB9 0x00 0x7F pParticipant->receiveSequenceNr 304 rtp.sequenceNr 306 ReceivedPacketsDropped 1 0xC9 0x00 pParticipant->receiveSequenceNr 306 rtp.sequenceNr 311 ReceivedPacketsDropped 4 0xB9 0x5F 0x00 pParticipant->receiveSequenceNr 311 rtp.sequenceNr 316 ReceivedPacketsDropped 4 0xB9 0x06 0x0C pParticipant->receiveSequenceNr 316 rtp.sequenceNr 321 ReceivedPacketsDropped 4 0xB1 0x00 0x00 pParticipant->receiveSequenceNr 321 rtp.sequenceNr 326 ReceivedPacketsDropped 4 0xB0 0x0A 0x40 pParticipant->receiveSequenceNr 326 rtp.sequenceNr 331 ReceivedPacketsDropped 4 0xB1 0x5B 0x00 pParticipant->receiveSequenceNr 331 rtp.sequenceNr 336 ReceivedPacketsDropped 4 0xB0 0x0B 0x7F pParticipant->receiveSequenceNr 336 rtp.sequenceNr 341 ReceivedPacketsDropped 4 0xB1 0x64 0x00 pParticipant->receiveSequenceNr 341 rtp.sequenceNr 346 ReceivedPacketsDropped 4 0xB9 0x00 0x7F pParticipant->receiveSequenceNr 346 rtp.sequenceNr 348 ReceivedPacketsDropped 1 0xB9 0x07 0x7F pParticipant->receiveSequenceNr 348 rtp.sequenceNr 353 ReceivedPacketsDropped 4 0xB9 0x5C 0x00 pParticipant->receiveSequenceNr 353 rtp.sequenceNr 358 ReceivedPacketsDropped 4 0xB9 0x26 0x00 pParticipant->receiveSequenceNr 358 rtp.sequenceNr 363 ReceivedPacketsDropped 4 0xB0 0x07 0x7F pParticipant->receiveSequenceNr 363 rtp.sequenceNr 368 ReceivedPacketsDropped 4 0xB1 0x5D 0x00 pParticipant->receiveSequenceNr 368 rtp.sequenceNr 373 ReceivedPacketsDropped 4 0xB0 0x5C 0x00 pParticipant->receiveSequenceNr 373 rtp.sequenceNr 383 ReceivedPacketsDropped 9 0xB0 0x26 0x00 pParticipant->receiveSequenceNr 383 rtp.sequenceNr 435 ReceivedPacketsDropped 51 0xB2 0x7B 0x00 pParticipant->receiveSequenceNr 435 rtp.sequenceNr 453 ReceivedPacketsDropped 17 0x80 0x28 0x5F pParticipant->receiveSequenceNr 453 rtp.sequenceNr 455 ReceivedPacketsDropped 1 0x80 0x2B 0x5F pParticipant->receiveSequenceNr 455 rtp.sequenceNr 457 ReceivedPacketsDropped 1 0x80 0x2D 0x5F pParticipant->receiveSequenceNr 457 rtp.sequenceNr 458 0x90 0x32 0x5F pParticipant->receiveSequenceNr 458 rtp.sequenceNr 459 0x80 0x32 0x5F pParticipant->receiveSequenceNr 459 rtp.sequenceNr 461 ReceivedPacketsDropped 1 0x80 0x34 0x5F pParticipant->receiveSequenceNr 461 rtp.sequenceNr 463 ReceivedPacketsDropped 1 0x80 0x37 0x5F pParticipant->receiveSequenceNr 463 rtp.sequenceNr 465 ReceivedPacketsDropped 1 0x80 0x39 0x5F pParticipant->receiveSequenceNr 465 rtp.sequenceNr 467 ReceivedPacketsDropped 1 0x80 0x34 0x5F pParticipant->receiveSequenceNr 467 rtp.sequenceNr 471 ReceivedPacketsDropped 3 0xB3 0x7B 0x00 pParticipant->receiveSequenceNr 471 rtp.sequenceNr 476 ReceivedPacketsDropped 4 0xB8 0x7B 0x00 pParticipant->receiveSequenceNr 476 rtp.sequenceNr 481 ReceivedPacketsDropped 4 0xBD 0x7B 0x00 pParticipant->receiveSequenceNr 481 rtp.sequenceNr 486 ReceivedPacketsDropped 4 0xB2 0x7B 0x00 pParticipant->receiveSequenceNr 486 rtp.sequenceNr 491 ReceivedPacketsDropped 4 0xB7 0x7B 0x00 pParticipant->receiveSequenceNr 491 rtp.sequenceNr 496 ReceivedPacketsDropped 4 0xBC 0x7B 0x00 pParticipant->receiveSequenceNr 496 rtp.sequenceNr 501 ReceivedPacketsDropped 4 0xE0 0x00 0x40 pParticipant->receiveSequenceNr 501 rtp.sequenceNr 502 0xB9 0x00 0x7F pParticipant->receiveSequenceNr 502 rtp.sequenceNr 506 ReceivedPacketsDropped 3 0xB9 0x0A 0x40 pParticipant->receiveSequenceNr 506 rtp.sequenceNr 511 ReceivedPacketsDropped 4 0xB9 0x0B 0x7F pParticipant->receiveSequenceNr 511 rtp.sequenceNr 516 ReceivedPacketsDropped 4 0xB0 0x00 0x00 pParticipant->receiveSequenceNr 516 rtp.sequenceNr 521 ReceivedPacketsDropped 4 0xC1 0x19 pParticipant->receiveSequenceNr 521 rtp.sequenceNr 526 ReceivedPacketsDropped 4 0xB0 0x5D 0x00 pParticipant->receiveSequenceNr 526 rtp.sequenceNr 531 ReceivedPacketsDropped 4 0xB1 0x5F 0x00 pParticipant->receiveSequenceNr 531 rtp.sequenceNr 544 ReceivedPacketsDropped 12 0xB0 0x7B 0x00 pParticipant->receiveSequenceNr 544 rtp.sequenceNr 548 ReceivedPacketsDropped 3 0xB2 0x7B 0x00 pParticipant->receiveSequenceNr 548 rtp.sequenceNr 553 ReceivedPacketsDropped 4 0xB4 0x79 0x00 pParticipant->receiveSequenceNr 553 rtp.sequenceNr 558 ReceivedPacketsDropped 4 0xB7 0x7B 0x00 pParticipant->receiveSequenceNr 558 rtp.sequenceNr 563 ReceivedPacketsDropped 4 0xB9 0x79 0x00 pParticipant->receiveSequenceNr 563 rtp.sequenceNr 568 ReceivedPacketsDropped 4 0xBC 0x7B 0x00 pParticipant->receiveSequenceNr 568 rtp.sequenceNr 573 *** ReceivedPacketsDropped 4 0xBE 0x79 0x00

bjoernbau commented 3 years ago

Alternatively, do you have another device (eg ESP32) to test on the same network setup?

I now checked the behaviour using an ESP01. It behaves much better. When starting the score software (where the midi configuration commands get sent), there are some dropped packets (maybe due to wifi?). But when the midi notes are sent, 7 note on and all 8 note off commands are received.

Booting WiFi verbunden OK, now make sure you an rtpMIDI session that is Enabled Add device named Arduino with Host 192.168.42.20 Port 5004 (Name AppleMIDI-ESP8266 ) Then press the Connect button Then open a MIDI listener and monitor incoming notes Connected to session HP-ProBook-450-G2 pParticipant->receiveSequenceNr 49784 rtp.sequenceNr 346 ReceivedPacketsDropped -49439 0xB9 0x00 0x7F pParticipant->receiveSequenceNr 346 rtp.sequenceNr 347 0xB9 0x00 0x7F pParticipant->receiveSequenceNr 347 rtp.sequenceNr 348 0xC9 0x00 [...] pParticipant->receiveSequenceNr 352 rtp.sequenceNr 354 ReceivedPacketsDropped 1 0xB9 0x5C 0x00 pParticipant->receiveSequenceNr 354 rtp.sequenceNr 355 0xB9 0x0B 0x7F [...] pParticipant->receiveSequenceNr 490 rtp.sequenceNr 495 *** ReceivedPacketsDropped 4 0x80 0x28 0x5F pParticipant->receiveSequenceNr 495 rtp.sequenceNr 496 0x90 0x2B 0x5F pParticipant->receiveSequenceNr 496 rtp.sequenceNr 497 0x80 0x2B 0x5F pParticipant->receiveSequenceNr 497 rtp.sequenceNr 498 0x90 0x2D 0x5F pParticipant->receiveSequenceNr 498 rtp.sequenceNr 499 0x80 0x2D 0x5F pParticipant->receiveSequenceNr 499 rtp.sequenceNr 500 0x90 0x32 0x5F pParticipant->receiveSequenceNr 500 rtp.sequenceNr 501 0x80 0x32 0x5F pParticipant->receiveSequenceNr 501 rtp.sequenceNr 502 0x90 0x34 0x5F pParticipant->receiveSequenceNr 502 rtp.sequenceNr 503 0x80 0x34 0x5F pParticipant->receiveSequenceNr 503 rtp.sequenceNr 504 0x90 0x37 0x5F pParticipant->receiveSequenceNr 504 rtp.sequenceNr 505 0x80 0x37 0x5F pParticipant->receiveSequenceNr 505 rtp.sequenceNr 506 0x90 0x39 0x5F pParticipant->receiveSequenceNr 506 rtp.sequenceNr 507 0x80 0x39 0x5F pParticipant->receiveSequenceNr 507 rtp.sequenceNr 508 0x90 0x34 0x5F pParticipant->receiveSequenceNr 508 rtp.sequenceNr 509 0x80 0x34 0x5F

lathoub commented 3 years ago

I'm seeing the same drops when I have the callbacks print to serial (using DBG). When i disabled the DBG in the callbacks, only leaving a DBGfor ReceivedPacketsDropped, i only get 1 message (at the start, that is because of missing sequenceNr init), after that no more dropped packets.

So can you try to remove the debugging statements, only keeping the ReceivedPacketsDroppedmessage

(tested on a wired wESP32)

lathoub commented 3 years ago

Using an ESP32, on a slow wifi network, i see occasional drops - but not on a wired network.

lathoub commented 3 years ago

added firstMessageReceivedto particpant struct to avoid a ReceivedPacketsDroppedexception on first received packet

bjoernbau commented 3 years ago

I'm seeing the same drops when I have the callbacks print to serial (using DBG). When i disabled the DBG in the callbacks, only leaving a DBGfor ReceivedPacketsDropped, i only get 1 message (at the start, that is because of missing sequenceNr init), after that no more dropped packets.

So can you try to remove the debugging statements, only keeping the ReceivedPacketsDroppedmessage

(tested on a wired wESP32)

Thanks for your support. I now removed all debugging statements except of the "ReceivedPacketsDropped" (and even removed the HandleReceivedMidi callback for testing), but still get a lot of dropped packages. I noticed that packages get dropped when two packages (e.g. note commands) follow very quickly: In my score, I had some notes without any pause, so when one notes ends, the next starts directly. That led to the effect that only the note on commands get dropped, but not the note off commands. When I place pauses between the notes, then no commands get dropped.

I am quite not sure where to debug further. Maybe I will import the sketch to Atmel Studio as this makes debugging easier and will take a closer look to the ethernet library.

bjoernbau commented 3 years ago

added firstMessageReceivedto particpant struct to avoid a ReceivedPacketsDroppedexception on first received packet

After updating my code base to the master, I noticed that I do not get any exception callback at all. pParticipant->firstMessageReceived is initialised to true, but never gets set to false (it will never enter the following statement): https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/84f3dcae8a0a425c56b9d01d6bdbbcacb5e0a24b/src/AppleMIDI.hpp#L1015-L1022 I think the code snippet needs to be modified to:

if (pParticipant->firstMessageReceived == true)
    // avoids first message to generate sequence exception
    // as we do not know the last sequenceNr received.
    pParticipant->firstMessageReceived = false;
else if (rtp.sequenceNr - pParticipant->receiveSequenceNr - 1 != 0) {
    if (nullptr != _exceptionCallback)
        _exceptionCallback(ssrc, ReceivedPacketsDropped, rtp.sequenceNr - pParticipant->receiveSequenceNr - 1);
}
lathoub commented 3 years ago

Thanks for the fox - uploaded to master (and contributed to you)

lathoub commented 3 years ago

I'm not able to reproduce dropped packets on a wESP32 (Ethernet). In MIDI-OX I play a MIDI file to the wESP32 - no drops

lathoub commented 3 years ago

Correction - i do get the drops now as well :-(

lathoub commented 3 years ago

packets are read here:

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/cb72859458a3cb53ee3d798e703852327b263baf/src/AppleMIDI.hpp#L43-L58

similarly for the control packets.

I'm starting to think the packets are dropped here (use/order of available/ parsePacket/ read is unclear from reading the docs)

lathoub commented 3 years ago

Having increased the UdpTxPacketMaxSizeand MaxBufferSizeto 4096, i still get dropped packets on an ESP32 (Ethernet). Shifting my focus now to available() in AppleMIDI.h

https://github.com/lathoub/Arduino-AppleMIDI-Library/blob/cb72859458a3cb53ee3d798e703852327b263baf/src/AppleMIDI.h#L189-L220

bjoernbau commented 3 years ago

I am not sure if I can support you with the current debugging. If so, please let me know.

lathoub commented 3 years ago

i can now consistently reproduce the issue:

Playing the MID file thru MIDI-OX -> rtpMIDI issues 15 seperate MIDI messages at once - each in their own UDP packet/frame.
Knipsel

This library is single threaded and reading the UDP packets and parsing happens in the same thread - if the program does not execute fast enough, packets will be dropped - if the are sent as seperate messages. rtpMIDI does provide the mechanism to pack them all into 1 rtpMIDI message (multiple per packet), but somehow MIDI-OX/rtpMIDI1.1.14 do not do that. (I need to check if I play the same MID file from MacOS combines the messages)

So, what to do here? The library has been build assuming multiple MIDI messages per packet (when intended to be played at the same time) and minimum memory footprint. From the above, it looks like i need to optimize the parser for speed (currently I do not have the time to do that, so all help is welcome)

lathoub commented 3 years ago

When I play the same MID file on MacOS (using Aria Maestosa) I get 0 (zero) dropped packets, because the MIDI messages that are send at the same time, are put into the same packet - and not in seperate packages on Windows when using rtpMIDI 1.1.14. Albeit rtpMIDI 1.1.14 is not wrong, it should put MIDI messages that are send at the same time in 1 UDP packet (see spec)

On MacOS: Inkedmacos_LI

Knipsel2

On Windows using rtp .1.114 Knipsel

Knipsel3

I will try with another MID player than MIDI OX - trying to understand if its MIDI-OX or rtpMIDI 1.1.14

lathoub commented 3 years ago

Aria Maestosa is also available on Windows. When playing the same MID file, again, each MIDI message goes into a separate rtpMIDI packet, not using the multiple MIDI messages in 1 MIDI section

Knipsel

On MacOS, all Control Change and Program Changes are put into 1 rtpMIDI packet

InkedInkedmacos_LI

bjoernbau commented 3 years ago

This library is single threaded and reading the UDP packets and parsing happens in the same thread - if the program does not execute fast enough, packets will be dropped - if the are sent as seperate messages. rtpMIDI does provide the mechanism to pack them all into 1 rtpMIDI message (multiple per packet), but somehow MIDI-OX/rtpMIDI1.1.14 do not do that. (I need to check if I play the same MID file from MacOS combines the messages)

I had a short look to the EthernetENC library I use to drive the ENC28J60 network controller. The library configures 2048 Bytes of receive buffer in the ENC28J60. At first glance this should be sufficient to keep some packets while the AppleMIDI library is busy.

So, what to do here? The library has been build assuming multiple MIDI messages per packet (when intended to be played at the same time) and minimum memory footprint.

I could write Tobias Erichsen an email to ask him about his opinion if the rtpMIDI driver works properly. Maybe he is not aware about the fact that putting multiple messages into one data package would be preferable. What do you think?

bjoernbau commented 3 years ago

Knipsel3

When looking to this screenshot the question comes to me why parseDataPackets takes so much longer when a received packet got dropped compared to when it was processed properly. I think that's strange.

I can have a closer look to your library and do some more debugging at the weekend.

lathoub commented 3 years ago

When looking to this screenshot the question comes to me why parseDataPackets takes so much longer when a received packet got dropped compared to when it was processed properly. I think that's strange.

I also wondered that. When i remove the debug message in the handler, the time is similar to the other messages

lathoub commented 3 years ago

I have created another branch (v3.1) to work in

lathoub commented 3 years ago

New data point: Teensy 4.1 with Ethernet Kit: no dropped packets using rtpMIDI 1.1.14 (OK, Teensy 4.1 is 330 times as fast as a Mega :-))

lathoub commented 3 years ago

delete branch v3.1 - latest changes to master. Playing the sample MID file from above, I no longer get dropped messages on a Mega/W5500 (but do get them an an ESP32!?) @bjoernbau can you test on your setup using ENC28J60

bjoernbau commented 3 years ago

delete branch v3.1 - latest changes to master. Playing the sample MID file from above, I no longer get dropped messages on a Mega/W5500 (but do get them an an ESP32!?) @bjoernbau can you test on your setup using ENC28J60

Unfortunatly I do not notice any difference. I still get dropped packages, even when using the sample you linked before. I also tried Aria Maestosa where I noticed an interesting difference to the software I used before (TuxGuitar). Where TuxGuitar send note off commands, Aria Maestosa sends note on commands with velocity 0. All commands for stopping a note (TuxGuitar with note off and Aria Maestosa with note on commands velocity 0) come through, but most note on commands get dropped.

B0 0B 7F
*** ReceivedPacketsDropped 2
90 28 00
*** ReceivedPacketsDropped 1
90 2B 00
*** ReceivedPacketsDropped 1
90 2D 00
*** ReceivedPacketsDropped 1
90 32 00
90 34 5F
90 34 00
*** ReceivedPacketsDropped 1
90 37 00
*** ReceivedPacketsDropped 1
90 39 00
*** ReceivedPacketsDropped 1
90 34 00
B0 7B 00
*** ReceivedPacketsDropped 1

I attached a simple Midi file I used until now: Sample.zip

lathoub commented 3 years ago

With the latest code in the Master branch, i can play your sample MID file (zipped above) in TuxGuitar without a single dropped package on:

MKRZERO+ETHSHIELDW5500

Only on a ESP32 I get dropped packages. Moving to AsyncUDP is a big change in the code, not sure if I want to do that. The ESP8266 does not drop any packge, whilst the ESP32 drops a lot - i can't explain it.

I reverted back to reading 1 packet and parse directly

        if (readDataPackets()) // from socket into dataBuffer
            parseDataPackets();   // from dataBuffer into inMidiBuffer

        if (readControlPackets())  // from socket into controlBuffer
            parseControlPackets(); // from controlBuffer to AppleMIDI

which allows for really small buffers: 24 bytes for UDP, 64 for the MIDI messages.

The alternative approach, reading as much packets as possible first, then parse needs much more memory (256 in stead of 64)

        while (readDataPackets()) // from socket into dataBuffer
            parseDataPackets();   // from dataBuffer into inMidiBuffer

        while (readControlPackets())  // from socket into controlBuffer
            parseControlPackets(); // from controlBuffer to AppleMIDI
bjoernbau commented 3 years ago

Hi @lathoub I now converted the sketch to an Atmel Studio project to be more flexible with debugging. I switched the optimization level to O3 (optimization for speed) as Arduino (as far as I know) uses Os (optimization for size) what is quite undesired here. But still I get dropped packets. I also activated some debugging in the EthernetENC library, and as far as I understand all network packages get received by the controller.

*** ReceivedPacketsDropped 1
receivePacket [5A8-5E4], next: 5E8, stat: C0, count: 2 -> OK
receivePacket [5EE-62A], next: 62E, stat: C0, count: 1 -> OK
*** ReceivedPacketsDropped 1
receivePacket [634-670], next: 674, stat: C0, count: 11 -> OK
receivePacket [67A-6B6], next: 6BA, stat: C0, count: 29 -> OK
receivePacket [6C0-6FC], next: 700, stat: C0, count: 28 -> OK
receivePacket [706-742], next: 746, stat: C0, count: 27 -> OK
receivePacket [74C-788], next: 78C, stat: C0, count: 26 -> OK
*** ReceivedPacketsDropped 1
receivePacket [792-7CE], next: 7D2, stat: C0, count: 25 -> OK
receivePacket [7D8-14], next: 18, stat: C0, count: 24 -> OK
receivePacket [1E-5A], next: 5E, stat: C0, count: 23 -> OK
receivePacket [64-A0], next: A4, stat: C0, count: 22 -> OK
receivePacket [AA-E6], next: EA, stat: C0, count: 21 -> OK
*** ReceivedPacketsDropped 3
receivePacket [F0-12C], next: 130, stat: C0, count: 20 -> OK
receivePacket [136-172], next: 176, stat: C0, count: 19 -> OK
receivePacket [17C-1B8], next: 1BC, stat: C0, count: 18 -> OK
receivePacket [1C2-1FE], next: 202, stat: C0, count: 17 -> OK
receivePacket [208-244], next: 248, stat: C0, count: 16 -> OK
*** ReceivedPacketsDropped 4

By now, I think the EthernetENC library takes too much computation power, so the AppleMidi library does not get enough cpu time to run properly. A big disadvantage of the ENC28J60 network controller is that the whole IP stack needs to be done in software, and that seems to become obvious here.

I will try to get a shield with a W5100 or W5500 network controller as this will reduce the cpu load significantly. I will report as soon as I will have new knowledge.

Thanks again for your support so far and many greetings.

lathoub commented 3 years ago

I also tried with this one W5500 based over SPI interface without dropping packets

bjoernbau commented 3 years ago

I also tried with this one W5500 based over SPI interface without dropping packets

That is good to know. I just ordered this type of board yesterday evening.

lathoub commented 3 years ago

I added an additional update, reading packets before parsing then. It helps, but does not resolve the issue entirely (the issue remains that rtpMidi send 1 packet per MIDI message, in stead of the optimized method of putting them together in 1 packet - see spec MIDI Command Section)

I also noticed that playing the MIDI file via MIDI-OX never drops a packet, TuxGuitar and Area occasionaly drop a package. Each send different headers and footers.

Hardware: MKR ZERO + ETH field, ESP32, ESP32 + W5500 over SPI

@bjoernbau can you try playing the MIDI file with MIDI-OX on your ENC28J60

MIDI-OX output (never drops packets)

midiox

TuxGuitar

TuxGuitar

Aria

ariaMaestosa

lathoub commented 3 years ago

The default number of sockets on a W5500 is 8 with 2K buffers. Lowering the max amount of sockets, allow for larger buffers. The shipping Ethernet library does not allow setting the MAX_SOCK_NUM, it it fixed based on the ethernet chip (defining ETHERNET_LARGE_BUFFERSdoes not help).

Using the Ethernet3 library, the number of sockets for the W5500 can be defined

https://github.com/sstaub/Ethernet3/blob/205bcc0f0dedad9a2c05eee93f641474b5dc9df1/src/Ethernet3.h#L48-L54

Lowering the maxSockNumto 4, creates 4K buffers (double) and reduces the risk for dropped packets.

See the ESP32 + W5500 example how it works

lathoub commented 3 years ago

Modify the shipping Ethernet library, and change

from:

// Configure the maximum number of sockets to support.  W5100 chips can have
// up to 4 sockets.  W5200 & W5500 can have up to 8 sockets.  Several bytes
// of RAM are used for each socket.  Reducing the maximum can save RAM, but
// you are limited to fewer simultaneous connections.
#if defined(RAMEND) && defined(RAMSTART) && ((RAMEND - RAMSTART) <= 2048)
#define MAX_SOCK_NUM 4
#else
#define MAX_SOCK_NUM 8
#endif

to:

#ifndef MAX_SOCK_NUM
// Configure the maximum number of sockets to support.  W5100 chips can have
// up to 4 sockets.  W5200 & W5500 can have up to 8 sockets.  Several bytes
// of RAM are used for each socket.  Reducing the maximum can save RAM, but
// you are limited to fewer simultaneous connections.
#if defined(RAMEND) && defined(RAMSTART) && ((RAMEND - RAMSTART) <= 2048)
#define MAX_SOCK_NUM 4
#else
#define MAX_SOCK_NUM 8
#endif
#endif

Before Ethernet.h, define

#define MAX_SOCK_NUM 4
#define ETHERNET_LARGE_BUFFERS
#include <Ethernet.h>

This will also give the larger buffers

bjoernbau commented 3 years ago

@bjoernbau can you try playing the MIDI file with MIDI-OX on your ENC28J60

I checked MIDI-OX on my ENC28J60. The behaviour gets a little bit better. When playing the midi file you linked from Wikipedia, packets get dropped rarely, mostly some control commands at the start or stop of playback, but most note commands come through. When playing my sample file with adjecent notes, sometimes fewer packets get dropped, but most of the time there is no much difference.

I think chances are low to get better results with the old ENC28J60, so I will get rid of it. The W5500 already received here, but I did not have time until now to check. I will report how the W5500 works here.

lathoub commented 3 years ago

FYI: The Ethernet3 library is not stable, it stops working after a while - the modified Ethernetlibrary (see above), works like a charm (I also added #define ONE_PARTICIPANT for some additional performance gains out of AppleMIDI and #define MAX_SOCK_NUM 2 #define ETHERNET_LARGE_BUFFERS before the <Ethernet.h> include).

lathoub commented 3 years ago

highly recommend using https://github.com/lathoub/Ethernet The above 

#define MAX_SOCK_NUM 2
#define ETHERNET_LARGE_BUFFERS
#include <Ethernet.h>

don't seem to work in the shipping library; so modified them directly in my forked version.

Not sure if https://github.com/arduino-libraries/Ethernet is still maintained! :-(

bjoernbau commented 3 years ago

After all, I now got the W5500 running :-) It took me several hours to find out why it did not run out of the box: The ethernet library (w5100.cpp) uses the default Arduino pin 10 for SS, but my board (a bare AVR controller with the MightyCore package) has the SS at pin 4. So I needed to define PIN_SPI_SS_ETHERNET_LIB in the specific pins_arduino.h file. Very annoying bug.

I also use your modified ethernet library, but just changed #define SPI_ETHERNET_SETTINGS SPISettings(14000000, MSBFIRST, SPI_MODE0) to #define SPI_ETHERNET_SETTINGS SPISettings(20000000, MSBFIRST, SPI_MODE0) to get maximum SPI speed.

And finally, AppleMidi runs like a charme without any dropped packages :-)

Thanks a lot for your extensive support!

lathoub commented 3 years ago

Super to hear!

I also use your modified ethernet library, but just changed #define SPI_ETHERNET_SETTINGS SPISettings(14000000, MSBFIRST, SPI_MODE0) to #define SPI_ETHERNET_SETTINGS SPISettings(20000000, MSBFIRST, SPI_MODE0) to get maximum SPI speed.

I learned something new, thanks for sharing that.

Pls close this issue when you are ready

lathoub commented 3 years ago

This issue is described in the wiki