per1234 / EtherEvent

Easy to use password authenticated Ethernet communication between Arduinos and EventGhost Network Event Sender/Receiver or TCPEvents plugins.
MIT License
0 stars 0 forks source link

Not Properly Receiving Large Payload. #1

Open kdschlosser opened 8 years ago

kdschlosser commented 8 years ago

sorry about the book. I am just trying to provide as much information as possible.

I am trying to send Pronto IR HEX to my Arduino Mega 2560. As we know the code is huge. as true hex the code i am transmitting looks like this if stored on the arduino.

const uint16_t prontoCode[78] = { 0x00,0x6D,0x22,0x03,0xA9,0xA8,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15, 0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x3F,0x15,0x3F, 0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15, 0x15,0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15, 0x15,0x15,0x15,0x40,0x15,0x15,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15, 0x3F,0x15,0x3F,0x15,0x3F,0x15,0x702,0xA9,0xA8,0x15,0x15,0x15,0xE6E};

as a non hex form same code is as follows again if stored on the arduino. const char prontoCode[392] = { "0000 006D 0022 0003 00A9 00A8 0015 003F 0015 003F 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 003F 0015 003F 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0040 0015 0015 0015 003F 0015 003F 0015 003F 0015 003F 0015 003F 0015 003F 0015 00702 00A9 00A8 0015 0015 0015 00E6E"};

i believe it's 392 for that char. but at any rate o am setting the buffer size to 512, now I am only using 2.2K of the memory on the Arduino, I have not done a Heap Check. but i am sure there is still no problem. I am also using the TCP Events Plugin for EG I could never get this library to work at all with the standard Network Sender/Receiver. Besides having MD5 errors more often than I would like and now this problem. Other than that, works great. Fast too. So I do thank you for making this Library. ok So back to whats happening, Have a Prefix and Suffix Being sent along with the payload of the pronto code. it receives the payload as an event, and no matter what i set the buffers to it receives no more then 21 of the 392. even if i set both the event and payload buffer to 512 it returns true, And at compile time it makes no difference to the memory size, So I am not sure how this is allocated. I would post code, but in order to be relevent I would have to drop the whole thing, about 45K worth. to much to post. But to give an idea. my includes are as follows for the known libraries. MD5 SPI IRLIB Timer EthernetV2_0 EtherEvent and the utility\w5200 along with some I have created

I am using IRLIB for receiving only and using a modified version of the ProntoIR Library by probonopd its on git Hub. also a simple PIR library for shutting down TV ect... also have a Limited ZWave (MiCasaVerde Vera3) Library for controlling lights and such, as well as simple thermostat for controlling fireplace. and a control system for OSD generated by eventghost, and Serial Control of my Harman Kardon AVR. Now I can store this Pronto Code and 3 others like it in char format and have not a problem running it. But there are some things I would like to add onto and want to free up the memory.

Strange why it comes through as an event tho. and If i send a small payload say 10 digits with the large buffer set. the event comes through just fine then but no payload, if I shrink the payload to say 100 works fine. so I am not sure as to what the exact limit is. and if the problem is being caused by TCP Events or with the Library. But I am thinking the Library only because of the changing the buffer it starts to work. I have also tried this without any of the things I mentioned before just plain jane so to speak, and same problem. just the ethernet and the etherevent and MD5 on a different Mega . But if this is able to get those pronto codes sent from elsewhere. woah what a great utility, as there is no library that is already made that works with my samsung TV codes. Only pronto does. and trying to read the code from flash storage is fickle. hit or miss on working so having it sent would be best. works over serial.

Thanks again. Kevin

kdschlosser commented 8 years ago

Addition, just set a 400 buffer size. it set properly. and can receive a 10 digit, but once i send the pronto code. does the same thing. maybe it's having a problem with the spaces. not sure.

here is a serial output of whats happening

_sent event and Payload from Arduino to EG_ Theatre.Samsng.Power : 1 2

_Received Event from EG_ SamsngPwr

_received payload from EG_ 123456789000


_sent event and Payload from Arduino to EG_ Theatre.Samsng.Power : 1 2

_received event from EG_ 0022 00AC 00

same process, just changed the payload data coming from EG is all. and as you can see there is no received payload, and the event is actually a portion of the payload data. now that is with a 12 for event buffer and a 400 for a payload, on both.

if i increase the event buffer to say 400 when the pronto code comes through as the event, only 21 of the char's print out. and I have it checking for a total of 8 seconds for events after sending to EG, and only the one event shows up. so it's not a buffering to the Ethernet Shield problem. or it could be a limit to the buffer size on the Shield and it just drops the rest of the packets, either way that's pretty useless. it's a seedstudio w5200.

kdschlosser commented 8 years ago

nope not a memory problem either.

5403

Theatre.Samsng.Power

1

Theatre.Samsng.Power : 1

5573

4767

2200AC00AB001500410015

Theatre.Samsng.Power : 1 2

5573 <---- Free Space between HEAP and STACK at sending event to EG

4767 <---- Free Space between HEAP and STACK at receiving pronto Code this is with a 400 event and a 400 payload 2200AC00AB001500410015 <------- and this is what is received as an event and should be payload, odd thing is it's not the begining of the payload either, it's a bit in. i removed spaces to see if that was the problem, and it's not, but there are alot of 0's in the begining of the code, and this is exactly after all the 0's

EDIT\ Not the 0's either, it's starting at the 15th place into the code, consistently.

per1234 commented 8 years ago

Thanks for letting me know of this issue. I can reproduce the problem with any payload longer than 14 characters. I'm working on finding the bug now and will let you know as soon as it's fixed.

per1234 commented 8 years ago

I believe I have solved the issue. I have successfully received 400 character payloads now. The updated library is available for download here: https://github.com/per1234/EtherEvent/archive/master.zip

The problem had 2 parts.

Please let me know if this makes EtherEvent work for your application.

I am also willing to look into the problem with using the EG Network Event Sender/Receiver plugin if you want to give me more information on your issue(I have just tested with the example sketch and EG example tree and it works for me).

MD5 errors can be caused by too small of timeout value if you are still having this problem try setting a very conservative value by adding EtherEvent.setTimeout(2000) to your setup() to see if that is the cause.

kdschlosser commented 8 years ago

Dude your the man. Thanks for the quick response and fix. I have not tried it yet. In the middle of putting a rather large APC UPS in my home. Hopefully later tonight. Not all to concerned about the standard EG receiver. Just thought I should mention. Idk why is wasn't working. Got not response at all from it. And I didn't do to much digging. Oh one other thing. I really don't have a need for Password protecting my EG. And if I leave the password blank. Get MD5 errors. Blank being "" and if so is there a way to not have to load the MD5 library if I am able to not use a password? Its just more of something I don't Need. I am sure it would be easy to impliment with define seeing if MD5 is loaded or not. Just asking. I could do it. Would take me longer as I am not familiar with the code. And would it be ok if I put something together with the pronto library to make an EG Pronto transmitter/receiver? Something to distribute. All in one thing. It's a neat idea.

per1234 commented 8 years ago

I really don't have a need for Password protecting my EG.

An unauthenticated option is high on my to do list for this library. I also don't need authentication and the MD5 library takes a lot of memory and slows event sending/receiving a lot but is necessary to be compatible with the stock Network Event Sender/Receiver and TCPEvents EG plugins. Even with a blank password the EG plugin requires the MD5 authentication so they need to be modified also. I have been focusing more on the hardware aspects of my home automation system lately but now that I know there's at least one other person who would find the option useful I'll try to put it at a higher priority. If you do end up adding the feature please consider making pull requests to EtherEvent and TCPEvents.

would it be ok if I put something together with the pronto library

You are welcome to use my code in any way you want. I put an MIT license on it but really I'm just happy to share my work with the world and don't care how it's used. If you run across any issues or improvements I would appreciate you letting me know as I also use this library in my system.

kdschlosser commented 8 years ago

Sweet, Arduinos weren't really made to decode or encode for that matter. lol. K.I.S.S is a good motto. I personally don't know who would have EG open to a public IP. Set up another EG as a repeater if you are going to do that. But then again, I live in Colorado in the mountains. and closest neighbor is kinda far away. So if someone is trying to "Hack me" I am more concerned about the what for then the how. lol. I don't live near any kiddos playing hacker either. I am just finishing up the Hardware side of my Home automation project. What i do want to set up is location awareness for inside my home. I set up GeoFencing with EG and the webserver plugin. it works so so. I wish i can find a geofencing software that will do web posts that has a better smoothing for location. because we all have experienced the "driving in the middle of the field" with an automotive GPS. I may just have to script it into EG, but i didn't want to waste the transmits from the phone on bad data. but it's nice to run a series events based on distance from home. like turn the heat on. when i am 10 miles away, and when i am 1000 ft away to turn the lights on. and do things when leaving the fenced area too.

I don't know how to send any kind of a "PM" on git. but I would shoot ya my E-mail. maybe trade ideas and notes on what works and doesn't work. (to much crap doesn't work!!)

but I wanted to set something up using Arduinos PIR sensors BT modules and EG (my lighting control reports to EG) so if any of the 2 things happens in a room to open the shades if day and close them when i leave the room. turn tv off, etc.....BT would be for Phone connectivity (i am under the assumption that BT will connect to the closest transmitter). it would pretty much eliminate the need to touch panels all over the house. use the Arduino to transmit voice recognition data to EG.

per1234 commented 8 years ago

Well considering how limited the Arduino's AVR microcontroller is, the authentication is amazingly fast, though of course the newer encryption algorithms are probably slower. I'm just using it on my wired LAN so security is not an issue. It does worry me that I discovered an obvious serious security vulnerability with the Network Event Sender/Receiver Plugin(and TCPEvents) that has been there since 2005, fixed the problem, posted it to the EG forum months ago, and was completely ignored. The password authentication gives people an impression of security but the vulnerability makes the encryption completely useless so they might as well be sending their password in plaintext if they're not using my version of the plugins. I'm working on the unauthenticated EtherEvent option now.

Sounds like you're setting up a nice system! I'm just using Ethernet for my system because I'm old school like that. An alternative to bluetooth could be wifi that seems more consistent since you are already sending data through your network. Another thing I'd really like to add to EtherEvent is ESP8266 compatibility but I don't have a wifi router installed so I haven't even started looking into it yet.

kdschlosser commented 8 years ago

On another note. I forgot about this i just removed the lines from the EtherEvent.cpp pertaining to the remote IP. but there is a fault in it.

EthernetClient.h:27:12: note: candidate expects 1 argument, 0 provided Line 27: uint8_t *remoteIP(uint8_t remoteIP[]);

that's the modified Library file for the remoteIP

as I tried the remoteIP function and it didn't work.

here is the reference to it from the EtherEvent.cpp File

ifdef ethernetclientwithremoteIP_h //the include guard from the modified EthernetClient.h

          fromIP = ethernetClient.remoteIP();  //Save the IP address of the sender. Requires modified ethernet library

endif

now I am pretty sure it has to read and tell me if i am wrong, i have only been coding for 5 months now. But i didn't think you could return an array from a function only if the actual variable to store it in is passed to the function, unless you make it static. but that's pointless because it doesn't release the memory. so there is no need to return at all just pass the variable to have it modify the variable passed. and that's all..

unless you do this in the modified EthernetClient.h in the declaration of the function.

uint8_t remoteIP(uint8_t remoteIP = NULL);

or maybe I have just completely done something wrong. all i should have to do is put the modified Ethernet library into my libraries folder. i believe. If i remember correctly i have over written the Arduino Ethernet library with this one. and It didn't work either.

* edit Havent tested other problems with EtherEvent but got rid of the compile error*

kdschlosser commented 8 years ago

and the Debug routine in Etherevent doesn't work pointer problems.

kdschlosser commented 8 years ago

I also seem to have a problem with TCP Events. when EG crashes (not an if but WHEN, LOL). it orphans the Port. and when i reboot EG all i get from TCP Events is "An error occured while sending you event !" and there is no debug routine in the Send Event to explain exactly what the problem is., and to release the port I gotta reboot. I have found a program that sometimes will allow me to kill the orphaned port but not always, it's called currPorts made by NirSoft. It can be a real hassle as I have EG running on my 2012 R2 server which houses a whole comrodery of Virtual machines and a whole bunch of other things including several databases in mysql and MSSQL. so the boot time is rather longish. and because one of the Virtual machines is a Domain controller it pretty much stops everything until it's rebooted. As far as EG goes. it's pretty much a dead stick. no real development has happened to it in years. the creator walked away and hasn't been heard from since. he gave he left it to the "people".

and I am Ethernet also. only true reliable connection. and the ESP modules, are Kick ass.

80 MHZ Risc processor 32 bit with crap tons of memory and like 8 GPIO pins. would run a ported version of EtherEvent no problem. and no need for an arduino. They are working on getting an IR Library to work properly as there is no hardware Interrupts (I think, or they haven't figured out how to access them) I2C works, SPI works, and if You ditch the factory AT firmware and just code C right to it it runs alot better. it's like 3 lines of code to get it to connect. they have added the ESP to the Arduino IDE as a Board in a forked development of the IDE it's on git.

kdschlosser commented 8 years ago

EtherEvent is not functioning. I even used a one of then Example Sketches. It will not Receive and Event. it Can send them but i just get that "An error occured while sending you event !" from TCP Events I have rebooted several times, I can Ping the EtherEvent Arduino, The port is proper. and when it does try to receive it hangs the Arduino. I have tested it upside and down every which way I know how to Eliminate me as the problem. I am going to ditch the Pointer to the Serial and Just have it spit out the Debug to locate the problem, it may not be the pointer that's the problem with the debug, it could be something else in EtherEvent causing the NUL's to be spat out.

found the problem with the Debug, it runs a 2nd Serial.begin with a different BAUD Rate than what i use

per1234 commented 8 years ago

EthernetClient.h:27:12: note: candidate expects 1 argument, 0 provided Line 27: uint8_t *remoteIP(uint8_t remoteIP[]);

What modified Ethernet library are you using? That looks like an old version. It now returns the IPAddress type. Try installing https://github.com/per1234/EthernetMod/archive/W5x00.zip and see if it works.

But i didn't think you could return an array from a function only if the actual variable to store it in is passed to the function

That's correct and the old version did require an array to be passed to the function but you can return an IPAddress because it's not an array. This is how the official Arduino Ethernet library does EthernetUDP.remoteIP().

I also seem to have a problem with TCP Events. when EG crashes

I definitely have some EG crashes but haven't run into the dead port problem. I'm going to work on some improvements on TCPEvents but I'm not so good with python so if you know any I could definitely use some help.

As far as EG goes. it's pretty much a dead stick.

EG could use work for sure but I disagree about it being dead. Even though bitmonster abandoned it there are frequent updates still but I would like to see a lot of things done differently. The main guy pako seems to be really unwelcome to anyone but the core group contributing. There are 2 github repositories for it, both completely abandoned and it's still being developed on Sourceforge which is a total joke. I really like eventghost and would like to put some effort into it but I don't see how. It's a shame because there's a decent community around the project but no real infrastructure to allow them to participate other than the forum which also sucks. I gave Girder a try a while back to see what I was missing and I have never felt so grateful for EventGhost, crashes and all.

ESP modules, are Kick ass

Yes I think that adding support would make EtherEvent useful to many more people. I'd like to add support for running on an ESP but also for using an ESP8266 module connected to an Arduino via serial because it seems like there are still lots of libraries that are not ported to ESP8266..

EtherEvent is not functioning.

Are you still having the problem or was it caused by the debug issue? I just redownloaded and tested the basic and advanced examples and they are working for me.

found the problem with the Debug, it runs a 2nd Serial.begin with a different BAUD Rate than what i use

Yes that's definitely not ideal how I did that I guess maybe it's better to just do the Serial.begin() in the sketch or at least I should document the debug baud rate. I did the Serial.begin() in EtherEvent because often I will want to turn off debug output in my sketch and just have it turned on in EtherEvent only.

kdschlosser commented 8 years ago

nope I am still having problems with TCP events. the Debug routine shows it getting stuck in an infinate loop on the receiving i'll post the Debug information.

EtherEvent.begin

EtherEvent.setPassword(char)

Startup Complete

Theatre.Master.Init : _..**.**_

EtherEvent.send: attempting connection

EtherEvent.send: target: _..**.**_

EtherEvent.send: port: *****

EtherEvent.send: event: Theatre.Master.Init

EtherEvent.send: payload: _..**.**_

EtherEvent.send: connected, sending magic word

EtherEvent.send: cookiePassword: ab96:****

EtherEvent.send: hashWordMD5: ***

EtherEvent.send: connection closed

EtherEvent.availableEvent: connected

EtherEvent.availableEvent: magic word received

EtherEvent.availableEvent: automatically generated cookie:**

EtherEvent.availableEvent: cookiePassword:***:******

EtherEvent.availableEvent: cookiePasswordMD5: **

EtherEvent.availableEvent: authentication successful

EtherEvent.availableEvent: payload/event for loop

EtherEvent.availableEvent: bytesRead: 0

EtherEvent.availableEvent: payload/event for loop

EtherEvent.availableEvent: bytesRead: 22

EtherEvent.availableEvent: payload separator received

EtherEvent.availableEvent: payloadWithoutRelease

EtherEvent.availableEvent: payload/event for loop

EtherEvent.availableEvent: bytesRead: 10

EtherEvent.availableEvent: event length: 10

EtherEvent.availableEvent: event received: HarmanPwr1

gth: 10

EtherEvent.availableEvent: event received: HarmanPwr1

gth: 10

EtherEvent.availableEvent: event received: HarmanPwr1

gth: 10

EtherEvent.availableEvent: event received: HarmanPwr1

gth: 10

EtherEvent.availableEvent: event received: HarmanPwr1

kdschlosser commented 8 years ago

well you exactly hit the nail on the head with EG. that is what i mean tho. there are updates but the lack of support for it has really killed it. I think a lot of it is because of a language barrier between those working on it and the English Speaking people. I mean I have posted questions in there that go unanswered. Because of that alone one would be be discouraged for using or learning how to use it. I am sure there is a whole lot of things I am not using with eventghost because I simply don't know how or I don't know that it can do that. I am new to programming so I doubt i would be much help.

But i think I have gone very far in learning, so if there is any way i can help. I am also tenatious and keep at something until i get it working just right. or figure it out. sometimes that's a bad thing because i tend to redo things a lot. example is this program I am working on now.

It was working fine. i keep on recoding the thing. so say IR Receiving, EtherEvent,Sending Pronto those items. Original Sketch was 2 thousand lines. (I am stupid proofing the remote for my wife)so no matter what button is pressed it knows what state it should be on because of pressed buttons beforehand as well as what devices are turned on or off. so the actual "Device buttons" on the remote don't do anything. but at 2000 lines of code i have gotten it down to just over 400 lines and it's at a 45 ms start to event send speed between button clicks on the remote (and that's only due to how fast the remote is able to kick off send sequences). if it wasn't for the program size I could run this thing on an UNO. between the Ethernet Library and the IR Library that pretty much kills using an UNO.

kdschlosser commented 8 years ago

ok it seems to be working with the sample sketch now. i removed the modified ethernet library. and it runs properly with the Example sketch.

edit*_works with my sketch also_ i will try that newer moified ethernet library later on. as its not that important for me. and if at all you need access to a larger working environment let me know. I can set up as many virtual machine boxes as ya like and can set up access it say esp modules that I can wire up to arduinos for resetting and testing. i have mountains of this crap. i'll break down what Ihave running over here

2 Servers. SERVER1: 8 core AMD 32 gigs memory 8TB broken into 2 Raid5 Arrays. with 4 SSD's in a RAID0 (with write back turned on) for the OS and APPS has 2 1.2 GHZ intel IOP based Raid controller cards capable of supporting 128 HDD's on each controller.

Server 2: 4 Core AMD 16 Gigs memory 500 Gig raid 0.

network based 3 tuner Cable card ATSC Tuner.

2 24 Port Switches

5000VA APC Battery Backup

Wireless n 600

Cisco VPN Router

180 mbps down stream and 20 mbps up stream Internet Connection.

Server1 is running 2012R2 Hyper-V, MSSQL, MySQL, MediaPortal TVServer, EventGhost, and various other Domain Services. I can set up any kind of Virtual Box, but I have running Ubuntu, Android, Windows Server 2012 R2, and Windows 7.

I have but haven't finished all of them. are Nano PC's for the HTPC's House is structured wired (not 100 % finished).

so if you need access to equipment let me know I can set ya up a VPN tunnel. And if you need anything Arduino wise I am able to get a hold of it very easily. Microcenter is near by and they carry a lot. and a little further away is Sparkfun that actual warehouse and I can pick up from them. I also have a couple of the raspberry Pi2's

kdschlosser commented 8 years ago

ok back to running more electrical in my house. if ya need something email (eight 6 oh 9 sevn O 4 too one 2 at mms att net)

per1234 commented 8 years ago

if it wasn't for the program size I could run this thing on an UNO

The unauthenticated EtherEvent should reduce size quite a bit. Another option is to use the pubsubclient library instead of EtherEvent which is currently smaller(not sure if it will be smaller than unauthenticated EtherEvent). In the end it's nice to have the extra space of the 2560 so you can add features later and not have to worry so much about every byte.

it seems to be working with the sample sketch now.

Are you still having the infinite loop problem? If so and the issue happened when using your own sketch, could you try sending the same event to either the basic or advanced EtherEvent example sketch to see if the problem still happens. That will help us determine if it's caused by EtherEvent.

kdschlosser commented 8 years ago

Problem is with that modified library in some way. I omitted the indef for the senderIP from EtherEvent.cpp it still looped so I removed the Modifiec Ethernet Library and the problem went away. I have not tried the more updated Modified library yet. but the senderIP is not a biggie for me at the moment. if I Eliminate EG at some point (and I probably will) but I will still use the EtherEvent library to send data between the Arduino's, because it's simple and it works well. Tho I haven't tried that yet. The library should be able to send Events and Payloads back and forth between Arduino's correct?

Yet another question, I see the action for TCP events about asking for data and sending data. exactly what is that all about and how does it work and does it work with the EtherEvent library?

per1234 commented 8 years ago

I have not tried the more updated Modified library yet

I think the remoteIP() call is causing memory corruption which leads to the infinite loop issue. I just tested it out with the seeed W5200 shield to make sure that it wasn't caused by the W5200 and did find an unrelated bug in my modified Ethernet v1.1 library causing problems with auto-detecting the Wiznet chipset but that's fixed now(so download the latest version if you are going to use it) and senderIP() is working fine for me with W5100/.W5500/W5200.

I will still use the EtherEvent library to send data between the Arduino's, because it's simple and it works well.

I agree. I really like the event and payload concept. It just makes sense to me, maybe because of using EG so long. I originally was trying to use MQTT and that is a much more standard IoT communication protocol but the system just didn't really make sense to me so that made me decide to create EtherEvent.

The library should be able to send Events and Payloads back and forth between Arduino's correct?

EtherEvent definitely works for sending Arduino to Arduino. The only thing you might run into is that different timeout values might be needed. If you have dialed in the lowest possible value for sending to your PC and the Arduino-Arduino communication takes a little longer then there will be problems sending events with those settings so you need to find the timeout values that work for the slowest communication on your network. If you're just using the default settings then it should work for any because they're set pretty conservatively but it's best to get them as short as possible for the best performance(see the timeouts section of https://github.com/per1234/EtherEvent#troubleshooting)

I see the action for TCP events about asking for data and sending data

I have never used that feature and don't really understand it. It's not documented well at all by the plugin author. The best I can understand is that it's for sending information without triggering an event in EG. EtherEvent doesn't support it so you can only use it for sending EG to EG. With the unauthenticated option it will be impossible to add support for the send/receive data feature because it requires a keyword sent during the authentication process.

I have done most of the work on TCPEvents and EtherEvent to allow unauthenticated communication and it reduces the BasicUsage sketch size by about 11kB, making EtherEvent very similar in size to pubsubclient. The only work left to do is allow turning authentication off in the sketch instead of the user having to modify the library by commenting out the #include "MD5.h" line in EtherEvent.h. I have done it, but for some reason it adds 1kB so I need to work on it a bit more before it's ready for release.

kdschlosser commented 8 years ago

Thanks Boss, you know ya didn't have to jump right on doing the Authentication part. and if you Managed to shrink the program size by 11K i can shrink it the rest I am sure. I have already removed all of the type defs as I only use const char * anyways. I didn't see the need to have them. great option to have if ya don't know how to do the conversions. But in all honesty I would hope that someone that is using this does have some kind of basic programming knowledge. Like me :-D. I just automatically assumed that would be what it needed to see and it's what I used. and as it turned out it's the default for your Library. I don't know if that shrunk the Library at all Probably not as I don't think it would have used them at compile time and wouldn't have added them anyways. But at that 11 K shrink in program size. that's damned close to the 30K mark now. and I did manage to get my memory usage to below 1.5K I just have to check on the Heap usage as i believe that still puts me over the 2K and I don't think I am going to be able to do anything about that. There are a lot of buttons on the Remote and i do like having nice long winded Events in EG for readability. instead of trying to decode some kind of shorthand. especially when it has been a while and I go back to it to figure out a glitch that I stumbled across in my sketch and can't remember what the hell I did because of the shorthand, LOL. Been there already couple of times and it was easier to recode it than to try and figure out my shorthand, unfortunately doing that to decrease my Memory made the sketch larger. but I think I might be able to do something else to cut a sizeable chunk out of it. gotta mess with it later. Oh on another Note. Pronto codes work Perfect!!!!! either in 0000 fashion or 0x00 I am very excited about that. and it's fast too.first time use of the payloads. I seriously would only put a typedef on the payload to convert numbers to const char. because the events really can't be "auto generated" per say. not from like a sensor or something which can be sent as a payload. being a human and all. I think most people would use a char array or even a String not numbers.Thoes phrases would have to be coded in making it one of those 2 above, I wonder If i can make something that would export all of the triggers that are already programmed in EG and send them to the arduino automatically upon boot to build a table for the events. Kinda like button mapping on a remote. That way there is only one table to maintain instead of one on each device. I did alter my Remote with this neat little util for JP standard (some kind of standard for firmware on universal remotes) allowed me to make the button code in number order so they run from 0-80 instead of being longs and numbers all over the place. would make a table really easy to use as I can use remote code now to be the index number for the table of events. Next project is to cut apart the IRLIB and see if i can pull the decode section out for only Sony remotes .I know it compiles the whole decode section for every remote type because the decoder systematically goes through each one to see if it can decode it. and I am only using one type of remote code (Sony 12) easiest to use I am thinking that will free up a HUGE amount of program space as well as shrink the HEAP usage a lot.

per1234 commented 8 years ago

I have uploaded the unauthenticated capability etherevent and tcpevents. To use it:

I would hope that someone that is using this does have some kind of basic programming knowledge

Well of course they will need a little, but my goal is to reduce the requirement as much as possible so that someone could go from basic use of EG to adding an Arduino to their system with EtherEvent without too steep of a learning curve. Once they're sending events back and forth they will be hooked and learning more programming will be inevitable. That was pretty much my path, I had a little basic, html, php, python, and scheme experience but no real interest in putting a lot of effort into learning C++ then I wanted to control some relays over Ethernet using EG and here I am writing libraries. I could have learned faster and better another route but I wouldn't have done it because programming isn't my career, it had to be fun and Arduino did that for me by making it easy to get started.

I don't know if that shrunk the Library at all

The compiler shouldn't add any functions that aren't called, but it certainly is better just to use chars instead of EtherEvent having to do the conversion. On the other hand, in some cases it can save lots of SRAM to use the F() macro so that might make the overhead of the __FlashStringHelper type conversion worthwhile.

I think most people would use a char array or even a String not numbers.

I'm actually using const byte for my events and I just have descriptive variable names but it does make my EG log less readable. I use variables to send events from EG but unfortunately it doesn't allow variables for trigger events so I have to use the numbers there. Of course the double and float events are kind of ridiculous but I figured I'd add them just for the sake of completeness while I was at it. Who knows what the user's application may be.

I wonder If i can make something that would export all of the triggers that are already programmed in EG and send them to the arduino

I use a spreadsheet with all my event codes, variable names, and description comment and then that is automatically formatted into a header file for Arduino and a Python Script action for EG. So it makes it a bit easier to keep everything synced.

I did alter my Remote with this neat little util for JP standard

Wow, wish I could do that with mine. I'm using a PS3 bluetooth remote and I love the price, shape, button layout, and not needing line of sight but they never figured out the sleep system in the EG plugin right so it burns through batteries.

kdschlosser commented 8 years ago

Oh man I used to have a PS3 remote until it actually exploded some batteries. brand new ones i just put in fresh.The thing caused a dead short in the batteries and they literally got so hot it melted part of the remote, and it would always disconnect from the BT on the computer and I would have to go and pair it up again. what a pain that was. so i found this cheap remote called a one for all 4 device remote. turns out it's actually a URC (Universal Remote) who knew. i thik i paid like 20 beans for it. and it has backlight (sweet). but little did I know. i was trying to find remote codes to actually make all of the buttons useable. As we all know depending on the remote code depends what buttons are activated. I happened across some information about JP1 protocol. and that lead to a night of reading. and low and behold i stumbled upon this little Java Util that allows you to do pretty much anything you want to the remote. like pushing buttons between devices (volume) and having one button shoot multiple codes (for input selection on an AVR or TV), shift buttons. (short press is one code long press is another) effectively doubling the buttons. you can custom make your own ir sequences (that's what i did to make the remote codes in numerical order) so this remote has 42buttons per device, and another 4 that you can program for the actual device selection(kewl feature) and 4 devices, and then shifted so X2 = 336 buttons not including the couple of them you can use as "phantom buttons" the ones that can hold 6 codes for say input selection so add another 12X4 to that. its insane. and it's got a nice wide range. usb programmable. model is a URC6440. and the thing is built. it's a solid remote. and i have been running it for 2 months now pressing the hell outta the buttons more than a usual TV watcher would (because of testing) and still on the original el cheap o batteries.

and what do you mean about the triggering events with variables? ya lost me there. I did set up that table. did it backwards tho. table listing is on the Arduino and it sends it to EG. works great did a python dict. i am going to put it into a MySQL database i think.

it's funny. i got into this programming thing exactly like you did. wanted to trigger some relays over Ethernet to turn my fireplace on. next thing i know 5 months later i got micro controllers controlling my blinds. and TV remote and fireplaces, have automated weather information that at a press of a button shoots an OSD on my TV. as well as if someone rings my doorbell it mutes the radio and pauses whatever i am watching. same goes if i get a phone call on my cell. and my sms will read on on my TV also. as well as display photos i get in a text. and the number and contact name of who is calling. have the geo Fencing, a whole OSD menu system for setup menus of the whole system (i modified a nice plugin for EG the one that is made by Pako was horrible looking, but i did manage to find out that looked a hell of a lot better. but wasn't complete. if your interested lemme know. i can zip it up to ya. there are no real good config menus tho. But it does allow for multi layer menus (unlimited). has a nice partial transparent charcoal backdrop that fades in and out as you open and close the menus. I was going to make it so that the whole menu system could be loaded in one shot. but that was over complicating it. so it opens a menu and when you press right on one that has a sub menu it triggers an action. and based on that action you send the next batch of menu data, and it stores the old menu in a nested dictionary one level at a time. so if you back up it reads from the cached data opens the menu and then deletes the data from the buffer. The code is ugly as all hell. but it works. (learning curve). I will try out that new library in a bit. gotta make a depot run to get some aluminum. new construction project. make it so that if i want to do a wiring change i am not cutting off 30 dollars worth of zip ties. this will be the last time i am going to do that. been studying cable raceways and i got a plan. make it like a whales spine and ribcage upside down. c channel and some 1/4" round bar. drill a whole through the c stock and curve the round bar and slide it through. couple of rubber washers or o rings that are tight to keep it from spinning about.or even just flatten a spot on the round bar. should cost about 20 bucks to make like 16 feet of it.

kdschlosser commented 8 years ago

11:30:03 TCPEvents: (<class 'socket.error'>, error(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted'), <traceback object at 0x048E3C38>)

That's the problem I get sometimes with the damned port being orphaned with TCP events.

kdschlosser commented 8 years ago

EtherEvent.h:422:84: error: a function call cannot appear in a constant-expression

kdschlosser commented 8 years ago

that's the New one

per1234 commented 8 years ago

what do you mean about the triggering events with variables?

I'm guessing you mean on the EG side. If you look at the AdvancedUsage.xml example EG tree I include with EtherEvent you can see that I have a Python Command EG action to put the event to be sent in a global then you can send it using eg.globals.event (event is the variable name in this case but in my system the variables names are more descriptive). In my system I have a whole list of events codes all put in variables in a Python Script action in my Autostart macro like this:

eg.globals.eventOSDsmall=120  #show the payload on the osd of the target computer
eg.globals.eventOSDmedium=121  #show the payload on the osd of the target computer
eg.globals.eventOSDlarge=122  #show the payload on the osd of the target computer
eg.globals.eventDRclear=125
eg.globals.eventEventLogDR=123  #desktop remote
eg.globals.eventEventLogDRspeak=124

Unfortunately I can't do a trigger event for my EG macros like TCP.{eg.globals.eventDRclear} so I have to do TCP.125 instead and that is where the system breaks down. It's the sort of change I'd be willing to work on adding to EG if there was an easy way to contribute code.

i modified a nice plugin for EG the one that is made by Pako was horrible looking, but i did manage to find out that looked a hell of a lot better. but wasn't complete. if your interested lemme know.

Yes definitely. I'm using the OSM plugin and the Desktop Remote plugin for my menus.

Only one usage of each socket address (protocol/network address/port) is normally permitted

It looks like the socket is never closed because it crashes. The solution would be to close the socket when the TCPEvents plugin is initialized at EG startup but I'm not sure how that would be implemented.

EtherEvent.h:422:84: error: a function call cannot appear in a constant-expression

I guess it's the strlen() that's causing the problem, I don't get that error. What version of Arduino IDE are you using? I'll test it out and see if that's the problem.

per1234 commented 8 years ago

OK, strange that works in Arduino IDE 1.6.6 but not in previous versions. It must be a different version of the compiler. I moved them out of the class to the namespace and it's working in 1.6.5r2 for me now but I just found a completely different compile error using 1.6.0, haven't checked 1.0.6. It's tough because things keep changing and everyone is on a different IDE version.

kdschlosser commented 8 years ago

believe i fixed it. const byte payloadSeparatorLength = static_cast(strlen(EtherEventNamespace::payloadSeparator)); //includes space at the end const byte payloadWithoutReleaseLength = static_cast(strlen(EtherEventNamespace::payloadWithoutRelease)); const byte closeMessageLength = static_cast(strlen(EtherEventNamespace::closeMessage));

and this line in the CPP file also

byte sendDoubleDecimalPlacesDefault = 3;

gonna test it now

kdschlosser commented 8 years ago

hmmm, is there a code switch for this thing

kdschlosser commented 8 years ago

well the code didn't paste right there is a after the static_cast

per1234 commented 8 years ago

byte sendDoubleDecimalPlacesDefault = 3;

was that line causing an error?

kdschlosser commented 8 years ago

ok it receives ok. gotta test the sending

kdschlosser commented 8 years ago

yup. all good it appears

kdschlosser commented 8 years ago

and puts me just under the 32K mark. and i am down to 1.4K for globals and only a 200 byte change for dynamic. maybe imma run this on an uno.

kdschlosser commented 8 years ago

oh gota test the pronto code tho. that will make probably a 600 byte change. i am going to have to squeeze some more outta it

kdschlosser commented 8 years ago

that error was because it was a const and you tried to modify it somewheres i am guessing.

kdschlosser commented 8 years ago

it spat back an error about being a read only should probably just make it a define

_EDIT_ changed it to a const int and problem went away

per1234 commented 8 years ago

that error was because it was a const and you tried to modify it somewheres i am guessing.

Strange, I tested in 1.6.5r2, 1.6.0, and 1.0.6 without changing that line and they all compile fine.

kdschlosser commented 8 years ago

don't know. could have been a quirk, I'll change it back and see if it goes. how about the others??

kdschlosser commented 8 years ago

and now it compiled fine, and i didn't change anything else. huh who knows

kdschlosser commented 8 years ago

Did the static cast fix it in other IDE's. And I am not using the Arduino IDE. I am using avrDude. And the cores. Don't recall the version. I think it's 1.6.4 something. The first errors were that you can't have a function in a const declaration. So I took the const out. Then it was you can't have a static member that isn't const. So I made another set of variables to grab from the function to pass off to the static const byte. Then it the errors about the static cast. Lead me to believe the first error was wrong in its description. So I did the static cast on the static const byte. And it seems to have worked.

per1234 commented 8 years ago

const byte payloadWithoutReleaseLength = static_cast(strlen(EtherEventNamespace::payloadWithoutRelease));

With my fix the NoAuthentication example sketch compiles to 15386 flash and 515 globals with no compiler warnings caused by EtherEvent(Ethernet causes a bunch but that's the original library author's doing). With your fix it compiles to 15428 flash and 518 globals and has the "non-static data member initializers only available with -std=c++11 or -std=gnu++11 [enabled by default]" compiler warnings for all the changed lines. In fact just removing the static without doing the static_cast has the same effect. That warning makes me think there could be some problems with older IDE versions. So I think I will stick with my fix. To be honest I'm a little out of my depth with some of this class stuff, I just kind of stumble around until I find something that works but never really know what the best solution is. I guess that's a downside of not learning this stuff "the right way". I just try to evaluate based on compile size, memory use and speed.

hmmm, is there a code switch for this thing

yes, see the code section of https://guides.github.com/features/mastering-markdown/

kdschlosser commented 8 years ago

Ok. I do the same poke about to see if I can get it to run. Unfortunately I do know there are lots of screwy issues between versions of the arduino IDE. Especially the 1.6x bunch. It's a nightmare for developers. As I have said I have 5 month experience. At any kind of programming. So that problem is beyond me. I just read. And that's what I did to find a solution. Ilbiet not the best one. But it worked on my end. That's why I have said before. I tend to redo alot. Like my remote sketch. Redid that again last night. And I am down to like 200 or 250 lines. 1.46K memory use. Running pronto IRLib MD5 ethernet spi etherevent timer

kdschlosser commented 8 years ago

And I think the code switch is probably

[code] void test (){ If (blah) {} } [\code]

kdschlosser commented 8 years ago

Nope lol

kdschlosser commented 8 years ago

Nope they dont. Just indenting.

per1234 commented 8 years ago

It's a nightmare for developers

It's like a game of whack-a-mole, I fix a bug on one version and it causes a different bug on another. I'm trying to make it work with 3 different versions of the Ethernet library and every IDE version back to 1.0.6 and then I have a queue library for EtherEvent that has to be tested too. Still struggling with making unauthenticated option work with that one. I just found another unrelated bug with EtherEvent on 1.6.5r2 that doesn't show up on 1.6.6.

And I think the code switch is probably

wrap single lines in single backticks and multiple lines in three backticks

kdschlosser commented 8 years ago

Problem is things clash. And sometimes it's just not worth the effort to make it that backwards compatible. Unless ya don't got something more interesting to do other than rip your hair out. Idk. I'm getting up there in age. I need all the hair I got. So I have learned to just do something better. And usually in the process of doing that I will come across a solution to the other.

How come you use a bunch of single variables in EG? Instead of a dictionary??. I just prefer formatted the text on the arduino to be exactly the way a Dictionary is in python. Then plop it into the payload in EtherEvent. And just have ast.literal.eval () process the eg.event.payload. Works good. And everything is contained in one place with nice friendly named as keys. And what I did was like this.

{Power :{ Samsung:Off, Harman:Off}}, {Volume:{Up:0x5D,Down:0x5A,Set:-50dB}}

And so on and so forth.

per1234 commented 8 years ago

sometimes it's just not worth the effort to make it that backwards compatible

Yes, especially when I will never go back to using any of the old versions(I always use the latest hourly build of the IDE so I can help find bugs) and I don't think there are many other users of my libraries. I do like to try to make my public code as compatible as possible but sometimes there are tradeoffs. Now I even have to think about compatibility with other architectures. I just had to make some changes to EtherEvent because I found that bugs in previous versions of the IDE cause things that should work fine to not compile. The trouble is that a lot of times upgrading the IDE breaks code so there are still a lot of users on 1.0.x because they don't want to make the transition. I'm not super thorough in testing every change in my public code but if I find a problem I try to fix it.

How come you use a bunch of single variables in EG? Instead of a dictionary?How come you use a bunch of single variables in EG? Instead of a dictionary?

I didn't know about dictionary, I know very little python, just enough to get by in EG. My events on my Arduino are only 1 byte each so that's why I did it like that and then I just did the equivalent in EG.