Not Properly Receiving Large Payload.

[kdschlosser]

sorry about the book. I am just trying to provide as much information as possible.

I am trying to send Pronto IR HEX to my Arduino Mega 2560. As we know the code is huge. as true hex the code i am transmitting looks like this if stored on the arduino.

const uint16_t prontoCode[78] = { 0x00,0x6D,0x22,0x03,0xA9,0xA8,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15, 0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x3F,0x15,0x3F, 0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15, 0x15,0x15,0x3F,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15,0x15, 0x15,0x15,0x15,0x40,0x15,0x15,0x15,0x3F,0x15,0x3F,0x15,0x3F,0x15, 0x3F,0x15,0x3F,0x15,0x3F,0x15,0x702,0xA9,0xA8,0x15,0x15,0x15,0xE6E};

as a non hex form same code is as follows again if stored on the arduino. const char prontoCode[392] = { "0000 006D 0022 0003 00A9 00A8 0015 003F 0015 003F 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 003F 0015 003F 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 003F 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0015 0040 0015 0015 0015 003F 0015 003F 0015 003F 0015 003F 0015 003F 0015 003F 0015 00702 00A9 00A8 0015 0015 0015 00E6E"};

i believe it's 392 for that char. but at any rate o am setting the buffer size to 512, now I am only using 2.2K of the memory on the Arduino, I have not done a Heap Check. but i am sure there is still no problem. I am also using the TCP Events Plugin for EG I could never get this library to work at all with the standard Network Sender/Receiver. Besides having MD5 errors more often than I would like and now this problem. Other than that, works great. Fast too. So I do thank you for making this Library. ok So back to whats happening, Have a Prefix and Suffix Being sent along with the payload of the pronto code. it receives the payload as an event, and no matter what i set the buffers to it receives no more then 21 of the 392. even if i set both the event and payload buffer to 512 it returns true, And at compile time it makes no difference to the memory size, So I am not sure how this is allocated. I would post code, but in order to be relevent I would have to drop the whole thing, about 45K worth. to much to post. But to give an idea. my includes are as follows for the known libraries. MD5 SPI IRLIB Timer EthernetV2_0 EtherEvent and the utility\w5200 along with some I have created

I am using IRLIB for receiving only and using a modified version of the ProntoIR Library by probonopd its on git Hub. also a simple PIR library for shutting down TV ect... also have a Limited ZWave (MiCasaVerde Vera3) Library for controlling lights and such, as well as simple thermostat for controlling fireplace. and a control system for OSD generated by eventghost, and Serial Control of my Harman Kardon AVR. Now I can store this Pronto Code and 3 others like it in char format and have not a problem running it. But there are some things I would like to add onto and want to free up the memory.

Strange why it comes through as an event tho. and If i send a small payload say 10 digits with the large buffer set. the event comes through just fine then but no payload, if I shrink the payload to say 100 works fine. so I am not sure as to what the exact limit is. and if the problem is being caused by TCP Events or with the Library. But I am thinking the Library only because of the changing the buffer it starts to work. I have also tried this without any of the things I mentioned before just plain jane so to speak, and same problem. just the ethernet and the etherevent and MD5 on a different Mega . But if this is able to get those pronto codes sent from elsewhere. woah what a great utility, as there is no library that is already made that works with my samsung TV codes. Only pronto does. and trying to read the code from flash storage is fickle. hit or miss on working so having it sent would be best. works over serial.

Thanks again. Kevin

[kdschlosser]

Understood. I Don't know what much either, But I'm learning.I just wish the guy who created the database system for my movie and TV show library didn't decide to lock out the database and sell it to "system installers" I would have loved to incorporate that into EG But I gotta Stick with it, they have the largest collection of Movie and TV Show information on the planet. they surpassed IMDB couple years ago. they have the whole gambit all the actors and directors photos. and just about every movie on the planet they have the artwork and data for. crappy thing is they coded the thing for Media Center a dogs age ago. and they haven't changed it. and then locking the dbase stopped anyone from making any kind of plugin for it. it really blows.I would have loved to set up Voice indexing for all of the movies

Just finished up making my cable races they came out pretty good. all for just under 20 bucks. 16 feet worth

[kdschlosser]

I did find a glitch however in EtherEvent, it's concatenating the event.

[kdschlosser]

Oh never mind it's not. what it isn't doing is on TCP Event's side.

it's not putting the prefix and suffix together. it's only sending the suffix. I am using the actual Action Item Box

_EDIT_ I just wanted to thank you for the hard work on removing the auth. no matter what i did even with the timeout i was getting MD5 errors, and it was slow as hell. now it's super fast. I really appreciate it.

[per1234]

Your system sounds super sweet, I'm stoked if some of my code can be a part of it!

what it isn't doing is on TCP Event's side.

The problem is TCPEvents is only sending the prefix if it receives the TCPEvents keyword during authentication because it has to determine it's sending to another TCPEvents because Network Event Receiver doesn't receive prefix, it just uses the default. I can have EtherEvent send the keyword during authentication but with authentication disabled there is no way to do that so I will have to modify TCPEvents. That's no problem because if authentication is disabled then it has to be sending to TCPEvents or EtherEvent as I have no interest in making an unauthenitcated Network Event Sender/Receiver plugin because it's clearly inferior. This is also a way I could potentially enable the Send Data thing but that's probably a low priority for me. But I'll get the prefix support done pretty soon and let you know when because that's reasonable to expect EtherEvent to handle that.

I just wanted to thank you for the hard work on removing the auth

Sure, thank you for giving me the motivation to do it and the help finding bugs. I haven't been able to incorporate it into my system yet because I haven't figured out unauthenticated EtherEventQueue support yet(preprocessor trickyness) but I'm looking forward to the faster communications also.

[kdschlosser]

I am going to write a simple sketch to see how fast this thing can turn around an event. i'll set up a for loop in EG to send out to EtherEvents and set a timer on it and see how long it is before it replys. I know there is some overhead because of EG and what have you but I am running this on a 8 core so I think the weak link will be the 16MHZ Arduino. Simplest way is i am going to have to edit the TCP Events Source, Just an FYI I have not upgraded the TCP Events on another box. and the 2 can no longer communicate. But I am not receiving any errors either,. on either box. I will Upgrade and see if the problem persists

[kdschlosser]

Glitch 1: with TCP Events. if the Receiver has no password entered and the sender is sending with a password it launches an event on the receiver.

22:45:43 DefaultPrefix.quintessence

I am going to change a couple of lines in that TCP Events, that whole crap about lines being sent Unevaluated sucks. so i am going to nest the except to include ast.literal_eval so it will allow dicts and lists and what have you and not just a str

I'll post the code and what lines it's at so if you want to add it you are more than welcome to. Those are the only 2 problems i have found, the 2nd not being from your modifications and not really a glitch, just an annoyance.

[per1234]

I am going to write a simple sketch to see how fast this thing can turn around an event.

I'd be interested to see how much faster it is unautheticated vs authenticated. I've been meaning to do that but haven't gotten around to it. I've done it in the past to evaluate different modifications of EtherEvent. Even with authentication it was surprisingly fast. The only real slow downs I get are when two nodes try to send an event to the same receiver and there is a collision. Then you have to wait for the timeout but if you have the timeout values dialed in it's not so bad. There are a couple of timeouts set in TCPEvents that are way too long so you might want to modify those lines sock.settimeout(5.0) yes that's 5 seconds EG is completely blocked! I've been trying to modify TCPEvents so the timeouts can be set in the configurations but for some reason I haven't been able to make it work yet, my limited python skills holding me back.

I have not upgraded the TCP Events on another box. and the 2 can no longer communicate.

I just tested the latest version of TCPEvents communicating with my previous version and also with the original from the EG forum and it's working fine for me. I wonder what's going on?

22:45:43 DefaultPrefix.quintessence

That's the first step of the authentication. The way unauthenticated mode works is whatever comes in turns into an event. The user just needs to put everything on the network into unauthenticated mode for it to work. I guess you could consider the quintessence event an indicator of improper TCPEvents configuration.

i am going to nest the except to include ast.iterate_eval so it will allow dicts and lists

Cool, I'll check it out and add to my TCPEvents fork.

I've made the modifications to TCPEvents and EtherEvent to allow event prefix to be specified in TCPEvents but I just need to do a bunch of tests to make sure that it doesn't break any compatibility between any combination of authenticated/unauthenticated etherevents or tcpevents and network event sender/receiver before I can release it. That's always the tedious part of writing code I tend to put off.

[kdschlosser]

Well Hell, that was a lot easier than I thought it would be.

here goes. line 49 ish to 59 ish

somewhere in there *add *

import ast

under the SendEvent function

about line 405 ish and _replace this section_

if (evtPayloadStr is not None) and (evtPayloadStr != ""): try: self.eventPayload=eval(evtPayloadStr) except: eg.PrintError("Unable to evaluate the payload. Payload must be a valid python expression (exemple : \"some\"Text\"\")") eg.PrintError("Your string will be sent unevaluated.") self.eventPayload=evtPayloadStr

_with this code:_

if (evtPayloadStr is not None) and (evtPayloadStr != ""): try: if isinstance(evtPayloadStr,dict): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,list): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,tuple): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,file): sendFile = [] with open(evtPayloadStr) as f: for line in f: sendFile.append([line]) _<---the [] may have to be removed haven't tested it_ self.eventPayload = sendFile
except: try: self.eventPayload=ast.literal_eval(evtPayloadStr) except: eg.PrintError("Unable to evaluate the payload. Payload must be a valid python expression (exemple : \"some\"Text\"\")") eg.PrintError("Your string will be sent unevaluated.") print evtPayloadStr self.eventPayload=evtPayloadStr

That should pretty much take care of that god annoying problem. unless it is a seriously malformed python expression. Now i didn't include everything because using isinstance will also pull subclasses so the list is infinate

but I included the biggies

I went a little overboard :=D tested with strings and dicts not tuples and lists and i added files for the hell of it.

it should work with all. files are either put a path and filename into the payload slot, or put a variable containing a path or filename. and it will make a list with one line at each index. now when it is received you have to use

ast.literal_eval(eg.event.payload) in order to convert (if you will) the payload string back into the proper object type.

a file will be put back into a list but as a list with one line per index it is very easily written back into a file. heh... simple file transfer via EG nifty. just thought of that.

[kdschlosser]

dumbass thing. the indents were there as per github to post code. but nooooooooooo it couldn't post the code right. i'm sure you can figure it out.

[kdschlosser]

ok my program tabs, and they said 4 spaces. so now i try 4 spaces. what pain in the %&@

if (evtPayloadStr is not None) and (evtPayloadStr != ""): try: if isinstance(evtPayloadStr,dict): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,list): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,tuple): self.eventPayload=evtPayloadStr elif isinstance(evtPayloadStr,file): sendFile = [] with open(evtPayloadStr) as f: for line in f: sendFile.append([line]) <---the [] may have to be removed haven't tested it self.eventPayload = sendFile except: try: self.eventPayload=ast.literal_eval(evtPayloadStr) except: eg.PrintError("Unable to evaluate the payload. Payload must be a valid python expression (exemple : \"some\"Text\"\")") eg.PrintError("Your string will be sent unevaluated.") print evtPayloadStr self.eventPayload=evtPayloadStr

[per1234]

it couldn't post the code right. i'm sure you can figure it out.

Sure, that's nice to be able to handle different types. If you want multiple lines of code to be properly handled in markdown just add triple backticks on the line above and below. You can even get syntax highlighting by doing ```python at the top

[kdschlosser]

^#@$ it, I give up here is a screen shot of the code

[kdschlosser]

gotta fix one thing. it automatically tries to turn it into a string when you trigger an event from the actual dialog box. i am going to correct that now, works fine from python code tho

[kdschlosser]

it has to be wrapped in " is sent from the dialog box which is not a problem so if you want to use a variable from the box it has to be formatted as such

"{eg.event.payload}"

of or if you want to manually enter the data as say a list.

"[1,2,3,4,5]" "['1','2','3','4','5']" I have found that if you use double quotes on the outside and single inside you don't have to bother with all that backslash crap

[per1234]

I have added support to EtherEvent for the TCPEvents prefix field in the Send an Event action configuration. The newest version of TCPEvents is also required for unauthenticated use but I haven't merged in your modification yet. I'll let you know when I get that added.

[kdschlosser]

I'm not worried, Just make sure you remove the print lines as they are not really needed, I have tested it quite a bit and it seems to work ok without error now, just that annoyance thing with the windows dialog and the double quotes, but i really don't use it anyways and for someone just starting out i wouldn't think it was a big deal.I guess it is more of an annoyance to me. but i don't think that you can directly add variables to that box anyways without it wanting to make its self a string, the nature of windows i think. that's why bitmonster put in the flags ("{eg.event.payload}") so that way someone that coded a plugin for it would have the variable properly passed, and sending a list or a dict would be as normal if it was coded so there really isn't a change.

this building a database(Dict) crap is cooking my brain. I have to many state changes and what puts up a display and on what screen and what doesn't. I am thinking i really need to dive into MySQL for this portion of it. I think it would make it a lot easier. don't know how i would even begin to code out the connection portion of it. I wonder if anyone has made a plugin already for that. maybe i'll go and look

[kdschlosser]

well that was a for not. got the MySQL thing working. problem every query takes 2.7 seconds not good for remote code storage and OnScreen stuff. 2.7 seconds is an eternity. and no one that I can find has a solution. it seems as tho only the python connector has this problem.

[per1234]

That's insane. It's probably not even a large database too. I wonder if you could use an external application and somehow bring the result of the db query into EG. Just off the top of my head I'm thinking you could write the result to a text file and then read from the file with the File Operations plugin. Definitely not the best way to do it but I'll bet it would take way less than 2.7s.

[kdschlosser]

it's not a big database, it's the MySQL Connector for python. to put it simply oracle @$&#ed up on it bad. it's super simple tho. i do have to say that. but you have to query the database, then run a fore loop to collect the data. it's kinda stupid actually. or you can grab it in a single shot but you have to get the whole indexed row. i only want one cell. bit i have to loop through to get it. it's no biggie, I hand keyed my data for a startup database that loads in eg off the init, because the init actually loads a script before any plugins load. weird. but useful especially for this kind of application where you want that database to load first. I am going to code something in that will key off that init to fire the plugins in an order that I want. I have noticed that EG seems to pick the order at random. and if one plugin depends on another ya get the traceback errors. but at any rate. i tested a full loop with TCP Events and EtherEvent, No Auth. payload carrying the millis and just did the simple math between 2 events, and an event on the arduino wouldn't fire unless it received one from EG and vice versa, so it should be pretty accurate. I have nothing going on my lan. and a bare eventghost. I used my 6 core workstation for this so that should be plenty ample. gigabit Ethernet. not like it matters lol. but 12-13ms round trip i am going to use micros and see what it comes up with. I know it's faster than that. it's at the limit of EG i'm sure. because sending a PONG and a PING and a 5 digit payload at 12-13 ms is not saturating the 100mbps lan or the speed of the arduino i haven't done the math but the numbers don't seem to be correct for maxing anything, i could be wrong tho. crazy thing was at first i had it just doing the simple math and spitting it out the serial at like 230kbps. and it slowed it down a whole lot. lol.

[kdschlosser]

12528 on the micros all right in that area +- 50 micros

and Oh i set the Timeout to 500, not sure how much if any kind of a difference it makes without the AUTH

[per1234]

That seems reasonably fast to me. For just sending single events back and forth it I don't think that kind of delay should cause any problems. Unfortunately I did have to add some more lines to availableEvent() when I added TCPEvents prefix field support to handle the TCPEvents serverType payload formatting but it wasn't too much overhead and I think it's worthwhile for increased compatibility. I use EtherEventQueue so if I need to send a bunch of events from the arduino all at once it queues them up and just sends one per loop so it won't cause much blocking.

i set the Timeout to 500, not sure how much if any kind of a difference it makes without the AUTH

EtherEvent.setTimeout() sets the Stream timeout so for unauthenticated operation that is just the ethernetClient.readBytesUntil() in EtherEvent.availableEvent() when it's receiving the event and payload. So if something goes wrong with the sender and it never sends the event(or sends the event without the \n) after the connection is made it will wait for the timeout duration before disconnecting so the timeout plus the time required for the rest of the function to run is the maximum blocking time of EtherEvent.availableEvent(). If the event is received correctly then it doesn't have to wait for the timeout. So it's good to get the value as low as possible without causing a lot of failed event sends(using the queue helps because then there is the option to just try to resend after a bit if it does timeout). Timeout setting is especially important if you have a watchdog reset so that the blocking time will never cause the MCU to reset. There's nothing it in EtherEvent.send() in unauthenticated mode that uses the timeout which makes me realize I should disable that line in unauthenticated mode.

[kdschlosser]

ya know I have always been meaning to ask this. and I know is sounds cookey as hell. but couldn't you use your eventqueue or even code it directly into ether event to pulse a pin when an event needs to be sent and have one pin jumpered to another to trigger an interrupt??then it could be easily made to be non blocking that way. and I do know there is some way to pinout the ws chip or shield to pin 4 for interrupt on receive. just an odd thought

[kdschlosser]

well at 12 ms with a 500 timeout not a single event failed. it burned right quick with the loop, I think without the MD5 you won't have an issue with events stacking up.

[per1234]

but couldn't you use your eventqueue or even code it directly into ether event to pulse a pin when an event needs to be sent and have one pin jumpered to another to trigger an interrupt?

I'm not sure what you mean by when an event needs to be sent. With EtherEvent it needs to be sent when send() is called so there is no need for an interrupt. EtherEventQueue has the option to resend failed events after a delay and a timer interrupt would probably be more efficient than calling queueHandler() every loop to check if it's time but I don't see an advantage to using a pin interrupt. I have considered making availableEvent() and send() non-blocking by allowing other things to be done between the steps of the authentication process but it seems like this could cause the process to take longer and the whole time the ethernet connection is tied up so other devices can't connect and EventGhost is being blocked by TCPEvents so it seems best just to get it done as fast as possible. I've thought about trying to get an interrupt from the wiznet chip but I'm working with the W5100 modules right now which don't have the interrupt pin broken out and I haven't felt like trying to solder a greenwire to the tiny little pin. I am planning to switch over to the W5500 modules though in the new version of my hardware and they do have the interrupt pin broken out but I haven't tried it out yet. I think it's best to leave that implementation for the user to do in their code as it makes EtherEvent less flexible and more complex. I really haven't directly worked with interrupts much as I tend towards keeping things as simple as possible even if I lose a bit of performance. I do use a watchdog interrupt in my WatchdogLog library to write the program address to EEPROM before resetting and I just set up a pin change interrupt to wake up a battery powered board so I'm becoming more comfortable with them.

[kdschlosser]

idk either, it was a long couple of days of being awake at that point. sometimes i do my most creative work when i am like that. but other times my brain starts to go off into la la land. lol. apparently this was the latter of the 2. lol. at least 1/2 of it made sense. got the database portion done. well most of it. its rather large. (for what i am using it for) something like 3 thousand items. set it up so it dynamically builds in eg.for the lighting system in the house and then periodically writes it to mysql. and only reads it from mysql upon booting of EG that way i am not impacted on the query's all the time, it's funny tho, i can pull the whole thing in 2.7 seconds. or i can pull just one item in the same time. makes no sense why it's made that way..

[per1234]

I'm working on merging your payload type modification code into TCPEvents and I have a question. When you say:

now when it is received you have to use ast.literal_eval(eg.event.payload) in order to convert (if you will) the payload string back into the proper object type.

Would it make sense to do that in TCPEvents ServerHandler instead so it doesn't have to be done by the user by replacing the evals with ast.literal_eval here:

    def state3(self, line):
        line = line.decode(eg.systemEncoding)
        if line == "close":
            self.initiate_close()
        elif line[:8] == "payload ":
            if self.clientType=="TCPEvents" :
                try:
                    self.payload.append(eval(line[8:])[0])
                except:
                    eg.PrintError("Unable to eval the payload, receiving the full string")
                    self.payload.append(line[8:])
            else :
                self.payload.append(line[8:])
        elif self.clientType=="TCPEvents" and line[:12]=="dataRequest " :
            dataRequest = line[12:]
            try:
                result=[]
                result.append(eval(str(eval(dataRequest)[0])))

[kdschlosser]

Yeah it does. I had quickly added that into it as a trial for the receive payload. Because it is always a str. I believe TCP Events was written using an older version of EG. I do know how you are about backwards compatability so it would have to be a try except scenario. And putting the actual import ast right in the try except instead of at the top of the code. So of it fails the import it can resort back to the eval. I haven't really studied that portion of the code for long. But it's downright confusing to me. Because of the append you would think it to be a list or a tuple already. But it's not. I have never looked at append in depth but it seems odd to do that to a str. Instead of like so.

self.payload += line[8:]

So I had replaced that with the ast.literal_eval with no effect (I think it produced the same str) so either somewhere in the code is the actual handling of the payload and this isn't idk. Could be I fat fingered something also. I didn't spend much time on it. It just doesn't make sence how it's done now in the code. I don't think that str slicing and the += operator are anything newish to python then again what do I know. LOL

[per1234]

Ok, thanks, that lets me know I'm thinking about it correctly. I'll mess with it a bit because I think doing the whole process in TCPEvents is more user friendly and less documentation. I definitely want it to stay compatible with the Network Event Sender/Receiver plugin but I'm not too concerned with supporting really old versions of EG. The original author basically just pasted the Network Event Sender and Receiver plugins together and threw in the data actions so I think the code could be cleaned up a lot. The thing that bugs me the most is they have the exact same send authentication code 3 times so any modification takes 3x longer and 3x more testing. In C I would just put it in a function.

[kdschlosser]

without a doubt. I tried just now altering it again. even tho I am sending from TCP Events it pipes through as a Network Event Sender. so it falls into the "else" but what happens is i get an unhandeled python exception error. I am going to track down where the actual payload is sent in the event and maybe it can be evaluated there instead. but i think it has a problem trying to add it to the self.payload because the self.payload is actually already a list and that's why the append is there. then the self.payload is used to store the string in one of the indices and then the indicie is pulled to be put into the actual event payload. OOoo I know what i am going to do.

[kdschlosser]

self.payload.append(ast.literal_eval(line[8:]))

[kdschlosser]

because a dict a list or even a tuple can be stored in list .I think

[kdschlosser]

BINGO!!! that worked for sending a list

[per1234]

even tho I am sending from TCP Events it pipes through as a Network Event Sender.

In my last modification of TCPEvents(where I made the prefix field work unauthenticated) I changed it to default to TCPEvent type so it should always be TCPEvents unless send from Network Event Sender plugin.

[kdschlosser]

OK Kewl, I haven't downloaded the newest one yet. but lists and dicts work with that one liner modification testing tuple now

_EDIT_

tuple works with more than one entry ('test') no go. but this works. ('test1','test2') dunno could be a limitation of a tuple. for some reason i remember reading something about a tuple with one item having to be like this tho. ('test',)

[kdschlosser]

man now i am going to have to go through my code again and remove all of the ast's lol but this does make it a hell of a lot nicer and 100% more user friendly

[kdschlosser]

now I do know from my endeavors that if you want to pass a variable as such '{testVariable}' it has to be an eg global and not a local. so it would have to be like this

'{eg.globals.testVariable}'

to pass a local there is no need for the '{}' just pop in the variable name and go. but it appears as tho it should be able to take any kind of variable at the moment, int and float still have to be converted into a str, so that will have to be added to the sender side. Not sure if you have done that already yet or not. I don't want to post code for a modification you may have already worked out

_EDIT_

do you know if that is "proper coding etiquette" to stack functions like that. or is it best to make another variable to hold the information for that ever so brief instance?

self.payload.append(ast.literal_eval(line[8:]))

or is this better

tmpPayload = ast.literal_eval(line[8:]) self.payload.append(tmpPayload)

I would imagine the beforehand example would be better I would think it would just pass a memory location to the next function and not having to allocate more memory again.

[per1234]

Not sure if you have done that already yet or not. I don't want to post code for a modification you may have already worked out

No I haven't done any modifications on the payload handling at all. That would be great to add more commonly used types so they don't have to be converted by the user. Thanks for your work on this!

self.payload.append(ast.literal_eval(line[8:])) or is this better tmpPayload = ast.literal_eval(line[8:]) self.payload.append(tmpPayload)

I would lean towards the first. I would definitely do it that way on the Arduino but on PCs it isn't so much worry about every single byte and clock cycle though of course optimization is always a good thing. I guess working on microcontrollers is good training that way.

[kdschlosser]

gonna have to come up with some kind of a marker that way if a int has to be converted to a str before sending it can be converted back to an int on the receiving side. tried using 'int(12345)' no go there. maybe just an 'i(12345)'

[kdschlosser]

ok kewl. in C it is a lot easier to control what uses and doesn't use memory addresses. it's a bloody mystery in python thus far to me.

and you never know what someone will try and send tho. could be the whole Encyclopedia Britannica in one line. LOL

[per1234]

have to come up with some kind of a marker that way if a int has to be converted to a str before sending it can be converted back to an int on the receiving side.

Even if it just converted it to str on the send and didn't convert back to an int when received that's still one less thing the user has to do. With the marker EtherEvent will have to handle that also right? I don't mind the work but it does have some overhead of speed and memory.

[kdschlosser]

I don't know how that would be handled with EtherEvent. and would it make that much of an impact adding some sort of marker to the typedefs ? and there are also formatting issues with how python handles and "array" you would have to add support for arrays also (if you haven't already done so) to have it add the [ ] to each dimension and if it was an array of chars you would also have to add ' ' to each dimension as well. that's gonna get messy I think. and a dictionary is completely foreign idea to C that would have to be some kind crazy addon and arduino's don't have that kind of memory to handle that.unless a macro was created to handle the dictionary formatting on a per instance basis. actually it could be done that way for all of them. but that is beyond my knowledge. maybe an example function added so that wya they can see how the formatting needs to be done might be a better alternative?

[kdschlosser]

don't want to bloat this awesome coding you have done. gonna have to run some tests i think to actually see how much an impact say something like an int to format it and then back to an int would actually make on the arduino side. i am thinking it's almost gonna be something like .000002 micros or something like that. I basically already did it with the 12.5 ms test. convert the long to a char array then the char array back to a long using atol did the simple math to get the result. well that was one of the tests i did. and the same result of 12.5 +- a few hundredths

[per1234]

I was just referring to the int marker handling in EtherEvent. The impact shouldn't be too bad so I can go either way on that one. I guess the question is how often will a TCPEvents user need a sent int to be received as an int to decide if it's worth it. I agree that handling all the other payload types in EtherEvent is not really feasible and they should just understand that they are only automatically handled for TCPEvents to TCPEvents communication. It's best for the user to just do it in their Arduino code if they need it, that is a good idea to add an example for converting the array.

[kdschlosser]

and this is you baby. I'm just poking my nose about. I personally would leave EtherEvents alone. and let it do what it does. do the formatting code for TCP events. and again if there is a need for someone to use that portion of it then fine. a note does have to be added that if some other type of variable is sent to EtherEvent other than a str the python formatting will be sent with it as a str. example: if i send a list ['test1','test2'] it is going to show up in EtherEvents as a char array like this

testChar = { "['test1','test2']" }

[per1234]

Yes I agree, these are enhancements for TCPEvents and EtherEvent doesn't have to support every possible action just as it can never support python expression evaluation from the Request Data action.

[kdschlosser]

ast.literal_eval evaluates a string looks for the specific variable markers for python and if it finds them creates a variable of the proper type.

[kdschlosser]

Ya know. I think adding a simple checkbox to specify who the recipient is. to have it do the correct thing that way. on the TCPEvents side. I am thinking that might be the solution. if it's going to an Arduino then it will convert to a str and strip all of the python formatting.

[kdschlosser]

have it iterate through a dict and pull the keys and values and comma separate then send, same goes for a list, just comma separate so that way EtherEvent just makes it's usual char array with a comma delimitation or something to that effect

[kdschlosser]

Do me a favor shoot me a link to the latest TCP events, that way i am looking at close to the same thing you are.

[per1234]

Here it is: https://github.com/per1234/TCPEvents/archive/master.zip I have added:

• support for prefix field in unauthenticated mode and sending to EtherEvent(requires the most recent version of EtherEvent)
• unauthenticated data actions
• set timeouts in configuration