Some suggestions for a solution to adding better IR to my application.

[kdschlosser]

Dr Martensson,

Let me introduce myself.. My name is Kevin Schlosser and I am the Administrator/current Author of a program called EventGhost. I know you are familiar with EventGhost but do not use it yourself due to it being Windows only (for the time being).

There are several plugins which add IR receiving/sending capabilities to EventGhost but there is one that has been troublesome since the start. That is the MCE IR (eHome CIR) interface. The current version of it makes use of a Windows service called alternateMCEir. This service is written in c code and has a named pipe an application can connect to in order to transmit and receive IR. While it does kind of work.. it also does not. This problem is caused by the c code and also the python code.

I have removed the use of the service and written a binding in pure python as a replacement.. This works very well and has eliminated a plethora of issues. The old system was written incorrectly and was using the wideband receiving mode for all incoming IR transmissions. This receiving mode is a short distance mode 3-5" and it also has the AGC (automatic gain control) disabled. This was allowing all kinds of stray IR to be received and this was causing a portion of the problem., I am sure that there were also issues when doing the data conversion to pass the information along the named pipe as well. Now I am able to get the RLC data from the windows API directly. I have also worked out some of the timing issues that existed in the old system...

The other portion of the problem is the decoding capabilities. frankly.. they suck... LOL.. Trying to come by IR protocol specifications is proving to be pretty difficult. and doing a conversion from an existing arduino IR library is proving to be difficult at the very least.

The largest issue that exists with the current system is it does not normalize an IR code to match the specification. because of this the learned codes often will not work all of the time and also the generated short hex that gets shown to the user can change on multiple presses of the same button. I believe that normalizing the signal to exactly match a protocols specification is key to being able to provide the user with the same code when a button is pressed more then a single time. But also when learning IR to blast it.

I have read about that IR notation a little bit and it seems like it would be one hell of a process to write a handler for it in python. I did some searching to see if someone has already made such a thing. I have not been able to locate anything.

I would really prefer a system that is self contained. and would not require me to spawn a process to pass information via a command line argument. I am OK with a shared or static library and if something like that is not available to decode perhaps you may know where specifications for the different protocols can be obtained. I have the "standard" AV type protocols written already documentation is not that hard to locate for those.

I have spoken with @probonopd and making use of the database files he has to use in the creation of a server that the plugin can connect to.. He is OK with me doing that. I wanted to sue the web API he has set up to accomplish my task. But with the large number of users that EventGhost has and most of them using the IR portions of it We came to the conclusion that his hosting plan would probably have a seizure if it ended up getting queried to often. EventGhosts website sits on a dedicated server with plenty of resources available. however the engine that runs irdb is not public so I have to come up with some other way.

I would ideally like to display the actual button name that is pressed on a remote and not hex codes. ther hex is simply not all that user friendly for 1, and I would like to have the IR be polished. the hes codes do not look that great when being displayed in an HA system..

The irdb database is broken up by device/subdevice/function kind of an arrangement which is useless unless I can decode the various protocols properly.

If you have any suggestions or can point me in a direction that would be extremely helpful.

Thank you for taking the time to read this.

Kevin

[bengtmartensson]

Uh, that is a lot...

First, you don't appear to "know" any sane IR hardware. Lars' (bitmonster) favorite device was (is?) the USB-uirt, but I do not know if it is available still. Long story short, try this thing with this firmware. It communicates with the host computer over a serial terminal interface, using plaintext commands. Writing a EG plugin would be a few hours work. It can optionally also decode a few protocols (presently NEC1 and RC5), so you can easily invoke events when say, an RC5, D=5, F=17 arrives. See the dispatcher.

I don't really understand your interest in Irdb, it is used once to help find commands for a "new" device; it is not used during deployment.

Also have a look at JGirs. Combining that with EG would be a killer! (using Jython for running Python?)

Get back to me on this, and we'll we where we go from there.

[kdschlosser]

OK, I Like to write books as you can tell. No I really don't. More can be accomplished with a single message containing a bunch of answers/questions then 20 questions and 20 answers with 11 hours between each one. I try to be as thorough as possible in converting thoughts into words, I am not always successful in this. I also try to make it really simple so "reading into" instead of just "reading" should not be done. I write these things with the knowledge that this is not a private conversation. There are other people that will be reading it. someone that has more knowledge about this topic may find the verbage to be very basic. and a whole sentence may be able to be summed up in a single work or a small phrase. and that is fine and dandy.. when programming what is a really important thing to follow.. Readability!! So i use the long form for others that may be reading not just you and I I do my best to write in a manner that most should be able to understand without talking (typing) over anyone's head.

The MCE receivers actually are "sane", It took someone "insane" to spend the time to hammer out how to use one correctly LOL. It is what the users are using.. I am not the one that decides these things if. If I had the choice of picking a different receiver before I started working on this. I probably would have. But.. Now that I managed to get the binding working properly to the receiver.. That decision might go to these things. It works, and it works pretty easily. the drivers comes with Windows so no issues there either.

I personally have set up Arduinos and EventGhost using an Ethernet Shield That is what I have all over my house. But that does require a pretty decent amount of knowledge to get working. and it was actually a lot more work then getting the MCE IR Binding working properly.

The ability to use an MCE IR receiver is a snap.. you can have many of them plugged into USB as USB allows. 15 foot max cable limit.. not a problem.. Cheap Raspberry Pi, cheap Wifi USB and VirtualHere installed on the thing. USB over IP. This is something the average user has the ability to do with any major complications.

See now I sound like a sales man... !!

I have the problems all sorted out with capturing the data from the receiver. It does it correctly and without error. these receivers do not have decoders built into the hardware. so it has to be done by use of software...

Currently EventGhost maps the decoded IR to friendly text for only the MCE remotes.

I do not know what you remember about EventGhost, You do know the name of the original author. I have never spoken with him. I did try to track him down to transfer the domain names to us when we moved servers.. I was unsuccessful in contacting him.

EventGhost (EG) is an event/macro based software. the events get produced by all kinds of inputs in this case the pressing of a remote control button. the event gets displayed in a log so the user is able to determine what has taken place, these events are also what run the macros. and in the macros are actions or items that are to be run when an event occurs.. The trick here is we also need to be able to determine is a code is a repeat code or if there is a toggle bit involved. EG has looping events that will continue to run the same macro over and over again so long as the repeat codes continue or until a code with a flipped toggle bit is received.

So if i wanted the right direction key on a remote to move the mouse to the right I would do the following. make a macro and add an action to move the mouse to the right. I then tell that macro to run when the event occurs for the remote direction right button being pressed.

so the event would appear as "MCE.Navigation.Right" If there is no code to friendly name mapping the remote code may end up looking something like this "RC6.06.3D4A" those are examples but it gives you the basic idea.

the event is going to be different based on the protocol being used as some protocols have defined device, subdevice and command and others are device and command.

It simply takes the integer value of each of the above and converts it to hex and attaches the protocol name to the front and adds a "." between the different pieces. There is a reason for the "." 's but I am not going to explain the purpose to them at the moment.

IRDB has mappings that can be used to obtain these friendly names. you discussed this in your write up for JGirs.

Ideally I would like a system that can take this RLC IR code

+2338 -591 +591 -591 +1181 -591 +1181 -591 +1181 -591 +591 -591 +1181 -591 +591 -591 +591 -566 +591 -591 +591 -566 +591 -591 +1157 -26207

and have it hand back

Sony12, Cassette, PowerOn

which is what I believe that device 16, func 46 maps to for the Sony12 protocol

If I use IRIrScrutinizer to generate a code for Sony12 device 16 func 46 and export it it is going to provide an "ideal" code or what I call a normalized code. That code is built from the potocol specifications. It is spot on perfect. a receiver in a device that supports the SIRC12 code is going to be designed to operate exactly to the specification (well we hope anyway). there is an allowable tolerance rage because of interference or whatever other environmental or electrical issues may cause a change in the code during transmission. For sake of this rambling lets say that tolerance is +-10%. if the code is not within that tolerance then it is rejected.

If a code is learned we do not know where within that range the code actually resides. It could be at the farthest edges of the tolerance... If we use that code to transmit there is a high probability that it can land outside of the +-10% from the specification that the device uses. This in my opinion a useless code.

But... if I have the ability to feed that code into something and have it spit out the "ideal" or normalized version of it it is no longer a useless code.

If I input an IR code into IRIrScrutinizer when I export it it comes out as the same code I put in... No love there.. I would need to input the code. get the protocol, device, subdevice and function and then feed that back into IRIrScrutinizer in order to get a normalized "correct" code. Now this does not seem like a big deal. unless there is a central server involved.. more data = more $$$ more speed = more $$$... our software is free.. our support is free.. You know the drill on this i do not have to explain it..

example of what I al talking about above. you can skip this if you understand. This is here for others to read if they do not..

I went to RemoteCentral and grabbed a random pronto code.. and pasted it into IRIrScrutinizer and exported it. then i used the same protocol, device and function and generated one and exported it.

Imported pronto code.

0000 0066 0000 000d 005f 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0017 0018 0018 0018 0017 0018 0018 002f 0429

exported RLC from the imported pronto code.

+2338 -591 +591 -591 +1181 -591 +1181 -591 +1181 -591 +591 -591 +1181 -591 +591 -591 +591 -566 +591 -591 +591 -566 +591 -591 +1157 -26207

exported protocol, device, fun from imported code Sony12, 16, 46

Generated pronto code using Sony12, 16, 46 0000 0068 0000 000D 0060 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0018 0018 0018 0030 03F0

Generated RLC using Sony12, 16, 46

+2409 -602 +602 -602 +1204 -602 +1204 -602 +1204 -602 +602 -602 +1204 -602 +602 -602 +602 -602 +602 -602 +602 -602 +602 -602 +1204 -25290

as you can see these do not match. The generates one is an "ideal" code or a normalized code.

I have found that a learned code may or may not work 100% of the time. This is due to whatever external influences occur during transmission. In an ideal world nothing would change.. But this is not an ideal world.

... end of skip

If there is a way to make a single query passing in the RLC and have it hand back

protocol, device, subdevice, function, toggle bit

normalized RLC code

normalized Pronto code

If a code is a repeat code then pass back the protocol and "repeat" informing us that it is a repeat code. I do not know if this is true or not but it seems as tho protocols that use a toggle bit also include the code along with that toggle bit so the first return example can be used.

I keep on using RLC codes because that is the format of the data that I am able to get from the IR Receiver. I can convert this into pronto format is that is hat would nbe needed for an input value.

I would like to not have to spawn additional processes to decode/normalize an IR code. It would be much nicer If I was able to achieve this via a shared lib ideally. and if only a static lib option is available then I would have to write an extension module for python to connect to it.. I would be OK with doing that as well.

IRIrScrutinizer does not seem to have a normalize feature. and this can only be done through multiple queries. I would imagine that having a "clean" code would would increase the success of the device being able to read it.

Now IRDB has those 3 things stored but would require integer values to be able to collect friendly names. So there is going to need to be a decoder that sits in the middle. something that can turn that RLC code into the integer device, subdevice and command which I would then be able to collect the friendly names from IRDB. this lookup would only happen a single time if the code is stored in our database and is not locally stored. if a query is made and a code is found it will then return the friendly names. and these would then get stored locally so further queries for the same code would no longer need to be made.

I can also provide a lookup so users can lookup a code to blast. lookups methods could be as follows.

• brand/model
• brand/device
• brand/device/subdevice
• protocol/device
• protocol/device/subdevice
• brand/function
• protocol/function

IRDB's current file structure database is not ideal for performing all of the query types listed above. So I have been migrating from a file system based structure into a MySQL database

It sounds like JGirs has the potential in terms of ability to provide what I need. The only issue is the mechanism used to make the queries is not really designed for single connection/use It is designed as a server application to handle multiple/many queries.

I do not want to have the IR decoding abilities be dependent on an internet connection. Nor do I want to have addition server load for the decoding process. more load = more $$$. The lookup of friendly names is an option. it would not be a requirement for proper operation of the IR. In fact a user could connect the computer to the internet. press each button on the remote. disconnect the internet and it will continue to work with the friendly names, persistent between application restarts.

Ideally I would also like to avoid launching or installing anything that is going to add additional processes on a users computer. In this growing market of HA we need to make sure that the laws of K. I. S. S. (Keep It Stupid Simple) are followed. if a users sees something running and they do not know what it is.. the likely hood to just kill the process is pretty high.. and with a name like JGirs not really indicating that it is any kind of an IR decoding program increases that chance. I love my users and I love the fact that they do use EG, However there are users that do not work well with technology and frankly are as sharp as a bowling ball when dealing with technology. These cases I end up having them install teamviewer and i will take a look myself.

This is a time consuming process for me. If there is a way I have the ability to minimize that from happening I will always try to go that route. The other thing is the exchange of data that would need to be made between my server and their computer. Unless a connection is held open creating socket connections using Python on Windows is an expensive thing to have to do. It is going to really slow things down. and the connection could not be held open because of the number of concurrent connections to the server.. I know EventGhost is small time and to even have a dedicated server at a data center is not done all to often. We were forced to do this because "unlimited" shared hosting plans do have limits. One are innode caps. and 7000 files all with 32bit names being in the same folder is well above the 1 or 2 thousand cap that typically gets placed (forum attachments). The users are generous and help pay for a large portion of it with reoccurring donations they have set up. and i take care of any shortfall. which happens every month. There are not enough funds available to be able to pay for a server that would have the ability to have that many concurrent connections. Plus I believe that the decoding should be done on their end not that it would be a horribly CPU intensive thing to do on a one by one basis. How it would be with potentially several thousand happening at the same time, don't know, but it would be a hell of a lot higher then one at a time.

The other thing would be the need to have EventGhost run JGirs. Now while this seems harmless it is really not. It is not that hard to cause EventGhost to crash, This is because of it's ability to create python scripts and run them real time. If a child process opens a port and the parent process crashes the child process gets terminated in an unexpected manner or the process gets "stuck" but either way the socket created by that child process can end up getting orphaned. we actually see this happen more then we would like to.. only way to fix an orphaned process in Windows is you have to reboot the computer. No way to "unstick" it. The ports holds a reference to a process that no longer exists. and cannot be forcefully closed.

I do also have to say that the MCE IR Receivers are pretty decent pieces of hardware. they are designed to work in the 30kHz to 60kHz range you have the ability to change the receiving mode from wide band to narrow band depending if you are learning or not. They have to be made to be accurate within 50us. and the narrow band mode has an automatic gain control. They do not use the host processor when blasting or receiving IR signals. The issues were all in the API documentation. once gotten past their price to performance is pretty high.

[kdschlosser]

also EG is not able to be run on Jython because it uses wxPython for it's GUI. there would be a huge conversion that would need to be done. I do not personally care for the heap management and GC in Java. It's actually pretty bad to tell you the truth. when something gets garbage collected in python the memory gets released back to the OS almost instantly. with Java... It is a big question if it will and when it will. I have seen many applications that use Java that once additional memory is needed/used by Java and the application it never gets released back even if that memory is not used again. I do not know how a high level language could call that memory management but Java does. EventGhost is an extremely dynamic program, there is no prior knowledge of what is going to run or how much memory it may or may not use. It could be something the user runs only once a day but it can consume 2-3gb of memory when it performs the operation. Like creating graphs for a web interface is one such thing that can take place.. To not have that 2-3gb released back to the OS when it is not being used by the application has always puzzled me. The only way to have it released is to restart the application. People run EG not on dedicated machines, most users have it running in the background of their every day PC. this is one of the reasons why EG gets used. 79.25% of the computers in houses that have them are running Windows. and every single one of them has the ability to add some form of automation without spending a single penny. and without having to know how to program or manually write configuration files in order to get it to run.. EG provides this ability.

This is a tag line I came up with. "If you know how to drag and drop a file, then you are qualified to use EventGhost"

Lars follow the rules of K.I.S.S. when he wrote EG, he may or may not have knowingly did it.. but he did. EG uses approx 70mb of ram with a decent amount of things going on. if it spikes a high use.. it needs to return back to this number immediately after it is done doing whatever it is doing. and when I say returns back.. I mean what is reported by Windows as it's consumption heap that is allocated is in use by an application and cannot be used by anything other then that application. there for it is still in use by that application doesn't matter if it is actively using it at that moment or not.

[probonopd]

@kdschlosser would it be possible for you to give a 1-sentence summary of what you would like to do, and how harctoolboxbundle could help you do that?

[bengtmartensson]

I think this discussion might be better off in for example ("e.g."!!) the EG forum. I am registered there as "Barf", last post 2007!

Will get back on several of these issues. In the meantime, please check out IrpTransmogrifier that will answer several of you issues, I think.

[kdschlosser]

I just got finished trying IrpTransmogrifier, 1.3 seconds for it to start the JVM and to process the code and print out the results. That is to slow. I am fairly sure it's because of the JVM having to start. Otherwise it does do what I need. I tried using jnius as well and I end up with some error about not being able to allocate memory... when i have 16gb of memory available for use.

[kdschlosser]

also the old domain names no longer work properly because I was not able to get in contact with Lars aka bitmonster to update the DNS records when we moved servers. http://www.eventghost.net

[bengtmartensson]

I have written an answer in the EventGhost forum, and would like to continue there.

For this reason, I am closing this ticket as replaced.