FIXED: Generac will begin encrypting evolution 2.0 controllers starting with firmware 1.10. Will this block genmon use on Evolution 2. Also all new evolution 2 generators will be shipped with encrypted controllers.

[grsthegreat]

Use the template below if you have an issue or want to report a bug. If you have a question or a feature request you can ignore the questions below.

NOTE: If you are having issues with your serial connection, please read this page before posting:

https://github.com/jgyates/genmon/wiki/3.6---Serial-Troubleshooting

If you are having other issues, please see the following page:

https://github.com/jgyates/genmon/wiki/3.5---General-Troubleshooting

If you need to send you logs and registers to the developer, if you email is setup and working properly you can click send your logs on the About page in the web interface.

Expected Behavior

{Please write here}

Actual Behavior

{Please write here}

Steps to Reproduce (including precondition)

{Please write here}

Screenshot or Pictures relating to the problem (if possible)

{Please write here}

Your Environment

• Generator Model: {Please write here}
• Generator Registers: Submit via the About page
• Genmon Version: {Please write here}

[liltux]

I don't believe you can encrypt modbus rtu

[grsthegreat]

i dont know. according to the email i received, there doing this to stop 3rd party access to controller information. it will also prevent the use of Generacs own wireless monitor systems and only allow use of their proprietary wifi and mobile link systems. it wont effect evolution 1.0 controllers, as they need usb assisted updates, but i wont be updating my 1.0 system any more.

[liltux]

Was this an email from Generac?

[grsthegreat]

yes. as a heads up. man, i wish i didnt accidentally delete it.

[grsthegreat]

your post had me go look up the post recap from the march webinar on dealer connection...lo and behold the mention of port encryption IS noted there. it definitely states there that 3rd party connectivity solutions will NO LONGER be able to have automatic access to receive information and controlling generator. they will be introducing port encryption. this is to start 1st quarter 2020. The email told me that they will push this update to installed 2.0 controllers also. 2.0 evolution generators that are updated will encrypt the ports. so my customers that are using Genmon are probably going to be screwed.

personally, i will not be updating any controller that has version 1.0 evolution AND Genmon (including my own).

[liltux]

@grsthegreat sounds challenging, was there mention of the liquid cooled generators?

[swvalenti194]

Does this mean it'll automatically get pushed to the controllers or it would have to be manually done? Anyway to block this if it's automatically done OTA? Dumb question but how could they push OTA as I don't think my generator has any comms with Generac?

[rnatalie]

It’s not even possible on the old Evolution 2.0. I don’t know if the Sync 3.0 version will autoupdate or not.

On May 11, 2020, at 10:29 AM, swvalenti194 notifications@github.com wrote:

Does this mean it'll automatically get pushed to the controllers or it would have to be manually done? Anyway to block this if it's automatically done OTA? Dumb question but how could they push OTA as I don't think my generator has any comms with Generac?

[grsthegreat]

all i know is the evolution 2.0 are wifi accessible, and currently use the wifi to do the forced updates. with the V1.0 i had to upload all updates to a flash drive and install them when i serviced generators. i do not have to do this to the V2.0 units as its done thru wifi.

now if the generator isnt connected to the wifi at home, i guess the update wont take effect. the only problem is there are updates that are critical for these units. Generac has pushed thru several critical updates over the years that affect generator operations and when you do updated they include all changes as you cant pick and choose. luckily the V1.0 probably wont see many needed updates in the future so my unit is probably safe, but im worried about some of my clients that i talked into getting Genmon units instead of generacs mobile link setup.

the last liquid cooled unit i installed last year had an evolution controller but was not wifi capablle, so i would assume it will not be affected. older liquid cooled units that i was installing had versions of the Nexus controllers so wont be affected.

[zekyl314]

As a home owner, do I have the ability to get these 1.0 updates myself?

[grsthegreat]

As a home owner, do I have the ability to get these 1.0 updates myself?

no, they are accessed thru Generacs dealer service portal. they have to be installed thru dealer portal on the generator. if you use a dealer for service/valve adjust, etc they are supposed to provide the updates.

i have seen them sold on flash drives thru ebay in the past... not sure if they still are.

current evolution 1 firmware is V1.21

i dont know why generac wont allow homeowner updates, but im sure it has to do with them not wanting untrained people to access dealer area on controller, as they can cause havoc on the controller if they mess with the wrong parameters...such as frequency

[liltux]

@everyone: Evo/Evo1.0 or Sync 2.0 are the same controller. These controllers do not have WiFi capabilities and firmware must be updated manually via USB. Homeowners need to contact their local dealers for firmware. It does not require a dealer access to update firmware. Evo2.0 or Sync 3.0 are the same controller. These controllers do have WiFi capabilities. The WiFi can be turned off. These controllers are firmware updatable via WiFi push OTA and via USB. EVO 1 type controllers only have, to my knowledge, RS232 capabilities. EVO 2 type controllers have a RS232 and RS485 Port.

[grsthegreat]

you know, your right. i have always installed updates thru dealer input area, but now that i think about it they can also be updated thru the standard edit screen where you change time/date. i just never access it thru that area because i do my updates while accessing dealer screen and resetting maintenance push notices screen and checking rpm, hz and voltages during test run.

[jgyates]

Just curious. Is there a way to downgrade a Evo 2.0 controller's firmware from the console via USB? I assume Evo 2.0 still has USB firmware update capabilities? If yes, then people who have auto-upgraded 2.0 controllers could downgrade to keep using genmon, assuming they could get their hands on older firmware.

[grsthegreat]

Not to my knowledge. When you install usb drive the generator checks to see if the installed firmware is older or not. Ive never seen where you can downgrade it. I once had a unit tell me the firmware was the same as existing firmware and all it prompted was exit.

[buzzshot]

I haven’t seen what the update software looks like but if it’s not an exe I am sure you could mod the old version to look like the new version. And then dump that to the generac. That is if some one has an old version copy kicking around

[liltux]

@jgyates if you have an old revision I have been able to load an older firmware into controller. I have done this on both Evo1 and Evo 2. It does require wifi disabled on the Evo 2. The Evo 2 will compare flashdrive to remote server and only install the latest version.

[hokie21]

I would have thought that selecting Modbus rather than a proprietary protocol would have been because Generac wanted their products to integrate easily with 3rd party monitoring and control systems. There are other monitoring solutions in addition to Genmon that this change will break. It looks like https://www.omnimetrix.net/ is also using the Modbus connection to Generac generators. https://www.omnimetrix.net/wp-content/uploads/2018/05/IM-706-TrueGuard-PRO-NEXUS-Panel-Installation-Rev-F.pdf.

Hopefully Generac dealers as well as the folks at Omnimetrix are making some noise about this with Generac.

[lakee911]

It's quite possible that their "encryption" is something like a shift cipher or simply bastardized Modbus. I bet that it won't be tough to crack.

[UPS99]

There can't be much extra storage, CPU power, or free memory to do any serious encryption. They could, of course, copyright some part of the code, making it more difficult to distribute it.

[skipfire]

Is it known what firmware version will introduce this? I just disabled the WiFi on mine, but no clue if I have the encryption version yet and I have all the parts on the way to hopefully get this going.

[grsthegreat]

No, but the email i received said its coming in a future update soon.

[lakee911]

What's the current version? Presumably, the next version will introduce it. If nothing else, the current version is the last compatible version until we know better.

On Sat, May 16, 2020, 8:33 PM grsthegreat notifications@github.com wrote:

No, but the email i received said its coming in a future update soon.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jgyates/genmon/issues/374#issuecomment-629724797, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKY4J3Q325BH6IPGAYOWRDRR4WHJANCNFSM4M4HDO5Q .

[liltux]

Evo 1 is 1.21 Evo 2 is 1.9 Note they can change the firmware and not the revision number(if they so chose). The most recent update to Evo 2 was 1.9 (build ###) but it still only shows up as 1.9.

[UPS99]

Here is more info about the encryption:

https://fleetsupport.zendesk.com/hc/en-us/articles/360042081271-Controller-Encryption-FAQ-s

[hokie21]

Interesting that they only care about fixing the "security" problem on air cooled and not liquid cooled generators. @jgyates can you speak to Generac and see if they might consider "authorizing" Genmon? I'd be curious what kind of response you would get from them on this.

[jgyates]

I would be happy to reach out to them if I had a contact. Other than going to their web site and filling out the contact us form, I don't have any contacts with Generac. My guess is that they will ignore me. The only reason I can see for this encryption is to drive people to use their service.

[lakee911]

Send me your email address, Jason, and I'll send you the contact that I have. Maybe he'll be able to get you pointed in the right direction...

On Tue, May 19, 2020, 8:59 PM jgyates notifications@github.com wrote:

I would be happy to reach out to them if I had a contact. Other than going to their web site and filling out the contact us form, I don't have any contacts with Generac. My guess is that they will ignore me. The only reason I can see for this encryption is to drive people to use their service.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jgyates/genmon/issues/374#issuecomment-631170712, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKY4J6OE6477D2PFRXQIZLRSMTOJANCNFSM4M4HDO5Q .

[jgyates]

got it.

[Spuds]

Received a message last night from Mobile Link

Your generator has successfully uploaded a system upgrade. Your generator is ready to run with the latest software updates.

This update seems to have broken the communications. The generator now shows firmware 1.10.59 and the genmon status just shows Modbus Exceptions : xxxxx which is continually increasing.

[liltux]

Well it was fun while it lasted....

[UPS99]

The encryption argument by Generac is not valid. No one would have access to hack modbus. It they legitimately felt the need for added security - encrypting the ethernet / Wi-Fi traffic would be the only effective place to do it. This is very likely related to their new warranty requirements, which only provide the full extended coverage to customers who purchase and pay for the MobileLink plan and keep their generator connected to Wi-Fi.

[skipfire]

I actually came here because the first year of my MobileLink ended and I hated how little info it provided. Got the parts today, this weekend I'm building the cable and hopefully getting this all working on a Pi Zero (hopefully).

[grsthegreat]

just make sure you dont hook the generator up to wifi if its an evolution 2.0 controller. if its the older evolution 1.0, dont do any more firmware updates.

i just set aside one of my new evolution 1.0 spare controllers(purchased last year) onto my personal shelf in the shop. now i wont have to worry that a replacement V1.0 controller will be encrypted and wreck havoc on the genmon unit in my own 22kw unit.

[skipfire]

I disconnected it from WiFi as soon as I saw the post, so hopefully I got it before the update went to it.

[jgyates]

@skipfire if you are performing a new install, don't use V1.14.09 to do the install. I checked that in last night, but I found a bug that has been fixed in V1.14.10 (checked in a few min ago). If you did use V1.14.09 let me know and I can tell you how to fix it. The bug wires an invalid command line option to the crontab on a new install.

[skipfire]

Thanks, I haven't downloaded it yet, hoping to chew through the whole process including cable on Saturday (if the weather is ok). I also still have to take apart the generator enclosure to find the plug.

[ebovine]

I'm confirmed as having this problem as well. Unfortunately I didn't see the threads prior to them pushing the update.

[mhump711]

Received a message last night from Mobile Link

Your generator has successfully uploaded a system upgrade. Your generator is ready to run with the latest software updates.

This update seems to have broken the communications. The generator now shows firmware 1.10.59 and the genmon status just shows Modbus Exceptions : xxxxx which is continually increasing.

Can confirm mine stopped working and I am on 1.10.59 :( Sad they went through all this to make a quick buck.

[grsthegreat]

• Well it’s official... i am no longer installing Generac generators. I had a talk with my dealer rep and let them have it with what i thought about their money grab with encryption. I am no longer going to bid installing generac units. If i decide to Keep installing units, im going to switch to Kohler units. I have 6 customers that want generators, and im going to have to contact them And cancell The planned installs... no signed contracts yet, but was going to sign next week.

I may just stop installing generators all together.

I am still going to service Generac, but only for my existing clients. Im not taking on new Generac customers.

[jgyates]

If anyone with a Kohler unit was to work with me I can add support to the project.

[hokie21]

If you want to register your displeasure with Generac, here is the contact form: https://www.generac.com/service-support/contact-us/online-form

[ebovine]

Generac will likely use security as a justification for this. As one in the industry with ICS experience, their actions address an incredibly low risk item and are aimed at securing subscription revenue. Promotion of kohler over Generac appears to be a good strategy going forward. My installer is already tired of dealing with Generac’s certification game.

[jonathanpisarczyk]

I have lost access to my generac. So annoying! We should start a change.org ?

[hokie21]

I think a complaint to Generac directly using the online contact form I linked above will probably be more effective. I suspect that unless the Generac dealers revolt and stop specifying Generac for new installs, Generac has their minds made up on this.

[liltux]

Contact Generac directly and they may unlock your controller? One offs I doubt will cause Generac problems, but they may force you to void your warranty to use third party product.

[gzebrick]

I wonder if the Honeywell-branded version of the Generacs will also have the same encryption? (I have a Costco/Honeywell/Generac 17KW with an earlier Evolution-1 so as long as I don't manually update it I think I'll be safe)

[UPS99]

"Generac will likely use security as a justification for this. As one in the industry with ICS experience, their actions address an incredibly low risk item and are aimed at securing subscription revenue. Promotion of Kohler over Generac appears to be a good strategy going forward."

Encrypting Modbus is absurd. If this was really about security, they might add encryption to the Wi-Fi traffic.

Generac has targeted their own customers as the enemy. In particular business users of the expensive large scale monitoring solutions OmniMetrics and Power Telematics. Complaints may fall on deaf ears. Most effective might be if a large stockholder heard the details...

[jonathanpisarczyk]

How about a class action lawsuit?

[hokie21]

I think that industrial customers who use SCADA interfaced to Modbus are primarily using liquid cooled generators. That's the case with my company. That is likely why liquid cooled isn't included in the announcement.

I think a class action suit would likely result in a settlement with some money for the attorney and we would get free MobileLink service for 6 months. We probably wouldn't get the outcome we desire. If Generac is gong to change their approach, it will need to come from pushback by the dealers.