OTA update does not work properly

[devicter]

Hi jkandasa! We spent a lot of time with tekka, to sort out this problem.

The problem is that data is continuously lost packets.

If a sketch no larger than 32 kilobyte, all goes well.

If the size is more than that, then the result is always the firmware OTA:! FWP: CRC FAIL

In confirmation of that, I can send a 2 log file.

1. MYSController.log (2 firmware downloaded here 53 and 74 kilobytes)
2. MYCONTROLLER. log (there is only 53 kilobytes, and not just luck!)

Send mail on some logs?

[devicter]

I forgot to add.

This problem remains the same if you use Ethernet GW

[devicter]

and yet, the load time is increased by half.

[jkandasa]

@devicter can you send log files and sketch to my email address? jkandasa at gmail dot com

Also send your atmega chip details

[jkandasa]

@devicter I tested with year old OTA bootloader and works as expected. I have changed radio pin and channel number to default one and disabled LED indication.

I will try with recent OTA bootloader soon and update you the status.

[devicter]

I'm sorry, I'm using the bootloader from Moteino 16 MHz. Because the processing speed twice as high, it seems COM port driver does not have time to process the data. You can use this downloader to take from here. https://github.com/devicter/bootloaders

[jkandasa]

@devicter I do not have DualOptiboot setup ;)

[devicter]

Very sorry, but I say again, it works fine with MYSController https://www.mysensors.org/controller/myscontroller But it does not work for you. Although this problem can be solved very simply.

[jkandasa]

@devicter ok, let me have a look on code level. calculating CRC.

does it working at least one sketch or nothing?

[jkandasa]

If a sketch no larger than 32 kilobyte, all goes well. If the size is more than that, then the result is always the firmware OTA:! FWP: CRC FAIL

I see this from your comments, How do you calculate sketch size?

[jkandasa]

@devicter I have added minor fix, kindly check with snapshot version.

Changed it based on https://github.com/mysensors/MySensors/blob/development/core/MyOTAFirmwareUpdate.cpp#L161~L171

[devicter]

I tried again on the new version. :-( unfortunately the same result.

Log I sent in the mail.

[devicter]

yet there was an error on the page

[jkandasa]

@devicter clear your browser cache to resolve the above issue ^^

[devicter]

did not help!

[jkandasa]

@devicter try this http://forum.mycontroller.org/topic/47/gui-caching-issue-problem

[devicter]

I know how to update the cache ...

-(

[jkandasa]

@devicter I need to the following information to narrow this issue further.

All data from node (add print statement on your MySensors library at line#171)

• _nodeFirmwareConfig.blocks
• crc
• _nodeFirmwareConfig.crc
• firmware hex file, that you used to test.

Kindly do it for two firmware hex files and update result for both. Thank you!

[devicter]

this line should look like this?

OTA_DEBUG(PSTR("OTA:CRC:B=%04X,C=%04X,F=%04X\n"),_nodeFirmwareConfig.blocks,crc,_nodeFirmwareConfig.crc);

[jkandasa]

@devicter yes, this looks good. Also, I have added fix for GUI issue you have mentioned on node edit page. download latest snapshot version and check does this error still exist. Thanks!

[devicter]

screenshots files I sent an e-mail

[jkandasa]

@devicter I did some changes on CRC calculation. You have to upload your firmware once again to MyController server to calculate CRC.

Kindly run with latest snapshot version and report with OTA_DEBUG(PSTR("OTA:CRC:B=%04X,C=%04X,F=%04X\n"),_nodeFirmwareConfig.blocks,crc,_nodeFirmwareConfig.crc);

Thanks!

[devicter]

Again, that is not so

[devicter]

for information on the controller at the Tekka obtained here such figures

[jkandasa]

@devicter Could you please do a favor for me. Run OTA update as follows, I need debug log from all nodes and MYSController to go further.

• Take a hex file for all tests
• Send it from MYScontroller, need MYSController log as well as node log
• Send it from MyController, need node log

NOTE: For each run I see different CRC on node, does it changing every time?

I am really sorry to trouble you again and again. As I do not have this kind of setup with me, I fully depend on you to fix this issue.

[devicter]

Ok, I sent back to the email

[jkandasa]

@devicter Thank you so much for your valuable time!!

[devicter]

For me, too important a positive result.

[jkandasa]

@devicter looks like there is no issue with CRC calculation and in MyController. As per my understanding, problem with MySensors code. When we use SerialGateway it is super fast and completes the request within 500ms. When we go with other types of gateway, problem starts, because other than serial gateway works with polling mechanism. In OTA code we have retry mechanism, MY_OTA_RETRY_DELAY(default 500ms)

OTA:FWP:RECV B=0173
OTA:FRQ:FW REQ,T=0003,V=0013,B=0171
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013007101
OTA:FRQ:FW REQ,T=0003,V=0013,B=0171
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013007101
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:03001300710111924A95E9F746E449834CE44A8348E5
OTA:FWP:RECV B=0172
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:03001300710111924A95E9F746E449834CE44A8348E5
OTA:FWP:RECV B=0171
OTA:FRQ:FW REQ,T=0003,V=0013,B=016F
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013006F01
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:030013006F010E945421809198029091990264E0880F

From the above debug log, I see that block 0x172 data written for block 0x171 as well and 0x171 never used. This leads CRC error. I hope with MyController you didn't use Serial gateway and with MYS controller you have used serial gateway.

I have two ideas to fix this issue (I never tested this, I do not have hardware)

Options 1 (This is not a perfect fix):

Increase MY_OTA_RETRY_DELAY to 2s or above. (based on your intranet/internet speed)

Options 2 (this is may be a perfect fix):

Change the following line as mentioned, https://github.com/mysensors/MySensors/blob/development/core/MyOTAFirmwareUpdate.cpp#L102

_flash.writeBytes( (firmwareResponse->block * FIRMWARE_BLOCK_SIZE) + FIRMWARE_START_OFFSET, firmwareResponse->data, FIRMWARE_BLOCK_SIZE);

Test with latest MyController server. and do change on MySensors code as mentioned above.

Thank you!

[devicter]

I did everything you asked. But, unfortunately, the result is negative. ;-(

[devicter]

Oh, I do not quite understand, I still had to install an updated version MyController?

[jkandasa]

@devicter Yes, you have to use most recent SNAPSHOT version, I have revert back CRC changes on recent build. Which is must. Kindly try once again with most recent SNAPSHOT build. Thank you!

[devicter]

testing stages

1. I put the latest version MyController server
2. flashed your gateway with amendments #L102
3. flashed your microcontroller with amendments #L102

356613 OTA:FWP:FW END
357874 OTA:CRC:B=04A8,C=1E21,F=DEA5
357877 !OTA:FWP:CRC FAIL

Result old.

[jkandasa]

@devicter I could not reproduce this issue locally. We have some changes on SNAPSHOT version regards OTA. Can you try it now?

[jkandasa]

looks like reporter not interested on this issue.