BOSSAC support on Windows

joaodullius commented 3 years ago

I can build successfully to the arduino_nano_33_ble board, but when running west flash I get the following:

What would need to support BOSSAC on Windows?

nandojve commented 3 years ago

Hi João,

Are you using WSL ?

joaodullius commented 3 years ago

Hi Gerson,

No, I'm using Windows 10 native. WSL has no support to USB at the present moment.

nandojve commented 3 years ago

Ok, good, assuming you've read programming-and-debugging, I can imagine that it could be a PATH problem, maybe? Have you succeed programming using manual command line with bossac from Arduino version?

joaodullius commented 3 years ago

Hi Gerson,

Yes, I read it. I do have Arduino bossac.exe in my path, so I can run the following sucessfully:

PS C:\Work\nrf\v1.6.1\zephyr\samples\bluetooth\beacon> bossac.exe -d --port=COM7 -U -i -e -w .\arduino\zephyr\zephyr.bin                     
Set binary mode
version()=Arduino Bootloader (SAM-BA extended) 2.0 [Arduino:IKXYZ]
Connected at 921600 baud
identifyChip()=nRF52840-QIAA
write(addr=0,size=0x34)
writeWord(addr=0x30,value=0x400)
writeWord(addr=0x20,value=0)
version()=Arduino Bootloader (SAM-BA extended) 2.0 [Arduino:IKXYZ]
Device       : nRF52840-QIAA
Version      : Arduino Bootloader (SAM-BA extended) 2.0 [Arduino:IKXYZ]
Address      : 0x0
Pages        : 256
Page Size    : 4096 bytes
Total Size   : 1024KB
Planes       : 1
Lock Regions : 0
Locked       : none
Security     : false
Erase flash
chipErase(addr=0)

Done in 0.004 seconds
Write 103536 bytes to flash (26 pages)
[                              ] 0% (0/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0, size=0x1000)
[=                             ] 3% (1/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x1000, size=0x1000)
[==                            ] 7% (2/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x2000, size=0x1000)
[===                           ] 11% (3/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x3000, size=0x1000)
[====                          ] 15% (4/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x4000, size=0x1000)
[=====                         ] 19% (5/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x5000, size=0x1000)
[======                        ] 23% (6/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x6000, size=0x1000)
[========                      ] 26% (7/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x7000, size=0x1000)
[=========                     ] 30% (8/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x8000, size=0x1000)
[==========                    ] 34% (9/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x9000, size=0x1000)
[===========                   ] 38% (10/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xa000, size=0x1000)
[============                  ] 42% (11/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xb000, size=0x1000)
[=============                 ] 46% (12/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xc000, size=0x1000)
[===============               ] 50% (13/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xd000, size=0x1000)
[================              ] 53% (14/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xe000, size=0x1000)
[=================             ] 57% (15/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0xf000, size=0x1000)
[==================            ] 61% (16/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x10000, size=0x1000)
[===================           ] 65% (17/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x11000, size=0x1000)
[====================          ] 69% (18/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x12000, size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x13000, size=0x1000)
[=======================       ] 76% (20/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x14000, size=0x1000)
[========================      ] 80% (21/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x15000, size=0x1000)
[=========================     ] 84% (22/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x16000, size=0x1000)
[==========================    ] 88% (23/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x17000, size=0x1000)
[===========================   ] 92% (24/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x18000, size=0x1000)
[============================  ] 96% (25/26 pages)write(addr=0x34,size=0x1000)
writeBuffer(scr_addr=0x34, dst_addr=0x19000, size=0x1000)
[==============================] 100% (26/26 pages)
Done in 5.141 seconds

But I fail using west flash:

I took a look at bossac.py and it sends this "BOSSAC runner not support on Windows!~ by simply detecting that it is a Windows plataform: https://github.com/zephyrproject-rtos/zephyr/blob/97e1d60f66bb44f7fc3b55d0572bfbd4e3a815a9/scripts/west_commands/runners/bossac.py#L254

So, I was wondering what would need to be done to BOSSAC being supported on Windows. I see lot´s of Arduino usage on Windows, so it would be good to support this...

nandojve commented 3 years ago

João,

Current Zephyr bossac runner is not supported on Windows, most because of WSL. As you can see, it is not so difficult to enabled it. Maybe you could enabled it and update docs. I can help you in this process.

Could you run west -v flash .... on verbose mode and share results here, please?

Commend lines L254-L258 to run some tests https://github.com/zephyrproject-rtos/zephyr/blob/97e1d60f66bb44f7fc3b55d0572bfbd4e3a815a9/scripts/west_commands/runners/bossac.py#L254-L258
bossac runner should find your version, instead Zephyr one, if not
try set --bossac when invoking west to make sure your will use proper configs.
you may need set --bossac-port to indicate COM7

joaodullius commented 3 years ago

Gerson,

I commented the Windows checking lines and it is "almost" working.

PS C:\Work\nrf\v1.6.1\zephyr\samples\bluetooth\beacon> west -v flash -d arduino --bossac-port="COM7"
ZEPHYR_BASE=c:\Work\nrf\v1.6.1\zephyr (origin: env)
-- west flash: rebuilding
cmake version 3.17.3 is OK; minimum version is 3.13.1
Running CMake: 'c:\Work\nrf-tm\v1.6.1\toolchain\opt\bin\cmake.EXE' --build arduino
ninja: no work to do.
-- west flash: using runner bossac
runners.bossac: C:/Users/joaod/AppData/Local/Arduino15/packages/arduino/tools/bossac/1.9.1-arduino2/bossac.exe --help
runners.bossac: C:/Users/joaod/AppData/Local/Arduino15/packages/arduino/tools/bossac/1.9.1-arduino2/bossac.exe -p COM7 -R -e -w -v -b 'arduino\zephyr\zephyr.bin' -o 65536
Erase flash

Done in 0.001 seconds
Write 103536 bytes to flash (26 pages)
[==============================] 100% (26/26 pages)
Done in 4.683 seconds
Verify 103536 bytes of flash
[==============================] 100% (26/26 pages)
Verify successful
Done in 0.329 seconds
Set boot flash true

It is calling the right bossac from the path (not sure if I should keep it on the path or specify it on the --bossac parameter.) The issue is that -o 65536 parameter when west calls the bossac. If I try the same command without this offset (bossac.exe -p COM7 -R -e -w -v -b 'arduino\zephyr\zephyr.bin'), it works fine.

This 65536 relates to the 0x10000 that is the address of the code partition on zephyr,code-partition. As this address already appears "inside" the zephyr.bin, by calling the bossac with the -o 65536 we are "double offseting" the application to 0x20000, instead of 0x10000.

Is this how the zephyr bossac also works? I think we need either :

to move the partition to 0x00000 on the board dts for later on offset on bossac, or
leave the partition on 0x10000 and somehow don´t offset on bossac...

joaodullius commented 3 years ago

Just to be clear, this is the board (arduino_nano_33_ble) flash definition in the dts. It points to the 0x10000 that is where from the bootloader calls the application from.

&flash0 {

    partitions {
        compatible = "fixed-partitions";
        #address-cells = <1>;
        #size-cells = <1>;

        code_partition: partition@10000 {
            label = "code";
            reg = <0x10000 0xf0000>;
            read-only;
        };
    };
};

nandojve commented 3 years ago

João,

(not sure if I should keep it on the path or specify it on the --bossac parameter.)

You can set bossac path directly on cmd line in case you have path problems.

Is this how the zephyr bossac also works?

Zephyr follow shumatech/BOSSA@dbdd088 . There was a BOSSAC RFC that explain better some aspects.

Anyway, it seems there is a regression from #34947. This board is supposed to work with bossac, see 33523. I'll suggest focus on windows support first, since you have a temporary solution, and than later explore regression.

Do you have all info to setup bossac to run on Windows native?

I was thinking add a topic here that explains how setup bossac for Windows.

joaodullius commented 3 years ago

Gerson,

I just realized I'm using zephyr from Nordic´s NCS, so that´s a 2.6.0 tag and does not have the https://github.com/zephyrproject-rtos/zephyr/pull/34947. I'll give it another try using the head from the zephyr repo.

Update: On the Zephyr head I was able to west flash using bossac (by commenting those bossac.py lines) and the offset was correct.

So, we can say that we can support bossac under Windows native (at least the Arduino one). Would it make sense to PR the removal of that message? All I did was add the bossac.exe that was installed with the Arduino IDE to the path.

nandojve commented 3 years ago

João,

I just realized I'm using zephyr from Nordic´s NCS, so that´s a 2.6.0 tag and does not have the #34947.

Ok, good know that there is no regression.

Requirement List:

Know Windows environment type: WSL or Native. This check is necessary to provide proper info to users because, as you know, WSL doesn't work. Not because of USB but because west can't call windows apps from there. At least, that is the last info I have.
Add instructions at bossac documentation to allow users find the step by step Windows Native configuration.
Add board instructions to complement bossac configuration using Arduino version. You can link board to bossac documentation for a complete view.
[optional] create a backport for 2.6.
No tests are required since we don't have a way to build host on windows. Again, this is last info I have.

Make sure follow Contribution Workflow and Commit Guidelines to link your PR automatically with this issue.

It is good know that bossac runner can run on Windows Native CC: @mbolivar-nordic

joaodullius commented 3 years ago

Hi Gerson,

I just checked and the Python platform.system() when ran in the WSL returns "Linux".

So, this if platform.system() == 'Windows': is a condition only for Windows native and is not blocking running bossac on WSL. I was not able to find any python function that could differentiate being on "Linux native" from WSL.

nandojve commented 3 years ago

It seems the correct detection of WSL is a nightmare https://github.com/microsoft/WSL/issues/423. The problem is that now contaminate Linux detection since west should decide between WSL or Linux. This detection should be made available for all west software, on my opinion, but it will be better listen @mbolivar-nordic before you invest time on this.

My initial thought is make a method available that combine this two solutions: 1 - https://github.com/microsoft/WSL/issues/4555#issuecomment-813062697 2 - https://github.com/microsoft/WSL/issues/423#issuecomment-887928913

With both, it is possible cover the majority ( if not all ) use cases and know WSL version inclusive, fallback is Linux.

joaodullius commented 3 years ago

Hi Gerson,

After analysing the above suggestions, I believe the bellow code will work to identify WSL:

import platform
import os
if 'microsoft' in platform.uname().release.lower() or \
    os.getenv('WSL_DISTRO_NAME') != None or \
        os.getenv('WSL_INTEROP') != None:
            msg = 'CAUTION: BOSSAC runner not supported on WSL!'
            raise RuntimeError(msg)

Would you mind testing it on Linux?

joaodullius commented 3 years ago

Hi Gerson,

I tested and this is working Ok when the bossac.exe is on the Windows %PATH% environment. When passing the path through --bossac= there is an issue happening.

You can see above that when the path is got from the parser, it has ' ' around it (in blue). This prevents Windows from calling the command and we get an error.

If we run the same with bossac.exe in the path, there are no '' in the command:

I'll try to figure how to solve this before moving on.

nandojve commented 3 years ago

Just a remind, Windows uses space on path and it is needed involve full path with ( " ) to work. This can be another case that requires Windows detection to apply proper operations related to path.

joaodullius commented 3 years ago

--bossac is working fine even with spaces in the name.

Will proceed to update documentation.

nandojve commented 3 years ago

Great!

Will proceed to update documentation.

Sure, make a PR let's review it.

joaodullius commented 3 years ago

Hi Gerson,

The Arduino Nano 33 BLE (Sense) already talks about installing the Arduino IDE and using the path to bossac. This also works in Windows native. I'll just add an mention to --bossac-port="COM7" because the default is /dev/ttyACM0 that is not present on Windows.

I'm not sure if there is any other Bossac documentation that I should detail about this.

nandojve commented 3 years ago

I'm not sure if there is any other Bossac documentation that I should detail about this.

I was thinking add a topic here that explains how setup bossac for Windows + limitations.

joaodullius commented 3 years ago

@nandojve, for my better understanding, in the Linux/Mac platforms the bossac used is from the Zephyr? In other words, is it only needed an external bossac through path or --bossac parameter on Windows?

nandojve commented 3 years ago

@nandojve, for my better understanding, in the Linux/Mac platforms the bossac used is from the Zephyr?

Yes, we build bossac and distribute it on the Zephyr SDK (Mac/Linux). However, user can use their own version or any special version, like for the arduino_nano_33_ble board.

In other words, is it only needed an external bossac through path or --bossac parameter on Windows?

No, anyone can implement this protocol and use it by cmd line. Windows is always a special case OS that complicated things.

mbolivar-nordic commented 3 years ago

Hi there,

This detection should be made available for all west software, on my opinion, but it will be better listen @mbolivar-nordic before you invest time on this

Yes, WSL is in general a headache for us right now and it's not officially supported. I would love to see someone take this on across all runners, but based on my employer's current priorities, this is not a priority for me and I will not be doing the implementation myself anytime soon.

In terms of bossac on windows native, I guess it is about this part of the do_run method, right?

        if platform.system() == 'Windows':
            msg = 'CAUTION: BOSSAC runner not support on Windows!'
            raise RuntimeError(msg)
        elif platform.system() == 'Darwin' and self.port is None:
            self.port = self.get_darwin_user_port_choice()

Feel free to send a patch to change that if there's a way to make it work on Windows native.

I originally ported this in ab82264ace163261a549a9c0e4cf55b4eaa5ec58, but it was based off of functionality in a shell script that was removed in https://github.com/zephyrproject-rtos/zephyr/commit/81f2d21faca1b4fc0d8671f4390aba4846d23fb6#diff-24fcc9dc1cd10256e8720f1a7fd6815bc7c25e3789c61d5bfbb5b2e8103b4889. I don't really use it myself.

mbolivar-nordic commented 3 years ago

WSL is in general a headache for us right now and it's not officially supported

But see also #36561, which I will try to make time to get back to this week

joaodullius commented 3 years ago

@mbolivar-nordic , I drafted https://github.com/zephyrproject-rtos/zephyr/pull/37584 to differ from Windows to WSL. Have not submitted yet because I had no time to finish documentation about boccac in Windows. Will try to do this week.

beriberikix commented 2 years ago

FWIW, I just tried this by commenting out raise RuntimeError(msg) and setting the com port/bossac path and it worked.

balajikulkarni commented 1 year ago

@joaodullius, this PR is really useful for flashing Arduino on windows but seems to be still not merged. Is there anything I can help with to close on it?

joaodullius commented 1 year ago

@balajikulkarni unfortunately the original fork was removed, so I had to create a new fork and a new PR.

zephyrproject-rtos / zephyr

BOSSAC support on Windows #37538