maxgerhardt commented 3 years ago

What kind of issue is this?

[X ] PlatformIO Core. If you’ve found a bug, please provide an information below.

You can erase any parts of this template not applicable to your Issue.

Configuration

Operating system: Windows 10 x64

PlatformIO Version (platformio --version): version 5.2.0a6

Description of problem

The PlatformIO "test" procedure is such that PlatformIO adds its own Unity library to the build.

https://github.com/platformio/platformio-core/blob/834c7b0defe39ff998cd9dc856657aeb837c86ca/platformio/builder/tools/piomisc.py#L353-L361

this however directly conflicts with any framework which already has Unity compiled into it. Such is the case with the https://github.com/arduino/ArduinoCore-mbed in which the precompiled mbed-os library (libmbed.a) has the Unity object files.

Thus it is impossible to e.g. run unit tests with a board = nano33ble framework = arduino configuration without modifications.

Refer to the community topic here

Steps to Reproduce

Create a new Arduino Nano 33 BLE project
Add test\main.cpp from below
Run the "Test" project task

Actual Results

A bazillion linker errors due a conflict with .pio\build\nano33ble\libUnityTestLib.a and .platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a

c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o): in function `UnityBegin':
unity.c:(.text.UnityBegin+0x0): multiple definition of `UnityBegin'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):unity.c:(.text.UnityBegin+0x0): first defined here
c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o): in function `UnityEnd':
unity.c:(.text.UnityEnd+0x0): multiple definition of `UnityEnd'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):unity.c:(.text.UnityEnd+0x0): first defined here
c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErr64+0x0): multiple definition of `UnityStrErr64'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErr64+0x0): first defined here
c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErrDouble+0x0): multiple definition of `UnityStrErrDouble'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErrDouble+0x0): first defined here
c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErrFloat+0x0): multiple definition of `UnityStrErrFloat'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErrFloat+0x0): first defined here
c:/users/max/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.190214/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: warning: size of symbol `Unity' changed from 132 in .pio\build\nano33ble\libUnityTestLib.a(unity.c.o) to 160 in C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o)
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\nano33ble\firmware.elf] Error 1

Expected Results

Running unit tests is possible

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:nano33ble]
platform = nordicnrf52
board = nano33ble
framework = arduino

Source file to reproduce issue: test\main.cpp

#include <Arduino.h>
#include <unity.h>

void simple_test(void) {
    TEST_ASSERT_EQUAL(33, 33);
}

void setup() {
    delay(2000);

    UNITY_BEGIN();
    RUN_TEST(simple_test);
    UNITY_END();
}

void loop() {
    delay(1000);
}

Additional info

valeros commented 2 years ago

Hi @maxgerhardt ! Thanks for reporting. Indeed, the reasons for this behavior are obvious, but I'm not sure how we can solve this problem in a reasonable way. We cannot simply omit our .pio\build\nano33ble\libUnityTestLib.a in favor of the Unity located in mbed.a. We need a complete control over the Unity configuration, this is the main reason why PlatformIO also adds several additional files that implement standard routines (like redirecting test output, etc.) when compiling the Unity sources. On the other hand, we a bit reluctant to dynamically manipulate the libmbed.a archive and remove all references of the Unity code. I believe the best way out here is to ask the maintainers of ArduinoCore-mbed to exclude Unity from their framework package.

maxgerhardt commented 2 years ago

We need a complete control over the Unity configuration, this is the main reason why PlatformIO also adds several additional files that implement standard routines (like redirecting test output, etc.)

There might be special cases where e.g the pre-compiled Unity version is "functionally correct" and would not differ from what PlatformIO would use. Or when there are frameworks which won't remove their internal Unity version. It would make sense to add a fallback option to the platformio.ini to disable using PlatformIO's Unity version completely, like test_use_builtin_unity = no -- the user has to of course understand the implications of this then.

In the Arduino-mbed case (and probably for framework = mbed too?), no adverse consequences were observed by the user when removing the Unity object files from libmbed.a during normal firmware compilation, since the normal firmware does not use Unity, only the unit-testing firmware does. So again, an option to really make PlatformIO remove known Unity object files from the build (while backing up the old file) via an explicit "please do so" configuration option may come in handy if frameworks with precompiled blobs won't remove Unity.

The ArduinoCore-mbed people will surely see a problem with removing Unity since then no Arduino IDE user can use Unity (if they did). They might resolve that though be refactoring Unity into a library and compile it from source. That would be a good alternative.

ivankravets commented 2 years ago

Thanks for the report! See docs https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/examples/custom_unity_library.html

pratiksurgeonslab98 commented 2 years ago

Custom unity framework not working with mbedOS

I need help with running unit tests on platformio with mbedOS framework, I am using the custom unity test framework. I am referring to the following links: custom unity library

3342 and following the steps mentioned in this issue.

Below is the platformio.ini config file [env:nucleo_f411re] framework = mbed platform = ststm32 board = nucleo_f411re build_flags = -DPIO_FRAMEWORK_MBED_RTOS_PRESENT -std=gnu++11 build_unflags = -std=gnu++98 test_framework = custom lib_deps = throwtheswitch/Unity@^2.5.2 The test folder looks like this: /project ├── platformio.ini └── test ├── test_blink │ └── test_main.cpp └── test_custom_runner.py

While running the pio test I am getting the following errors:

Collected 1 tests (test_blink) Error: Traceback (most recent call last): File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/main.py", line 102, in main cli() # pylint: disable=no-value-for-parameter File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/cli.py", line 71, in invoke return super().invoke(ctx) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(args, *kwargs) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), args, **kwargs) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/cli.py", line 138, in cli runner = TestRunnerFactory.new( File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/factory.py", line 67, in new return runner_cls(test_suite, project_config, options) File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/unity.py", line 191, in init self._tmp_pre_upgrade() File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/unity.py", line 201, in _tmp_pre_upgrade pkg = lm.get_package(self.EXTRA_LIB_DEPS[0]) TypeError: 'NoneType' object is not subscriptable

============================================================

An unexpected error occurred. Further steps:

Verify that you have the latest version of PlatformIO using pip install -U platformio command
Try to find answer in FAQ Troubleshooting section https://docs.platformio.org/page/faq/index.html
Report this problem to the developers https://github.com/platformio/platformio-core/issues

============================================================

@valeros @ivankravets @maxgerhardt please help me out.

ivankravets commented 2 years ago

@pratiksurgeonslab98, thanks, we fixed the development version of PlatformIO Core. Could you re-test with pio upgrade--dev?

pratiksurgeonslab98 commented 2 years ago

@ivankravets Thanks for the prompt response. The above error is solved but the test cases are not getting printed on the terminal of platformio.

This is the present output:

roy@roy-XPS-15-9510:~/SG_Firmware_Gen_I/master/SG_Firmware_Gen_I/Blink_LED$ pio test Verbosity level can be increased via -v, -vv, or -vvv option Collected 1 tests

Processing test_blink in nucleo_f411re environment

Building & Uploading... xPack OpenOCD x86_64 Open On-Chip Debugger 0.11.0+dev (2021-10-16-21:15) Licensed under GNU GPL v2 For bug reports, read http://openocd.org/doc/doxygen/bugs.html debug_level: 1

srst_only separate srst_nogate srst_open_drain connect_deassert_srst

target halted due to debug-request, current mode: Thread xPSR: 0x01000000 pc: 0x080038a0 msp: 0x20020000 Programming Started Programming Finished Verify Started Verified OK Resetting Target shutdown command invoked Testing... If you don't see any output for the first 10 secs, please reset board (press reset button)

None of test case results are printed.

maxgerhardt commented 2 years ago

Did you press the reset button on the board? What's the exact project?

pratiksurgeonslab98 commented 2 years ago

@maxgerhardt yes, i pressed the reset button but still it didn't show any test result. The test main.cpp file has this code `#include

include

void setUp(void) { // set stuff up here }

void tearDown(void) { // clean stuff up here }

void simple_test(void) { TEST_ASSERT_EQUAL(33, 33); }

int main(int argc, char **argv) { printf("Unity Hello\r\n"); UNITY_BEGIN(); RUN_TEST(simple_test); UNITY_END(); }`

Nothing was printed

maxgerhardt commented 2 years ago

Do regular printf()s work in the firmware code? Without adding the code for creating the serial on the right pins? Are you running on a regular Nucleo F411RE? Can you upload the whole project somewhere for inspection?

pratiksurgeonslab98 commented 2 years ago

@maxgerhardt

Do regular printf()s work in the firmware code? Without adding the code for creating the serial on the right pins? Are you running on a regular Nucleo F411RE? Can you upload the whole project somewhere for inspection?

Yes regular printf() works on fine on firmware. I am running on Nucleo f411re board

After opening the serial terminal on putty to read stlink serial data, the test is running. The results are displayed, it is coming in a jumbled way like this.

To replicate the steps,

Keep the particular port open on putty
and run the pio -test command on platformio terminal

On platform io terminal, this is the output I am getting: `Testing... If you don't see any output for the first 10 secs, please reset board (press reset button)

Please specify test_port for environment or use global --test-port option. --------------------- nucleo_f411re:test_blink [ERRORED] Took 5.66 seconds ---------------------

============================================ SUMMARY ============================================ Environment Test Status Duration

nucleo_f411re test_blink ERRORED 00:00:05.658 =========================== 1 test cases: 0 succeeded in 00:00:05.658 =========================== `

I am also attaching my project file in zipped format for further inspection. Please note that the fourth test case is expected to fail since

void test_function_calculator_division(void) { TEST_ASSERT_EQUAL(31, 32); } addition, multiplication, and subtraction is supposed to PASS.

Blink_LED.zip

Thank You.

dudeperf3ct commented 1 year ago

I followed the exact steps listed in Custom Unity Library — PlatformIO and tests don't pass on Arduino Nano 33 BLE board.

PlatformIO version: 6.1.5a4

Project structure and content is same for all files as the documentation. The following error comes from running pio test

*********************************************************************************************************************************************************************************************************************************************************************************
If you like PlatformIO, please:
- follow us on Twitter to stay up-to-date on the latest project news > https://twitter.com/PlatformIO_Org
- star it on GitHub > https://github.com/platformio/platformio
- try PlatformIO IDE for embedded development > https://platformio.org/platformio-ide
*********************************************************************************************************************************************************************************************************************************************************************************

Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 1 tests
Platform Manager: Installing nordicnrf52
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Platform Manager: nordicnrf52@9.4.0 has been installed!

Processing test_blink in nano33ble environment
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Building & Uploading...
Tool Manager: Installing platformio/toolchain-gccarmnoneeabi @ ~1.80201.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: toolchain-gccarmnoneeabi@1.80201.181220 has been installed!
Tool Manager: Installing platformio/framework-arduino-mbed @ ~3.1.1
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: framework-arduino-mbed@3.1.1 has been installed!
Tool Manager: Installing platformio/tool-sreccat @ ~1.164.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%
Tool Manager: tool-sreccat@1.164.0 has been installed!
Tool Manager: Installing platformio/tool-openocd @ ~2.1100.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-openocd@2.1100.211028 has been installed!
Tool Manager: Installing platformio/tool-bossac-nordicnrf52 @ ~1.10901.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-bossac-nordicnrf52@1.10901.201022 has been installed!
Tool Manager: Installing platformio/tool-scons @ ~4.40400.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-scons@4.40400.0 has been installed!

unity.c:(.text.UnityDefaultTestRun+0x0): multiple definition of `UnityDefaultTestRun'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityDefaultTestRun+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o): in function `UnityBegin':
unity.c:(.text.UnityBegin+0x0): multiple definition of `UnityBegin'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityBegin+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o): in function `UnityEnd':
unity.c:(.text.UnityEnd+0x0): multiple definition of `UnityEnd'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityEnd+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErr64+0x0): multiple definition of `UnityStrErr64'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErr64+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErrDouble+0x0): multiple definition of `UnityStrErrDouble'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErrDouble+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErrFloat+0x0): multiple definition of `UnityStrErrFloat'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErrFloat+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: warning: size of symbol `Unity' changed from 132 in .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o) to 160 in /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o)
collect2: error: ld returned 1 exit status
*** [.pio/build/nano33ble/firmware.elf] Error 1
Uploading stage has failed, see errors above. Use `pio test -vvv` option to enable verbose output.
--------------------------------------------------------------------------------------------------------------- nano33ble:test_blink [ERRORED] Took 1.29 seconds ---------------------------------------------------------------------------------------------------------------

==================================================================================================================================== SUMMARY ====================================================================================================================================
Environment    Test        Status    Duration
-------------  ----------  --------  ------------
nano33ble      test_blink  ERRORED   00:00:01.285
=================================================================================================================== 1 test cases: 0 succeeded in 00:00:01.285 ===================================================================================================================

maxgerhardt commented 1 year ago

Unable to reproduce.

$ pio test --without-uploading --without-testing -vvv
Collected 1 tests (test_blink)

Processing test_blink in nano33ble environment
------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
CONFIGURATION: https://docs.platformio.org/page/boards/nordicnrf52/nano33ble.html
PLATFORM: Nordic nRF52 (9.4.0) > Arduino Nano 33 BLE
HARDWARE: NRF52840 64MHz, 256KB RAM, 960KB Flash
DEBUG: Current (blackmagic) External (blackmagic, cmsis-dap, jlink)
PACKAGES: 
 - framework-arduino-mbed @ 3.1.1 
 - tool-sreccat @ 1.164.0 (1.64) 
 - toolchain-gccarmnoneeabi @ 1.80201.181220 (8.2.1)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 37 compatible libraries
Scanning dependencies...
No dependencies
Building in test mode
MethodWrapper(["checkprogsize"], [".pio/build/nano33ble/firmware.elf"])
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [==        ]  16.2% (used 42576 bytes from 262144 bytes)
Flash: [=         ]   8.2% (used 81012 bytes from 983040 bytes)

Core: 6.1.5a4, Nordic nRF52: 9.4.0.

Grab the project file, unpack it, and run the same above as me in the CLI. Does it still show an error?

unity_tests.zip

dudeperf3ct commented 1 year ago

I was able to run the test using unity_tests.zip.

Is it possible to perform the tests for TFLite library using platformIO? A sample test can be found here.

maxgerhardt commented 1 year ago

I was able to run the test using unity_tests.zip.

Then something is wrong in your project. Adding the TFLite library also shouldn't magically pull Unity in. The error message is showing a conflict from the compiled libraries (.pio/libdeps or lib/ folder) and mbed-os (.pio/build/nano33ble/lib0fc/libUnity.a vs ARDUINO_NANO33BLE/libs/libmbed.a.

Do you have Unity in the lib_deps of the platformio.ini or in the lib/ folder of the project? Have you 100% added the custom test runner script at the correct place with the correct content, and additional placed test_framework = custom in the platformio.ini? Please upload the whole project you're having problems with otherwise.

dudeperf3ct commented 1 year ago

Removing .pio folder seemed to do the trick. Thanks! It works in my project as well. I have the same content as mentioned in the documentation for both platformio.ini and test folder. I use 2 libraries TFLite and TinyMLx for running ML models on Arduino.

Is it possible to perform the unit testing for TFLite library using platformIO? A sample unit test can be found here or here for reference.

I wonder if GoogleTest Runner can be of help in this case.

nicklasb commented 1 year ago

Hm. I am trying running unity as a custom task, however, the tests never "end", in that the results are never returned even through UNITY_END(); has been called. I am following the example here: https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/examples/custom_unity_library.html

I am running the tests on a RasperryPi Pico W. Even if I remove all the tests, and only runs the test framework, nothing is printed to output. It does seem as it sort of ends though, as it becomes possible to update the pico without disconnecting the USB, which is not possible if it has crashed. But there is no print output at all except for the things my code outputs.

Thing is that it works on all other platforms, Native, pure arduino, ESP-IDF, and so on.
(interestingly the pico platform seems more picky when it comes to some pointer constructs, had to change to memcpy all over)

Config: [env:RaspberryPi_Pico] platform = raspberrypi board = pico framework = arduino test_framework= custom

nicklasb commented 1 year ago

Tried with the Earle Philhower core as well, alas, no luck, no unity-output.

maxgerhardt commented 1 year ago

This issue is getting kind of flooded with different problems that may or not be related to the original problem. For the Pico, if you use the default USB serial, you might be running into the problem that PlatformIO choses the wrong serial port because the Pico has not yet initialized, like https://github.com/platformio/platformio-core/issues/3742.

I'll test if I can reproduce this.

maxgerhardt commented 1 year ago

A few observations: Using test code of

#include <Arduino.h>
#include <unity.h>

void setUp(void) { }
void tearDown(void) { }

int addition_function_under_test(int a, int b)
{
    return a + b;
}

void test_calculator_addition(void) {
    TEST_ASSERT_EQUAL(32, addition_function_under_test(25, 7));
}

void setup() {
    // wait until serial port has connected
    while(!Serial) {}
    Serial.println("UNIT TEST START!!");
    UNITY_BEGIN();
    RUN_TEST(test_calculator_addition);
    UNITY_END();
}

void loop() {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(100);
    digitalWrite(LED_BUILTIN, LOW);
    delay(500);
}

and opening a CLI with pio test --without-testing -vvv, the resulting firmware does open a serial port, that when opened in e.g. hterm, and the DTR button is pressed (otherwise no output), the output "UNIT TEST START!!" does appear, but no further. However, the blink sequence in the loop() does trigger, meaning it did run through successfully.

This leads to the conclusion that the Unity version included in ArduinoCore-mbed for the Pico is not outputting its content to the USB serial port at least, maybe the hardware UART, or not all.

Not tested with Arduino-Pico yet, but in general that should be less of a headache since it shouldn't include Unity.

maxgerhardt commented 1 year ago

Solved for ArduinoCore-mbed: Their Unity version is working once you add a magic declaration that stdio output is supposed to go to Serial, the USB-serial.

See https://github.com/arduino/ArduinoCore-mbed/issues/174

So, using

#include <Arduino.h>
#include <unity.h>

REDIRECT_STDOUT_TO(Serial);

at the top of the test file makes it work with ArduinoCore-mbed.

grafik

maxgerhardt commented 1 year ago

And as expected, Arduino-Pico (Earlephilhower) put up no fight at all, working instantly.

I've created a project that is compatible to be tested with both cores, just using a #ifdef ARDUINO_ARCH_MBED check in the ArduinoCore-mbed case.

https://github.com/maxgerhardt/pio-pico-unity-testing

==================== SUMMARY ==================== 
Environment          Test           Status    Duration
-------------------  -------------  --------  ------------
mbed_pico            test_embedded  PASSED    00:00:08.153
earlephilhower_pico  test_embedded  PASSED    00:00:08.138
=== 2 test cases: 2 succeeded in 00:00:16.291 ===

nicklasb commented 1 year ago

WRT:

REDIRECT_STDOUT_TO(Serial); This worked for me, or rather it solve my problem because I use platform = raspberrypi. However, if i changed to:
```
platform = https://github.com/maxgerhardt/platform-raspberrypi.git
board_build.core = earlephilhower
```
...I got a declared out of scope for REDIRECT_STDOUT_TO(Serial).
Uploading not working properly, that is also only a problem I have on earlephilhower.

Thanks!!

adowg67 commented 6 months ago

I sure would like this to work as " EASY as its says it is" am stuck and I am certain it is NOT becuase I do not want to learn this waste of time programming crap . I am not wasting my remaining time or life doing this dum shit. Especially when it is not me . this stupid vs code is what is so dysfunctional and insecure, and got all kinds of glitchs and malfunctions . thats why it never works the same was no matter how many times i open it. its always some different crap . also now why do i have to download this same crap 20 X per day ????? just make it so I can compile my firmware for my 3D printer without all the mental gymnaswtics would you please???

nicklasb commented 6 months ago

I sure would like this to work as " EASY as its says it is" am stuck and I am certain it is NOT becuase I do not want to learn this waste of time programming crap . I am not wasting my remaining time or life doing this dum shit. Especially when it is not me . this stupid vs code is what is so dysfunctional and insecure, and got all kinds of glitchs and malfunctions . thats why it never works the same was no matter how many times i open it. its always some different crap . also now why do i have to download this same crap 20 X per day ????? just make it so I can compile my firmware for my 3D printer without all the mental gymnaswtics would you please???

I am afraid we can't. Must say, this was probably the most impressively out-of-topic outburst I have ever seen, and I've seen a few. :-)

Feeling your pain, though, I would recommend you to share this rant with the manufacturer of your 3D-printer instead of posting it here. It should not be that complicated.

SebastianVivoverse commented 4 months ago

I am still having the exact same issue as the OP for Arduino Nano 33 BLE.

platformio / platformio-core

Unit testing impossible for frameworks including Unity themselves #3980

Configuration

Description of problem

Steps to Reproduce

Actual Results

Expected Results

If problems with PlatformIO Build System:

Additional info

Custom unity framework not working with mbedOS

3342 and following the steps mentioned in this issue.

Processing test_blink in nucleo_f411re environment

include