search

numaru / vscode-ceedling-test-adapter

Ceedling Test Adapter for the VS Code Test Explorer
MIT License
37 stars 13 forks source link

Test Explorer showing "red bang" icon instead of test status - Cannot use ruby code in project.yml #108

Open Shayamir opened 2 years ago

Shayamir commented 2 years ago

I've configured Ceedling to run using the plugin. I've also added xml_tests_report to my project.yml. For some reason, after running tests, I get a "red bang" icon, instead of a test status.

RedBang

My Ceedling version: $ ceedling version Ceedling:: 0.31.1 Unity:: 2.5.4 CMock:: 2.5.4 CException:: 1.3.3 report.xml.txt

Any ideas how I can debug this further?

numaru commented 2 years ago

You should be able to see the output of the ceedling command in the OUTPUT panel by clicking on the test. Can you share this message?

The report.xml seems legit to me, no error in it, only one failure.

Shayamir commented 2 years ago

It looks like this: stdout:

Test 'test_bar.c'

Running test_bar.out...

TEST OUTPUT

[test_bar.c]

FAILED TEST SUMMARY

[test_bar.c] Test: test_cs_access_write_lan_bar0_msb At line (176): "Expected 1 Was 0"

OVERALL TEST SUMMARY

TESTED: 5 PASSED: 4 FAILED: 1 IGNORED: 0

Shayamir commented 2 years ago

BTW, I've made 1 test fail, to check the different status icons. BTW2, I was sure that the results are being parsed back from XML report and that the OUPUT is not so relevant... Probably I was mistaken.

Shayamir commented 2 years ago

Any idea how I can debug? Can you help?

Shayamir commented 2 years ago

I've debugged the issue. My project.yml (relevant sections) looks like this:

:project:
  :build_root: "#{ENV['BUILD_ROOT']}"
  :test_file_prefix: test_
  :default_tasks:
    - test:all

:environment:
  - REPO_ROOT: "#{ENV.fetch('REPO_ROOT', ENV['PWD'])}"
  - BUILD_ROOT: "#{ENV.fetch('BUILD_ROOT', ENV['PWD'] + '/build/unittest_ceedling/debug')}"

:paths:
  :test:
    - "#{ENV['REPO_ROOT']}/controller/**/test"
  :source:
    - "#{ENV['REPO_ROOT']}/controller/**"

When the extension runs, in file: adapter.ts, function: getXmlReportPath, I see this path: 'C:\prj\mmg\#{ENV['BUILD_ROOT']}\artifacts\test\report.xml'

It seems like Environment variable is not being expanded.

any idea how to proceed?

Shayamir commented 2 years ago

Narrowed it down to this line: const ymlProjectBuildDirectory = ymlProjectData[':project'][':build_root']; returns: ymlProjectBuildDirectory = '#{ENV['BUILD_ROOT']}'

numaru commented 2 years ago

Sorry for the late reply and thank you for taking the time to debug the issue. Do you use ruby code elsewhere in the project.yml? Do you really need those environment variables?

How I would implement it:

1st option

2nd option

Shayamir commented 2 years ago

3rd Option I'd write a routine in typescript files to read from .yml and if It finds '#{ENV[' string, it'll replace it with the corresponding environment variable.

4th Option Don't fix this, and document which variables on project.yml cannot use environment variables.

Since Ceedling .yml files support environment variables, other people might encounter this problem.

Right now, for me, I've decided to not use the environment variable in :project:build_root: And the extension is working fine. BTW, thank you very much for a great extension.

numaru commented 2 years ago

Great! I will let it as it is for now and keep the issue open.

breakersun commented 1 year ago

Got the same problem here: ceedling versions(on Windows 10) ❯ ceedling version
Ceedling:: 0.32.0 Unity:: 2.5.4 CMock:: 2.5.4 CException:: 1.3.3

Project

---
:project:
  # how to use ceedling. If you're not sure, leave this as `gem` and `?`
  :which_ceedling: gem
  :ceedling_version: 0.32.0

  # optional features. If you don't need them, keep them turned off for performance
  :use_mocks: TRUE
  :use_test_preprocessor: TRUE
  :use_preprocessor_directives: FALSE
  :use_deep_dependencies: FALSE
  :use_auxiliary_dependencies: TRUE
  :use_backtrace_gdb_reporter: FALSE

  # tweak the way ceedling handles automatic tasks
  :build_root: build
  :test_file_prefix: test_
  :default_tasks:
    - test:all

  # performance options. If your tools start giving mysterious errors, consider 
  # dropping this to 1 to force single-tasking
  :test_threads: 8
  :compile_threads: 8

  # you can specify different yaml config files which modify the existing one
  :options_paths: []

  # enable release build (more details in release_build section below)
  :release_build: FALSE

# specify additional yaml files to automatically load. This is helpful if you
# want to create project files which specify your tools, and then include those
# shared tool files into each project-specific project.yml file.
:import: []

# further details to configure the way Ceedling handles test code
:test_build:
  :use_assembly: FALSE
  :toolchain_include: []

# further details to configure the way Ceedling handles release code
:release_build:
  :output: MyApp.out
  :use_assembly: FALSE
  :artifacts: []
  :toolchain_include: []

# Plugins are optional Ceedling features which can be enabled. Ceedling supports
# a variety of plugins which may effect the way things are compiled, reported, 
# or may provide new command options. Refer to the readme in each plugin for 
# details on how to use it.
:plugins:
  :load_paths: []
  :enabled:
    - module_generator               # handy for quickly creating source, header, and test templates
    - stdout_pretty_tests_report
    - xml_tests_report

# override the default extensions for your system and toolchain
:extension:
  :executable: .out

# This is where Ceedling should look for your source and test files.
# see documentation for the many options for specifying this.
:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support
  :libraries: []

# Configuration Options specific to CMock. See CMock docs for details
:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
    - :ignore_arg
    - :expect_any_args
    - :array
    - :callback
    - :return_thru_ptr
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

# Configuration options specific to Unity. 
:unity:
  :defines:
    - UNITY_EXCLUDE_FLOAT

# You can optionally have ceedling create environment variables for you before
# performing the rest of its tasks.
:environment: []

with all tests passed, still get a red bang symbol: image