lowRISC / opentitan

OpenTitan: Open source silicon root of trust
https://www.opentitan.org
Apache License 2.0
2.59k stars 782 forks source link

[bazel] Some unittest are not runnable on host by mistake #24446

Open pamaury opened 3 months ago

pamaury commented 3 months ago

Description

While working #24443, I realize that the unittest //sw/device/silicon_creator/lib/cert:cert_unittest had become unrunnable on host by mistake due to the following dependency:

Dependency chain:
    //sw/device/silicon_creator/lib/cert:cert_unittest (a54725)
    //sw/device/silicon_creator/lib/cert:cert (97da4a)
    //sw/device/silicon_creator/lib:otbn_boot_services (97da4a)   <-- target platform (@local_config_platform//:host) didn't satisfy constraint @platforms//cpu:riscv32

This is now fixed by #24440 but it illustrates that it is very easy to make a unittest unrunnable on host by mistake. Unfortunately, since we run host tests using //..., those incompatible tests will simply be skipped in CI so they are disable without anyone realizing.

Out of curiosity, I looked at the list of test that are currently incompatible with host:

# incompatible.cquery
def format(target):
  if "IncompatiblePlatformProvider" in providers(target):
    return target.label
  return ""

run with

$ bazel cquery 'tests(//sw/device/...)' --output=starlark --starlark:file=incompatible_targets.cquery
@//sw/device/silicon_creator/lib/cert:cert_unittest
@//sw/device/silicon_creator/lib/ownership:ownership_unlock_unittest
@//sw/device/silicon_creator/lib/sigverify:sigverify_unittest
@//sw/device/silicon_creator/rom_ext/keys/fake:fake_unittest
@//sw/device/silicon_creator/rom_ext/prodc/keys:keys_unittest
@//sw/device/silicon_creator/rom_ext/sival/keys:keys_unittest

This means that those tests have been disabled by mistake, I think all of them due to the extra dependency on //sw/device/silicon_creator/lib:otbn_boot_services.

Short-term solution Fix the dependency and make those tests runnable again.

Long-term solution We need to ensure that we don't disable such tests silently in CI. This could either be a lint in the CI, or maybe a way to mark some tests in bazel as "must run on host".

pamaury commented 3 months ago

@moidx @cfrantz @timothytrippel

timothytrippel commented 3 months ago

I think a lint check is probably easiest since its a simple query?

pamaury commented 2 months ago

Done in #24484, could you look at the other unrunnable tests? I think they are all due to otbn boot services