CI pipelines and automated tests for Core

[pranav12321]

what

Support for test infrastructure. CI pipelines for ISA tests, mock dummy peripheral traits device support, RPC and application support. potentially testbenches for real hardware tests (there are some approaches to this. To be fleshed out).

steps

• [ ] [[ steps ]]

[[ What steps will we need to take to implement this feature? ]]

where

branch: N/A, this is a tracking issue

open questions

[rrbutani]

@pranav12321 I think the CI part of this issue is covered by #174. Testing and test coverage is definitely something worth improving, perhaps we can have this issue focus on that?

Some thoughts:

• ISA tests

  • I think we actually do have satisfactory infrastructure here (infra for single instruction tests, bare interpreter tests); coverage could use some work though

• mock dummy peripheral traits device support

  • yup, I think having simple mock tests (like those that the ADC impl has) is valuable just to guard against regressions and give us some confidence that the test suite will catch accidental interface breaking changes
  • in general I think the value of using mocks is a bit questionable but in this specific case it seems like time well spent since we don't actually have "client code" for these peripheral trait implementations in core; all of that lives in device crates like utp-tm4c

  • RPC

  • definitely; I think this is covered by #188 though

  • and application support

  • this one is a little trickier; maybe you can expand on what you have in mind @pranav12321?
  • unit tests on the lc3-application-support crate (wherever possible) are one thing but end-to-end testing (in the style of selenium, etc) is a little more controversial, imo
  • for one thing afaik there aren't really any terminal based UI testing tools and I'm skeptical that the upside to be had from having such tests (i.e. tests that do things like making sure pressing tab changes the TUI tab that's focused) justifies the cost
  • it seems like we can get most of the benefit from having good unit test coverage on widgets and writing automated testing for the low hanging fruit in the end-to-end space (i.e. resize the window randomly and ensure that the TUI doesn't crash, feed the TUI random input and make sure it doesn't crash, etc.)
  • in any case, I think the latter (e2e testing) is an issue that should live on the TUI repo

  • potentially testbenches for real hardware tests

  • this is definitely an issue for the TM4C repo and ut-utp/tm4c#1 aims to have the first-order version of this (smoke tests in CI)
  • with the xtask stuff in the tm4c repo we now do have all the infrastructure we need for this
  • the open question in my mind is whether we want to gate CI runs on this repo on these device CI runs but we can figure that out later