OCPP test naming convention

[drmrd]

Proposal

When adding a test to libocpp that partially (or fully) validates libocpp's conformance to a functional requirement in an OCPP standard, prefix the name of the test with said functional requirement and an underscore.

Example with a Functional Requirement

Consider this unit test from a recent PR tests that a charging profile with purpose TxProfile and a non-positive EVSE ID is considered invalid by the OCPP 2.0.1 smart charging handler. This test is intended to verify that libocpp (partially) satisfies functional requirement K01.FR.16 from OCPP 2.0.1, which states the following:

TxProfile SHALL only be be [sic] used with evseId >0.

Without this proposal, a good, descriptive name for this test might be

IfTxProfileHasEvseIdNotGreaterThanZero_ThenProfileIsInvalid

I am proposing here that we use the name

K01FR16_IfTxProfileHasEvseIdNotGreaterThanZero_ThenProfileIsInvalid

instead.

Example with a Table Row ID

There may at times be a need to test behaviors mandated in portions of the OCPP 2.0.1 standard other than functional requirements. For instance, each use case in OCPP 2.0.1 begins with a summary table. If one of these tables contains behavioral information that isn't captured clearly in a functional requirement for the table's use case (e.g., about error handling), a test of that behavior should be prefixed with Table<table number>Row<row number>_. For example, when testing a behavior described in row 8 ("Remark(s)") of Table 173, prefix the name of said test with Table173Row8_.

Rationale

As we work to build out the test pyramid for libocpp, many of the unit, integration, and end-to-end tests being created will validate behaviors mandated by one of the OCPP standards. Prefixing the subset of these tests that are validating behaviors from an OCPP standard has a number of benefits, including the following:

• Easy OCPP Test Identification: This convention makes it much easier to identify when a change has broken our OCPP implementation (vs. broken adherence to an internal convention, white paper, or other standard that is of lower priority for an OCPP-centered library).

• Easier Gap Analysis: With this convention, viewing test reports makes it much easier for a potential adopter or contributor who is familiar with OCPP to identify what functionality does/doesn't exist in libocpp today.

• Distinguishing Behavioral Invariants: Sometimes tests validate behaviors that we think are good ideas at the time but that change with our understanding of our users' needs. Tests of the functional requirements in OCPP standards are not of this type and should almost never need to be modified except to accommodate refactors and new OCPP errata. These functional requirement prefixes also serve as flags for tests that shouldn't be modified without a fair bit of thought.

• Enable Future Testing Capabilities: Adding these labels also makes it easy to identify tests that could be extracted into, say, a charging station test harness in the future.

[Pietfried]

Hi @drmrd , thanks for this proposal. It sounds like a very good approach to me! Once we have verified that this naming pattern works fine for us moving on with writing the test cases, I would suggest to put the content of this issue into some form of documentation in the tests directory of this repository.

[drmrd]

Thanks for the suggestion @Pietfried! We can add some info about the naming convention in a README file for the tests directory in a future PR.

[shankari]

@drmrd just to clarify, this is the naming convention for 2.0.1 tests. Is there a similar proposal for naming 1.6J tests, or is the expectation that since the 1.6J code will eventually be superseded by 2.0.1, that we do not need such a convention for the 1.6J tests?

[shankari]

@drmrd can you please respond to this question about OCPP 1.6J test case naming conventions? I can then get caught up on my approval backlog! @couryrr-afs for visibility

[drmrd]

@shankari: Apologies for the delayed reply. You are correct that this is only a proposal for 2.0.1 test names. We have not attempted to devise a reasonable naming schema for 1.6J, primarily due to my team's focus being exclusively on the new standard. The convention we've adopted here shouldn't prevent a separate naming scheme being proposed for 1.6J tests in the future, if someone is inclined to create one.