tests: Updated to use latest `lexer` API

[tcoyvwac]

Any change needs to be discussed before proceeding. Failure to do so may result in the rejection of the pull request.

Please provide enough information so that others can review your pull request:

Explain the details for making this change. What existing problem does the pull request solve?

• The lexer datatype API changed, and the tests did not compile.
• The tests now are updated with the new API to compile safely, but the lexer datatype tests (still) fail to pass (as previously).

Code formatting:

git clang-format

Closing issues:

N/A

[tcoyvwac]

Thanks for merging @SaptakBhoumik! Grateful! :smile_cat: