search

telefonicaid / fiware-orion

Context Broker and CEF building block for context data management, providing NGSI interfaces.
https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/orion-api.md
GNU Affero General Public License v3.0
212 stars 265 forks source link

Separate "blocks" for .test for regular functional test and valgrind test. #1222

Open fgalan opened 9 years ago

fgalan commented 9 years ago

Raised in https://github.com/telefonicaid/fiware-orion/pull/1184/files#r38409031

Sometimes (specially in these .test where timming is important) the REGEXPECT part in a .test is different for functional test and execution under valgrind.

Currently, we are solving this using "wide" REGEX(), such as:

TIMES: REGEX((0|1), (1|2|3), (3|4), (5|6), (9|10), (13|14), (14|15|16), (24|25|26), (34|35|36), (35|36|37|38), (37|38|39|40), (39|40|41|42), (51|52|53|54), (63|65|66), (67|72), (69|74))

If would be great to develop "marks" in the .test so we can separate the part that is to be used under regular functionla test and valgrind test. Something like this:

#BEGIN_VALGRIND
TIMES: REGEX((0|1), (1|2|3), (3|4), (5|6), (9|10), (13|14), (14|15|16), (24|25|26), (34|35|36), (35|36|37|38), (37|38|39|40), (39|40|41|42), (51|52|53|54), (63|65|66), (67|72), (69|74))
#BEGIN_NOVALGRIND
TIMES: REGEX((0, 2, 4, 5, 9, 13, 14, 24, 34, 35, 37, 39, 53, 65, 67, 69|0, 2, 4, 5, 9, 13, 14, 24, 34, 35, 37, 39, 51, 63, 67, 69))
#END

Effort: 7 man day

kzangeli commented 9 years ago

A simpler way of doing this would be to do a CPP (C PreProcessor) pass of the --REGEXPECT-- part before comparing it with the real output. We already send the --REGEXPECT-- part of the test file to a separate file. We just need to add "cpp xxx.regexpect -o xxx.regexpect.preprosessed"

In the case of a functest running under valgrind, cpp would be invoked with -DVALGRIND and thus, the following construction can be used inside the ..REGEXPECT-- part:

#ifdef VALGRIND
TIMES: REGEX((0|1), (1|2|3), (3|4), (5|6), (9|10), (13|14), (14|15|16), (24|25|26), (34|35|36), (35|36|37|38), (37|38|39|40), (39|40|41|42), (51|52|53|54), (63|65|66), (67|72), (69|74))
#else 
TIMES: REGEX((0, 2, 4, 5, 9, 13, 14, 24, 34, 35, 37, 39, 53, 65, 67, 69|0, 2, 4, 5, 9, 13, 14, 24, 34, 35, 37, 39, 51, 63, 67, 69))
#endif

Doing some simple tests I have seen that the preprocessor adds some informational lines to its output - these lines we need to suppress somehow.

Apart from this little problem, the modification to the func test suite should be pretty simple.