search

Azure-Samples / LUIS-DevOps-Template

Template repo encapsulating best engineering practices and DevOps using GitHub Actions for the development of a LUIS application.
MIT License
21 stars 17 forks source link

Unit test failing #8

Closed daltskin closed 4 years ago

daltskin commented 4 years ago

Under the section Testing the LUIS app seeing the following behaviour:

Test Count: 15, Passed: 14, Failed: 1, Warnings: 0, Inconclusive: 0, Skipped: 0

It is failing on the last unit test within unittests.json:

NLU.DevOps.ModelPerformance.ConfusionMatrixTests.FalseNegativeEntityValue('vacation-request', '{"start-date":[{"type":"date","values":[{"timex":"XXXX-05-05","value":"2019-05-05"},{"timex":"XXXX-05-05","value":"2020-05-05"}]}],"days-duration":[{"days-number":[10]}]}', 'I want to request a vacation on may 5th for 10 days to have time with my family')
Actual utterance does not have entity value matching '{"start-date":[{"type":"date","values":[{"timex":"XXXX-05-05","value":"2019-05-05"},{"timex":"XXXX-05-05","value":"2020-05-05"}]}],"days-duration":[{"days-number":[10]}]}'.

dotnet nlu version: dotnet-nlu 0.7.3PS

AndyCW commented 4 years ago

This is a great catch. This test worked fine up until May 4th. Now it is May 6th 2020, the date recognizer is now returning values in 2020/2021 rather than 2019/2020. Testing at its best. I'll have to update the tests to make them year insensitive.

AndyCW commented 4 years ago

Problem fixed by #19