Different behaviour parsing policy syntax the first time under valgrind/memcheck

RudolfCardinal commented 2 weeks ago

This is slightly odd, and potentially related to https://github.com/ucam-department-of-psychiatry/camcops/issues/363.

When running CamCOPS under Linux via valgrind/memcheck through Qt Creator, the first time you view a list of patients, the "bad policy" symbol appears, with these errors:

...
camcops[1898457]: 2024-06-26T10:36:20.495: info: menulib/menuwindow.cpp(414): Clicked: Menu item: title="Choose patient"
camcops[1898457]: 2024-06-26T10:36:21.202: warning: lib/idpolicy.cpp(233): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"
camcops[1898457]: 2024-06-26T10:36:22.261: info: menulib/menuwindow.cpp(414): Clicked: Menu item: patient=<hidden>
camcops[1898457]: 2024-06-26T10:36:30.855: warning: lib/idpolicy.cpp(233): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"
camcops[1898457]: 2024-06-26T10:36:30.861: warning: lib/idpolicy.cpp(233): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"
camcops[1898457]: 2024-06-26T10:36:30.923: warning: lib/idpolicy.cpp(233): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"
camcops[1898457]: 2024-06-26T10:36:30.931: warning: lib/idpolicy.cpp(233): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"
...

It vanishes again later. The server's policy text is as shown. And I've not been able to reproduce in release or plain debug modes.

martinburchell commented 2 weeks ago

Yes I saw that too when running with Valgrind. I'd dismissed it as user error as it went away as soon as I'd re-fetched from the server.

RudolfCardinal commented 2 weeks ago

It seems fine in the debugger. But under Memcheck, the regular expression is not being parsed consistently.

With a debugging insert:

void IdPolicy::tokenize(QString policy_text)
{
    m_valid = true;

    // Single line, whitespace trimmed, newlines/tabs etc. removed
    policy_text = policy_text.simplified();

    // QString::split() splits using the regex to match BOUNDARIES.
    // Our regex matches CONTENT, so we use QRegularExpression functions.
    // https://dangelog.wordpress.com/2012/04/07/qregularexpression/
    const QRegularExpression re(TOKENIZE_RE_STR);
    QRegularExpressionMatchIterator it = re.globalMatch(policy_text);
    QStringList words;
    while (it.hasNext()) {
        const QRegularExpressionMatch match = it.next();
        const QString word = match.captured(1);
        words << word;
    }
    // Debugging under memcheck:
    qWarning() << "... TOKENIZE_RE_STR = " << TOKENIZE_RE_STR;
    qWarning() << "... policy_text = " << policy_text;
    qWarning() << "... words = " << words;
    for (const QString& word : words) {
        const QString element = word.toLower();
        const int token = nameToToken(element);
        if (token == BAD_TOKEN) {
            reportSyntaxError(QString("unknown word: %1").arg(word));
            invalidate();
            return;
        }
        m_tokens.append(token);
    }
    // check syntax:
    AttributesType blank_attributes;
    for (const QString& name : s_name_to_token.keys()) {
        blank_attributes[name] = false;
    }
    if (idPolicyChunk(m_tokens, blank_attributes) == ChunkValue::SyntaxError) {
        invalidate();
    }
}

One run:

camcops[2314512]: 2024-06-26T15:55:17.828: warning: ../tablet_qt/lib/idpolicy.cpp(172): ... TOKENIZE_RE_STR =  "\\s*(\\w+|\\(|\\))"
camcops[2314512]: 2024-06-26T15:55:17.829: warning: ../tablet_qt/lib/idpolicy.cpp(173): ... policy_text =  "forename AND surname AND sex AND dob AND anyidnum"
camcops[2314512]: 2024-06-26T15:55:17.829: warning: ../tablet_qt/lib/idpolicy.cpp(174): ... words =  QList("AND", "surname", "AND", "sex", "AND", "dob", "AND", "anyidnum")
camcops[2314512]: 2024-06-26T15:55:17.831: warning: ../tablet_qt/lib/idpolicy.cpp(237): Syntax error in policy ("chunk can't start with AND/OR/')'"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"

Another:

camcops[2315554]: 2024-06-26T15:56:26.417: warning: ../tablet_qt/lib/idpolicy.cpp(172): ... TOKENIZE_RE_STR =  "\\s*(\\w+|\\(|\\))"
camcops[2315554]: 2024-06-26T15:56:26.417: warning: ../tablet_qt/lib/idpolicy.cpp(173): ... policy_text =  "forename AND surname AND sex AND dob AND anyidnum"
camcops[2315554]: 2024-06-26T15:56:26.417: warning: ../tablet_qt/lib/idpolicy.cpp(174): ... words =  QList()
camcops[2315554]: 2024-06-26T15:56:26.420: warning: ../tablet_qt/lib/idpolicy.cpp(237): Syntax error in policy ("policy incomplete"); policy text is: "forename AND surname AND sex AND dob AND anyidnum"

??

ucam-department-of-psychiatry / camcops

Different behaviour parsing policy syntax the first time under valgrind/memcheck #364