philomena-dev / philomena

Next-generation imageboard
GNU Affero General Public License v3.0
84 stars 29 forks source link

match_query: unit test and rewrite for TypeScript #208

Closed liamwhite closed 3 months ago

liamwhite commented 3 months ago

This moves all of the match_query clientside parsing library into TypeScript. There should be no substantial logic changes to the internal parse logic here, but three things were discovered during testing and fixed:

  1. Incorrect escaped quote processing in terms. This arose from the translation of the following Ruby code
    def normalize_term!(match, quoted)
     if quoted
       match.gsub!('\"', '"')
     else
       match.gsub!(/\\(.)/, '\1')
     end
    end

    as

    SearchTerm.prototype._normalizeTerm = function() {
     if (!this.wildcardable) {
       return this.term.replace('\"', '"');
     }
     return this.term.replace(/\\([^\*\?])/g, '$1');
    };

    which is a very subtle mistake, as the string literal escaping rules are different in JS.

  2. Inconsistency in relative date parsing. For absolute dates, the lte and gt operators use the "upper" date in the date range, and the lt and gte operators use the "lower" date. For relative dates, only the "lower" date was used. This is inconsistent with the behavior of the server-side parser, which uses the same behavior for both date types, so it was changed to match.
  3. Use of the deprecated/optional substr JS API. These correspond to the slice method used in the Ruby parser.
    # Truncate string and restart the token tests.
    @search_str = @search_str.slice(match.size,
                                   @search_str.size - match.size)

    Just a minor fix to change these to substring.

This has 100% test coverage in unit tests and has also been manually integration tested.