Need suggestion to execute complex search

[JohnLopen]

I need suggestion or similar implementation to search for multiple strings. It works for a few strings but when it's close to a hundred it throws an error:

SearchError: The query to RediSearch had a syntax error: "Syntax error at offset 3624 near 2M80400206C1026P". 3|ec-digikey-api | This is often the result of using a stop word in the query. Either change the query to not use a stop word or change the stop words in the schema definition. You can check the RediSearch source for the default stop words at: https://github.com/RediSearch/RediSearch/blob/master/src/stopwords.h

Function to find matches in repository:

`async findMatches(partNumbers: string[]) { let query = this.repository?.search()

    // Create the query with multiple OR conditions
    partNumbers.forEach((partNumber, index) => {
        partNumber = partNumber.toUpperCase();
        // For the first part number, start the query without OR
        if (index === 0) {
            query = query?.where('normalized_pn').matches(`*${partNumber}*`)
                .or('part_number').matches(partNumber);
        } else {
            // Chain OR conditions for subsequent part numbers
            query = query?.or('normalized_pn').matches(`*${partNumber}*`)
                .or('part_number').matches(partNumber);
        }
    });

    // Fetch all results matching the query
    const results = await query?.return.all();
    return results;
}`

Sample array of string: [ '2M80400507ZNU67S', '2M80400106NF53P', '2M80400507MT813SA', '2M80400407NF64S', '2M80400307M10200P', '2M80400106NF145PA', '2M80400206MT1026PA', '2M80400307NF67SA', '2M80400106M1026S', '2M80400106M919PA', '2M80400507M1026FA', '2M80400307MT1026SA', '2M80400106ZNU710P', '2M80400106M1026PA', '2M80400106C813PA', '2M80400106ZNU64PA', '2M80400301ZNU66SA', '2M80400300M1026SA', '2M80400301M10200S', '2M80400206NF1026S', '2M80400106M919SA', '2M80400301MT1026P', '2M80400206ZNU9200S', '2M80400106NF1026SA', '2M80400106NF67PA', '2M80400401MT66S', '2M80400407NF53PA', '2M80400106M1237PA', '2M80400307ZNU66S', '2M80400401MT1026PA', '2M80400507ZNU813PA', '2M80400301NF710PA', '2M80400301ZNU710P', '2M80400407ZNU813S', '2M80400400NF813S', '2M80400206C1026P', '2M80400307MT1237PA', '2M80400507M1237E', '2M80400106ZNU813P', '2M80400206ZNU145P', '2M80400301M64P', '2M80400106NF94SA', '2M80400106NF64P', '2M80400106NF813PA', '2M80400507MT813PA', '2M80400300M94S', '2M80400301M10200P', '2M80400106M710SB', '2M80400507ZNU623FA', '2M80400307NF64P', '2M80400106ZNU919S', '2M80400106ZNU53SA', '2M80400507ZNU94S', '2M80400307NF64S', '2M80400106ZNU1026PA', '2M80400106ZNU1026SA', '2M80400206M710P', '2M80400106C1026P', '2M80400300C813P', '2M80400106M64S', '2M80400507NF813P', '2M80400206MT1026S', '2M80400206ZNU64S', '2M80400106M10200P', '2M80400307NF12201P', '2M80400206MT66P', '2M80400106ZNU828P', '2M80400206ZNU1026SA', '2M80400307M813PA', '2M80400106NF145P', '2M80400300M94P', '2M80400307ZNU61A', '2M80400106NF64S', '2M80400106NF67P', '2M80400507M919P', '2M80400106NF67S', '2M80400106NF1026P', '2M80400106ZNU53P', '2M80400106Z166PA', '2M80400106M1412SA', '2M80400307NF710S', '2M80400301MT813AA', '2M80400106M710P', '2M80400106NF12201S', '2M80400106NF710P', '2M80400206NF813P', '2M80400206NF1026PA', '2M80400106M919P', '2M80400106NF813S', '2M80400106MT1237SA', '2M80400106ZNU145P', '2M80400301ZNU813SA', '2M80400407M710SA', '2M80400300ZNU1026SA', '2M80400507ZNU710SA', '2M80400206ZNU14235PA', '2M80400106M1237S', '2M80400106M1026P', '2M80400106M710PA', '2M80400106MT1237AA', ... 85 more items ]

[c0stal]

I am also facing the same problem, I have a query running on TAG field using OR clause. It works fine for less than 70-80 elements but breaks with this error message: The query to RediSearch had a syntax error: \\\"Syntax error at offset 6116 near .... whenever it goes beyond 100. Is there any limit on number of characters?

[guyroyse]

There is an upper limit to what RediSearch can handle in a query before blowing out the stack. The way I implemented my query build is a bit naive and—while perfectly adequate for the majority of users—probably exacerbates this error.

The change needed within Redis OM is to build up an AST from all the calls in the query so that I can generate a query with fewer parentheses. This is not something that will happen quickly unless a friendly and helpful PR comes along.

As a work around, you could generate the query yourself and execute it using the .searchRaw function like this:

const query = partNumbers.map(pn => `@normalized_pn:{${pn}}`).join('|')
const entities = repository.searchRaw(query).return.all();

[guyroyse]

Actually, an even cleaner way would be:

const query = `@normalized_pn:{${partNumbers.join('|')}}`;
const entities = repository.searchRaw(query).return.all();