Long expression generates an error "too many inputs"

[yonghah]

Problem description

When I send a reqeust with a complex expression (and hence long such as total url's length = 1851 characters), titiler returns "too many inputs" error. When I change the request with the same url and only shrinked expression, titiler returns the result without any issue.

Is there any setting for maximum query length? If so, how can I relax the condition?

Expected Output

Return image response

Environment Information

Titiler 0.11.6 / OAS3 Deployed to AWS with Lambda on Python 3.10

The request with a problem: https://ou60yuvp0i.execute-api.us-east-1.amazonaws.com/cog/tiles/14/8193/8190.png?url=https://stereo-e16.s3.amazonaws.com/raster/factor/stereo_cog_r1.tif&expression=0.57103%2Ab1%2B0.99672%2Ab2%2B0.96904%2Ab3%2B0.20021%2Ab4%2B0.27429%2Ab5%2B0.79971%2Ab6%2B0.72686%2Ab7%2B0.94084%2Ab8%2B0.18491%2Ab9%2B0.82333%2Ab10%2B0.2628%2Ab11%2B0.93301%2Ab12%2B0.38127%2Ab13%2B0.24539%2Ab14%2B0.10738%2Ab15%2B0.67462%2Ab16%2B0.27597%2Ab17%2B0.27698%2Ab18%2B0.50362%2Ab19%2B0.8538%2Ab20%2B0.27273%2Ab21%2B0.74617%2Ab22%2B0.4796%2Ab23%2B0.22111%2Ab24%2B0.65345%2Ab25%2B0.11374%2Ab26%2B0.09267%2Ab27%2B0.98799%2Ab28%2B0.24427%2Ab29%2B0.18995%2Ab30%2B0.15173%2Ab31%2B0.99163%2Ab32%2B0.59891%2Ab33%2B0.88331%2Ab34%2B0.97323%2Ab35%2B0.89888%2Ab36;0.04474%2Ab1%2B0.60977%2Ab2%2B0.43399%2Ab3%2B0.67842%2Ab4%2B0.39043%2Ab5%2B0.17055%2Ab6%2B0.11867%2Ab7%2B0.35566%2Ab8%2B0.94484%2Ab9%2B0.91253%2Ab10%2B0.30639%2Ab11%2B0.81236%2Ab12%2B0.98909%2Ab13%2B0.23044%2Ab14%2B0.81381%2Ab15%2B0.98246%2Ab16%2B0.97092%2Ab17%2B0.46153%2Ab18%2B0.99879%2Ab19%2B0.2217%2Ab20%2B0.5304%2Ab21%2B0.95593%2Ab22%2B0.01583%2Ab23%2B0.15223%2Ab24%2B0.07902%2Ab25%2B0.90673%2Ab26%2B0.86554%2Ab27%2B0.51667%2Ab28%2B0.60937%2Ab29%2B0.07176%2Ab30%2B0.74472%2Ab31%2B0.68408%2Ab32%2B0.99638%2Ab33%2B0.86553%2Ab34%2B0.74682%2Ab35%2B0.27824%2Ab36;0.00529%2Ab1%2B0.17842%2Ab2%2B0.10294%2Ab3%2B0.96833%2Ab4%2B0.85393%2Ab5%2B0.0152%2Ab6%2B0.00695%2Ab7%2B0.07031%2Ab8%2B0.60713%2Ab9%2B0.20663%2Ab10%2B0.72968%2Ab11%2B0.22667%2Ab12%2B0.42386%2Ab13%2B0.59142%2Ab14%2B0.83484%2Ab15%2B0.2196%2Ab16%2B0.51653%2Ab17%2B0.93309%2Ab18%2B0.32701%2Ab19%2B0.02677%2Ab20%2B0.98461%2Ab21%2B0.20406%2Ab22%2B0.01055%2Ab23%2B0.42558%2Ab24%2B0.00408%2Ab25%2B0.69651%2Ab26%2B0.7623%2Ab27%2B0.13883%2Ab28%2B0.99697%2Ab29%2B0.23217%2Ab30%2B0.91416%2Ab31%2B0.20706%2Ab32%2B0.26038%2Ab33%2B0.21719%2Ab34%2B0.22536%2Ab35%2B0.04243%2Ab36

Request without any problem: https://ou60yuvp0i.execute-api.us-east-1.amazonaws.com/cog/tiles/14/8193/8190.png?url=https://stereo-e16.s3.amazonaws.com/raster/factor/stereo_cog_r1.tif&expression=0.57103%2Ab1%2B0.99672%2Ab2%2B0.96904%2Ab3%2B0.20021%2Ab4%2B0.27429%2Ab5%2B0.79971%2Ab6%2B0.72686%2Ab7%2B0.94084%2Ab8%2B0.18491%2Ab9%2B0.82333%2Ab10%2B0.2628%2Ab11%2B0.93301%2Ab12%2B0.38127%2Ab13%2B0.24539%2Ab14%2B0.10738%2Ab15;0.04474%2Ab1%2B0.60977%2Ab2%2B0.43399%2Ab3%2B0.67842%2Ab4%2B0.39043%2Ab5%2B0.17055%2Ab6%2B0.11867%2Ab7%2B0.35566%2Ab8%2B0.94484%2Ab9%2B0.91253%2Ab10%2B0.30639%2Ab11%2B0.81236%2Ab12%2B0.98909%2Ab13%2B0.23044%2Ab14%2B0.81381%2Ab15;0.00529%2Ab1%2B0.17842%2Ab2%2B0.10294%2Ab3%2B0.96833%2Ab4%2B0.85393%2Ab5%2B0.0152%2Ab6%2B0.00695%2Ab7%2B0.07031%2Ab8%2B0.60713%2Ab9%2B0.20663%2Ab10%2B0.72968%2Ab11%2B0.22667%2Ab12%2B0.42386%2Ab13%2B0.59142%2Ab14%2B0.83484%2Ab15

[vincentsarago]

that's really interesting.

you expressions are

['0.57103*b1+0.99672*b2+0.96904*b3+0.20021*b4+0.27429*b5+0.79971*b6+0.72686*b7+0.94084*b8+0.18491*b9+0.82333*b10+0.2628*b11+0.93301*b12+0.38127*b13+0.24539*b14+0.10738*b15+0.67462*b16+0.27597*b17+0.27698*b18+0.50362*b19+0.8538*b20+0.27273*b21+0.74617*b22+0.4796*b23+0.22111*b24+0.65345*b25+0.11374*b26+0.09267*b27+0.98799*b28+0.24427*b29+0.18995*b30+0.15173*b31+0.99163*b32+0.59891*b33+0.88331*b34+0.97323*b35+0.89888*b36', '0.04474*b1+0.60977*b2+0.43399*b3+0.67842*b4+0.39043*b5+0.17055*b6+0.11867*b7+0.35566*b8+0.94484*b9+0.91253*b10+0.30639*b11+0.81236*b12+0.98909*b13+0.23044*b14+0.81381*b15+0.98246*b16+0.97092*b17+0.46153*b18+0.99879*b19+0.2217*b20+0.5304*b21+0.95593*b22+0.01583*b23+0.15223*b24+0.07902*b25+0.90673*b26+0.86554*b27+0.51667*b28+0.60937*b29+0.07176*b30+0.74472*b31+0.68408*b32+0.99638*b33+0.86553*b34+0.74682*b35+0.27824*b36', '0.00529*b1+0.17842*b2+0.10294*b3+0.96833*b4+0.85393*b5+0.0152*b6+0.00695*b7+0.07031*b8+0.60713*b9+0.20663*b10+0.72968*b11+0.22667*b12+0.42386*b13+0.59142*b14+0.83484*b15+0.2196*b16+0.51653*b17+0.93309*b18+0.32701*b19+0.02677*b20+0.98461*b21+0.20406*b22+0.01055*b23+0.42558*b24+0.00408*b25+0.69651*b26+0.7623*b27+0.13883*b28+0.99697*b29+0.23217*b30+0.91416*b31+0.20706*b32+0.26038*b33+0.21719*b34+0.22536*b35+0.04243*b36']

which needs all those bands (36)

['b35', 'b26', 'b27', 'b17', 'b10', 'b31', 'b25', 'b4', 'b34', 'b29', 'b36', 'b24', 'b6', 'b14', 'b22', 'b33', 'b3', 'b9', 'b30', 'b5', 'b1', 'b13', 'b19', 'b12', 'b11', 'b28', 'b23', 'b2', 'b20', 'b7', 'b8', 'b15', 'b18', 'b16', 'b21', 'b32']

And I think the error comes from https://github.com/pydata/numexpr/blob/master/numexpr/interpreter.cpp#L1024-L1027 and then is related to https://github.com/numpy/numpy/issues/4398

As soon as you have less than 32 bands in your expression it should work 😓

This is really unfortunate and not a TiTiler issue

We could maybe optimized rio-tiler to only pass the needed bands per bloc in https://github.com/cogeotiff/rio-tiler/blob/main/rio_tiler/expression.py#L74 (replacing local_dict=dict(zip(bands, data))) but it's not a quick fix and will not resolve the real issue from Numpy/Numexpr.

While I admire that you are using expression for this (I never thought to see this kind of large expression 😅) I think using the custom algorithm might be better suited

Note: I'm going to move this to discussion because it's not a BUG in titiler