upstash / ratelimit-js

Rate limiting library for serverless runtimes
https://ratelimit-with-upstash.vercel.app/
MIT License
1.72k stars 33 forks source link

Command 1 [ evalsha ] failed: NOSCRIPT No matching script. Please use EVAL. #122

Closed sgrodzicki closed 2 weeks ago

sgrodzicki commented 1 month ago

While running the following script (backed by Upstash Redis) I'm getting a NOSCRIPT error on a flushed Redis instance (SCRIPT FLUSH):

const redis = Redis.fromEnv();

redis.scriptFlush(); // only for debugging

const ratelimit = new Ratelimit({
  redis,
  prefix: 'auth',
  limiter: Ratelimit.slidingWindow(5, '15 m'),
  analytics: true,
  enableProtection: true,
  ephemeralCache: false,
});

try {
    const { success, pending, reason, deniedValue } = await ratelimit.limit(user.email, {
        ip,
        userAgent,
        country,
    });

    await pending;

    if (!success) {
        log({
            message: 'Rate Limit Exceeded',
            level: 'warn',
            meta: {
                ip,
                userAgent,
                country,
                reason,
                deniedValue,
            },
        });
    }
} catch (error) {
    console.log(error); // Command 1 [ evalsha ] failed: NOSCRIPT No matching script. Please use EVAL.
}

I'm aware of the fallback mechanism, so it might be the second evalsha call causing it:

https://github.com/upstash/ratelimit-js/blob/19011929b8b80e92dae4c227300d1a54e8a4b09c/src/hash.ts#L21-L38

Redis log (MONITOR) looks like this:

1728062526.773431 [0 140.82.121.4:58234] "EVALSHA" "e1391e429b699c780eb0480350cd5b7280fd9213" "2" "auth:sebastian@example.com:1920069" "auth:sebastian@example.com:1920068" "5" "1728062526414" "900000" "1"
1728062526.773911 [0 140.82.121.4:58234] "EVAL" "\n  -- Checks if values provideed in ARGV are present in the deny lists.\n  -- This is done using the allDenyListsKey below.\n\n  -- Additionally, checks the status of the ip deny list using the\n  -- ipDenyListStatusKey below. Here are the possible states of the\n  -- ipDenyListStatusKey key:\n  -- * status == -1: set to \"disabled\" with no TTL\n  -- * status == -2: not set, meaning that is was set before but expired\n  -- * status  >  0: set to \"valid\", with a TTL\n  --\n  -- In the case of status == -2, we set the status to \"pending\" with\n  -- 30 second ttl. During this time, the process which got status == -2\n  -- will update the ip deny list.\n\n  local allDenyListsKey     = KEYS[1]\n  local ipDenyListStatusKey = KEYS[2]\n\n  local results = redis.call('SMISMEMBER', allDenyListsKey, unpack(ARGV))\n  local status  = redis.call('TTL', ipDenyListStatusKey)\n  if status == -2 then\n    redis.call('SETEX', ipDenyListStatusKey, 30, \"pending\")\n  end\n\n  return { results, status }\n" "2" "auth:denyList:all" "auth:ipDenyListStatus" "sebastian@example.com" "127.0.0.1"
1728062526.774460 [0 lua] "SMISMEMBER" "auth:denyList:all" "sebastian@example.com" "127.0.0.1"
1728062526.774815 [0 lua] "TTL" "auth:ipDenyListStatus"
1728062526.775166 [0 lua] "SETEX" "auth:ipDenyListStatus" "30" "pending"
1728062526.876307 [0 140.82.121.4:58234] "SCRIPT" "load" "\n  local currentKey  = KEYS[1]           -- identifier including prefixes\n  local previousKey = KEYS[2]           -- key of the previous bucket\n  local tokens      = tonumber(ARGV[1]) -- tokens per window\n  local now         = ARGV[2]           -- current timestamp in milliseconds\n  local window      = ARGV[3]           -- interval in milliseconds\n  local incrementBy = ARGV[4]           -- increment rate per request at a given value, default is 1\n\n  local requestsInCurrentWindow = redis.call(\"GET\", currentKey)\n  if requestsInCurrentWindow == false then\n    requestsInCurrentWindow = 0\n  end\n\n  local requestsInPreviousWindow = redis.call(\"GET\", previousKey)\n  if requestsInPreviousWindow == false then\n    requestsInPreviousWindow = 0\n  end\n  local percentageInCurrent = ( now % window ) / window\n  -- weighted requests to consider from the previous window\n  requestsInPreviousWindow = math.floor(( 1 - percentageInCurrent ) * requestsInPreviousWindow)\n  if requestsInPreviousWindow + requestsInCurrentWindow >= tokens then\n    return -1\n  end\n\n  local newValue = redis.call(\"INCRBY\", currentKey, incrementBy)\n  if newValue == tonumber(incrementBy) then\n    -- The first time this key is set, the value will be equal to incrementBy.\n    -- So we only need the expire command once\n    redis.call(\"PEXPIRE\", currentKey, window * 2 + 1000) -- Enough time to overlap with a new window + 1 second\n  end\n  return tokens - ( newValue + requestsInPreviousWindow )\n"
1728062526.908374 [0 140.82.121.4:58234] "EVALSHA" "e1391e429b699c780eb0480350cd5b7280fd9213" "2" "auth:sebastian@example.com:1920069" "auth:sebastian@example.com:1920068" "5" "1728062526414" "900000" "1"
1728062526.908798 [0 lua] "GET" "auth:sebastian@example.com:1920069"
1728062526.909129 [0 lua] "GET" "auth:sebastian@example.com:1920068"
1728062526.909450 [0 lua] "INCRBY" "auth:sebastian@example.com:1920069" "1"
1728062526.909835 [0 lua] "PEXPIRE" "auth:sebastian@example.com:1920069" "1801000"
CahidArda commented 1 month ago

Hey @sgrodzicki,

This looks like an error which occurs in the latest release when enableProtection: true.

After some investigation, I think this is caused by auto pipelining in redis client. You should be able to address the issue by disabling it:

const redis = Redis.fromEnv({ enableAutoPipelining: false });

In the meantime, we will investigate why this is the case.

CahidArda commented 2 weeks ago

Hi @sgrodzicki,

sorry for the late reply, we released @upstash/redis v1.34.3, which should address this issue. You should be able to upgrade to the latest redis version and create a redis client without { enableAutoPipelining: false }

CahidArda commented 2 weeks ago

Closing the issue since the error was addressed.