Simple redis throws error no wait_for_capacity_sync

[gingerwizard]

Following (v0.3.0)

#!/usr/bin/python3
import sys
import traceback

import openai
from openlimit import ChatRateLimiterWithRedis

openai.api_key = "<token>"
request_timeout = 3

rate_limiter = ChatRateLimiterWithRedis(
    request_limit=3500,
    token_limit=90000,
    redis_url="redis://localhost:6379/0"
)

@rate_limiter.is_limited()
def call_openai(**chat_params):
    return openai.ChatCompletion.create(**chat_params)

def extract_sentiment(text):
    if text == "":
        return "NEUTRAL"
    messages = [{"role": "system",
                 "content": "You are an AI language model trained to analyze and detect the sentiment of hackernews forum comments."},
                {"role": "user",
                 "content": f"Analyze the following hackernews comment and determine if the sentiment is: positive, negative or neutral. "
                            f"Return only a single word, either POSITIVE, NEGATIVE or NEUTRAL: {text}"}]
    try:
        params = {
            "model": "gpt-3.5-turbo",
            "messages": messages,
            "max_tokens": 30,
            "temperature": 0,
            "request_timeout": request_timeout
        }
        response = call_openai(**params)
        sentiment = response.choices[0].message.content.strip()
        return "NEUTRAL" if sentiment == "" else sentiment
    except Exception as e:
        traceback.print_exc()
        return f"ERROR - {e}"

for size in sys.stdin:
    # collect a batch for performance
    for row in range(0, int(size)):
        print(extract_sentiment(sys.stdin.readline().strip()))
    sys.stdout.flush()

throws an exception

AttributeError: 'NoneType' object has no attribute 'wait_for_capacity_sync'
ERROR - 'NoneType' object has no attribute 'wait_for_capacity_sync'
Traceback (most recent call last):
  File "/opt/udfs/sentiment_distributed.py", line 39, in extract_sentiment
    response = call_openai(**params)
  File "/opt/udfs/.venv/lib/python3.9/site-packages/openlimit/utilities/context_decorators.py", line 22, in wrapper
    with self.rate_limiter.limit(**kwargs):
  File "/opt/udfs/.venv/lib/python3.9/site-packages/openlimit/utilities/context_decorators.py", line 44, in __enter__
    self.rate_limiter._request_bucket.wait_for_capacity_sync(1)
AttributeError: 'NoneType' object has no attribute 'wait_for_capacity_sync'

printf '6\n\nThis page seems to cover most of the prerequisites, and some other advice: <a href="https:&#x2F;&#x2F;github.com&#x2F;yandex&#x2F;ClickHouse&#x2F;blob&#x2F;master&#x2F;doc&#x2F;build.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yandex&#x2F;ClickHouse&#x2F;blob&#x2F;master&#x2F;doc&#x2F;build.m...</a><p>And it does say this: <i>&quot;With appropriate changes, build should work on any other Linux distribution...Only x86_64 with SSE 4.2 is supported&quot;</i>\nThanks so much for that. It definitely looks like the documentation might be out of sync as there are also some references to ODBC support in the code:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;yandex&#x2F;ClickHouse&#x2F;blob&#x2F;32057cf2afa965033cf850a081e7a9dfa306594d&#x2F;dbms&#x2F;src&#x2F;ODBC&#x2F;info.cpp" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yandex&#x2F;ClickHouse&#x2F;blob&#x2F;32057cf2afa965033c...</a><p>Going to look into building a Spark data source for this so we can see how it well it compares to databases like Cassandra.\nWithout supporting other operating systems it is hard to consider it as an alternative to anything. We have several clusters using another operating system than the one supported by ClickHouse. Unfortunately few customers are going to invest into a different platform to try out something new like this. The lower the entering bar for new tech is the better.\nThis is huge. It seems to me that it&#x27;s similar to BigQuery but has many other features that I didn&#x27;t see in other databases.<p>AggregatingMergeTree is especially one of them and allows incremental aggregation which is a huge gain for analytics services.<p>Also it provides many table engines for different use-cases. You don&#x27;t even need a commit-log such as Apache Kafka in front of ClickHouse, you can just push the data to TinyLog table and and move data in micro-batches to a more efficient column-oriented table that uses different table engine.\n\n' | python3 sentiment.py

[shobrook]

I will look into this either tomorrow or Friday. Thanks for the heads up!

[tahamukhtar20]

You can try doing something like this => @rate_limiter.is_limited() async def call_openai(kwargs): print("Calling OpenAI API...") result = openai.ChatCompletion.create(kwargs) print("OpenAI API call completed.") return result

on inspecting the source code, the sync part of the Redis one isn't properly implemented, but the async one works, as in the aenter dunder that wait_for_capacity() function is properly called unlike the enter dunder.

This worked for me, hope this helps.

[yishaiSilver]

I'm running into the same issue, but I can't seem to get the async function to work.