Getting TagNotRegisteredError when my key have special characters (Using custom decorator)

[shijo-keyvalue]

Hi @Krukov , I am using a custom decorator on top of @cache decorator to enhance it to achieve some of our custom requirements. My custom decorator code is given below.

I am getting TagNotRegisteredError , when by key have some special characters like '$' , '?' etc. Providing the error log and our custom decorator code below.

1. What could be the cause of this Error ? Not getting this error , when I am directly using @cache decorator.
2. Is the usage of cache.cache() function in custom decorator correct ?

               # Construct the original cashews cache decorator with the modified arguments
               original_cache_decorator = cache.cache(*decorator_args, **updated_decorator_kwargs)

Used redis and cashews modules : redis-py=5.0.0 cashews=6.3.0

Error :

  File "<project-path>/./src/common/api_cache_util.py", line 336, in wrapper
    return await decorated_func(*func_args, **func_kwargs)
  File "<path>/lib/python3.8/site-packages/cashews/wrapper/decorators.py", line 57, in _call
    return await thunder_protection(decorator)(*args, **kwargs)
  File "<path>/lib/python3.8/site-packages/cashews/decorators/locked.py", line 96, in _wrapper
    return await task
  File "<path>/lib/python3.8/site-packages/cashews/decorators/cache/simple.py", line 64, in _wrap
    await backend.set(_cache_key, result, expire=_ttl, tags=_tags)
  File "<path>/lib/python3.8/site-packages/cashews/wrapper/tags.py", line 109, in set
    self._tags_registry.check_tags_registered(key, tags)
  File "<path>/lib/python3.8/site-packages/cashews/wrapper/tags.py", line 43, in check_tags_registered
    raise TagNotRegisteredError(f"tag: {difference} not registered: call cache.register_tag before using tags")
cashews.exceptions.TagNotRegisteredError: tag: {'client_id:499:projects'} not registered: call cache.register_tag before using tags

Custom decorator implementation

from cashews import cache

def api_cache(*decorator_args: Any, **decorator_kwargs: Any) -> Callable:
    """
    A custom api caching decorator , which enhances the functionality of the @cache() decorator from the cashews library by
    introducing custom cache key generation and client-specific tag prefixing for cache invalidation.

    :param decorator_args: Positional arguments to be passed to the cashews cache decorator.
    :param decorator_kwargs: Keyword arguments to be passed to the cashews cache decorator.
    :return: The decorated asynchronous function.
    """

    def decorator(func: Callable) -> Callable:
        """
        The actual decorator function which wraps around the target asynchronous function.

        :param func: The asynchronous function to be decorated.
        :return: The wrapped asynchronous function.
        """

        @wraps(func)
        async def wrapper(*func_args: Any, **func_kwargs: Any) -> Any:
            """
            Invoked in place of the original asynchronous function, this wrapper function takes charge of
            modifying the cache key and tags based on the extracted client ID. It then constructs and applies the original
            @cache decorator from the cashews library with the above mentioned modifications in key and tags before
            invoking the original function.

            :param func_args: Positional arguments passed to the original function.
            :param func_kwargs: Keyword arguments passed to the original function.
            :return: The result of the original asynchronous function.
            """

            # Check if caching should be skipped
            if should_skip_api_cache():
                # Directly call and return the original function's result
                logger.info(f"Skipped API caching : {func.__module__} -> {func.__name__}")
                return await func(*func_args, **func_kwargs)

            try:
                # Check if cache invalidation required
                func_context = FunctionContext(func, func_args, func_kwargs)
                invalidate_cache_context = await get_cache_invalidation_context(func_kwargs, decorator_kwargs)
                if invalidate_cache_context.should_invalidate:
                    logger.info(f"Bypass and clear API cache : {func.__module__} -> {func.__name__}")
                    return await handle_bypass_and_clear_cache(func_context, decorator_kwargs, invalidate_cache_context)

                updated_decorator_kwargs = update_cache_key_and_tags(func_args, func_kwargs)

                # Construct the original cashews cache decorator with the modified arguments
                original_cache_decorator = cache.cache(*decorator_args, **updated_decorator_kwargs)
                decorated_func = original_cache_decorator(func)

                # Call the original function
                return await decorated_func(*func_args, **func_kwargs)
            except Exception as e:
                logger.error(f"Error occurred during API caching for {func.__module__} -> {func.__name__}: {str(e)}", exc_info=True)
                return await func(*func_args, **func_kwargs)

        def update_cache_key_and_tags(func_args: Tuple, func_kwargs: Dict) -> Dict:
            """
            Updates the cache key and tags based on the client ID.

            :param func_args: Positional arguments passed to the original function.
            :param func_kwargs: Keyword arguments passed to the original function.
            :param client_id: The extracted client ID.
            :return: A dictionary of updated decorator keyword arguments.
            """
            client_id = extract_client_id(func_kwargs)
            func_context = FunctionContext(func, func_args, func_kwargs)
            custom_key = build_custom_key(func_context, client_id, decorator_kwargs)
            modified_decorator_kwargs = {**decorator_kwargs, "key": custom_key}

            # If tags are present, prefix them with the client ID for client-specific caching
            if TAGS_ARG in modified_decorator_kwargs and isinstance(modified_decorator_kwargs[TAGS_ARG], list):
                modified_decorator_kwargs[TAGS_ARG] = build_client_id_prefixed_tags(
                    client_id, modified_decorator_kwargs[TAGS_ARG]
                )

            return modified_decorator_kwargs

        return wrapper

    return decorator

    def build_custom_key(func_context: FunctionContext, client_id: str, decorator_kwargs: Dict) -> str:
    """
    Construct a unique caching key based on the provided function, its arguments, and client ID.

    :param func: The target function.
    :param func_args: Positional arguments of the function.
    :param func_kwargs: Keyword arguments of the function.
    :param client_id: A client identifier.
    :return: The constructed custom cache key.
    """
    module_name = func_context.func.__module__
    function_name = func_context.func.__name__
    version_str = f":{VERSION_STR}={API_CACHE_VERSION}" if API_CACHE_VERSION else ""
    args_str = build_args_str(func_context.args)
    kwargs_str = build_kwargs_str(func_context.kwargs)
    client_id_str = f":{CLIENT_ID_STR}={client_id}" if client_id else ""
    return f"{API_CACHE_PREFIX}{module_name}:{function_name}{version_str}{client_id_str}:{args_str}:{kwargs_str}"

[Krukov]

Hi @shijo-keyvalue

1) that is a bug with tags "templating" - fix is on the way 2) LGFM but just wondering why do you need to write this custom api cache decorator? Can you share a code of extract_client_id?

[shijo-keyvalue]

Hi @Krukov Thanks for the quick response. Appreciate it.

1. Is there any workaround for this issue. Currently I am catching TagNotRegisteredError and deleting cache entry from redis directly (as cache entry with given key is getting added to redis , even if exception is thrown)

2. We wanted bit more control on caching. For example - (a) we wanted to bypass and clear cache when a specific request header is present or specific key is present in redis cache. (b) Also we had requirement for building custom tag entries by prefixing certain attributes present in jwt token (Kind of key templating) . We were unable to solve this without custom decorator at the time we started this. Please let me know , if these can be solved without custom decorator.

PS : I just saw documentation about CacheDeleteMiddleware. It was not there earlier when we started with this.

[Krukov]

1. Probably you can try to wrap a tag with re.escape

2. a) yes you can bypass and remove cache by wrap call to context manager invalidate_further - for example add a middleware to your fastapi app

   
   from cashews import invalidate_further

async def middleware(request, call_next): if request.headers.get(_YOUR_HEARED) == _CLEAR_CACHE_HEADER_VALUE: with invalidate_further(): return await call_next(request)

    return await call_next(request)

b) yes probably now we can't use advance templating in tags ( like in keys) - it is easy to fix and this [MR](https://github.com/Krukov/cashews/pull/188) implement it - and it will looks like following

```python
@cache(ttl="1h", tags=["items", "client_id:{token:jwt(client_id)}"])

[Krukov]

Fixed in 7.0.0