redis / redis-om-python

Object mapping, and more, for Redis and Python
MIT License
1.1k stars 108 forks source link

Redis-om model randomly returns NotFoundError #648

Open llastowski opened 1 month ago

llastowski commented 1 month ago

The model below, with indexed fields, as pasted below, returns data without issues in 13 cases and fails 2, running the same 15 unit tests core function to retrieve the object from Redis. In to cases, however, it fails to pull data with no apparent reason; The workaround added to find ALL entries when catching exception works, but the same behavior is observed in multiple other places with different models, all related to the same query.

The environmental variables and the name parameter are the same for all tests, but it only fails on 2 runs of the same function. It also fails to find the object in related Python debugger console attached at breakpoint.

Model definition:

class PlatformR(JsonModel):
    """DDI Platform representation."""

    model_config = ConfigDict(
        strict=False,
    )

    id: str = RedisField(index=True)
    name: str = RedisField(index=True)

    hostname: str = RedisField(index=True)
    company: str = RedisField(index=True)
    api_version: Optional[str] = None
    credentials: Optional[PlatformCredentials] = None

Lookup function

    def get_hostname_by_name(self, ddi_platform: str) -> str:
        """Get hostname by platform name."""
        logging.debug("Get hostname by platform name")
        try:
            return (
                PlatformR.find(PlatformR.name == ddi_platform).first().hostname
            )
        except NotFoundError:
            logging.error(f"Platform {ddi_platform} not found")
            ### Workaround - find all object, filter manually by Python iteration
            for platform in PlatformR.find().all(): 
                if platform.name == ddi_platform:
                    return platform.hostname
            raise HTTPException(
                status_code=status.HTTP_400_BAD_REQUEST,
                detail=f"Platform {ddi_platform} not found",
            )
llastowski commented 1 month ago

The error is observed in 0.3.2 and 0.3.1, previously we were using 0.2.1 without observing this issue.

slorello89 commented 3 weeks ago

How large is the index you are querying, and how large is the result set when you call all?

ivanbelenky commented 2 weeks ago

Experiencing the same issue with an index sized at 10K approximately. Issue with Object.get(Object.id == 'abcd...'). Not experiencing issues with Object.find(Object.id == 'abcd...').all()

redis-om version 0.3.1

llastowski commented 2 weeks ago

How large is the index you are querying, and how large is the result set when you call all?

there are 4 keys in that specific model in the DB, approx 10KB in total, the index is 128 Bytes Whole database in the specific environment is just 56 keys