man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
http://arcticdb.io
Other
1.52k stars 93 forks source link

lib.read report ('E_ASSERTION_FAILURE Truncate bounds start_row=12560 end_row=12560 outside valid range 12560',) #1847

Closed deyu9999 closed 1 month ago

deyu9999 commented 1 month ago

Describe the bug

working code: vertionItem = ac_lib.read(symbol=order_book_id, date_range=(start_date, end_date), columns=columns) not working code: vertionItem = ac_lib.read(symbol=order_book_id, query_builder=q, columns=columns)

when using not working code, got error reported here: [2024-09-24 15:48:57.289] [arcticdb] [error] E_ASSERTION_FAILURE Truncate bounds start_row=12560 end_row=12560 outside valid range 12560 when using working code, all else same, I got expected data.

Don't know why.

Steps/Code to Reproduce

working code: vertionItem = ac_lib.read(symbol=order_book_id, date_range=(start_date, end_date), columns=columns) not working code: vertionItem = ac_lib.read(symbol=order_book_id, query_builder=q, columns=columns)

def query_bar_lib(self, lib_name: str, order_book_id: str, start_date: datetime, end_date: datetime, columns: [] = None) -> DataFrame:
    """
    query table
    """
    try:
        if not self.ac.has_library(lib_name):
            return None

        ac_lib = self.ac.get_library(lib_name)
        q = QueryBuilder()
        if start_date is None:
            start_date = datetime(1990, 1, 1)
        if end_date is None:
            end_date = datetime.now()
        q.date_range((start_date, end_date))

        vertionItem = ac_lib.read(
            symbol=order_book_id, query_builder=q, columns=columns)

        return vertionItem.data
    except Exception as e:
        raise e
        return None

Expected Results

get expected results and no error raised.

OS, Python Version and ArcticDB Version

Python: 3.9.16 (main, May 15 2023, 23:46:34) [GCC 11.2.0] OS: Linux-5.4.119-19-0009.11-x86_64-with-glibc2.28 ArcticDB: 4.2.1

Backend storage used

Tencent COS

Additional Context

No response

IvoDD commented 1 month ago

Can you try upgrading to a latest arcticdb version (4.5.0 has a lot of QueryBuilder improvements)?

I tried locally reproducing your issue with 4.5.0 and didn't manage to.

If you still get the error can you send over the exact start_date and end_date you're using and the stored dataframe (not necassarily the whole dataframe, just the shape and first and last index would be enough). Thanks!

poodlewars commented 1 month ago

Closing as no reply