search

v6d-io / v6d

vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)
https://v6d.io
Apache License 2.0
818 stars 117 forks source link

bug: I want append certain num of null rows to vertex table, then write to parquet, got failed #1840

Closed acezen closed 3 months ago

acezen commented 3 months ago

Describe your problem

The ArrowFragmetWriter has to append certain number of nulls to vertex table if the table size it not align to vertex chunk size. But if the table got timestamp column, when it append the nulls, and write with parquet, got error:

In chunk 1 expected type timestamp[ms, tz=[UTC]] but saw timestamp[ms, tz=UTC]

The test case can use to reproduce the problem.

run like :

arrow_fragment_append_nulls_and_write /tmp/vineyard.sock /workspaces/GraphScope/gstest/ldbc_sample/comment_0_0 /workspaces/GraphScope/gstest/ldbc_sample/comment_replyOf_comment_0_0

If is is a bug report, to help us reproducing this bug, please provide information below:

  1. Your Operation System version :Ubuntu20.04
  2. The version of vineyard you use (vineyard.__version__): 0.21.5
  3. Versions of crucial packages, such as gcc, numpy, pandas, etc.:
  4. Full stack of the error (if there are a crash):
  5. Minimized code to reproduce the error:

If it is a feature request, please provides a clear and concise description of what you want to happen:

What is the problem:

The behaviour that you expect to work:

Additional context

Add any other context about the problem here.