search

ceph / s3-tests

Compatibility tests for S3 clones
MIT License
297 stars 287 forks source link

rgw/s3select: json output format #548

Open albin-antony opened 8 months ago

galsalomon66 commented 4 months ago

the following functions run_s3select run_s3select_output run_s3select_json run_s3select_csv_json_format

they have much in common, it is possible to combine them into one function.

galsalomon66 commented 4 months ago

@albin-antony, please review comments the goal of s3-tests (in the context of s3select) is to check the integration with RGW

in the case of Parquet, it needs to make sure that RGW is correctly built with arrow-library the s3select-unit-tests does an automatic conversion (CSV->parquet), processes the same SQL statement for each data source, and compares results.

In the JSON output format, it needs to ensure that output-serialization parameters pass through the RGW into the s3select engine. the JSON output format functionalities are tested in s3select-unit-tests.

albin-antony commented 4 months ago

@galsalomon66 Thanks for the review. Addressed the comments. Please take a look

galsalomon66 commented 3 months ago

@albin-antony please rebase