[native] Add TPC-DS connector

pdabre12 commented 3 months ago

Description

Adds the native TPC-DS connector and the codegen (dsdgen) files. The TPC-DS connector will be a wrapper for the data generator (dsdgen) provided by the TPC organization, originally implemented in C. This implementation must mimic the behavior of the original C implementation. DuckDB has a TPC-DS connector that uses C++ files to wrap the C implementation, and we will use these C++ files for our implementation.

Motivation and Context

The goal is to add a TPC-DS connector to enhance our ability to write end-to-end tests for Prestissimo and conduct micro-benchmarks in Velox. This addition will significantly improve our testing capabilities by ensuring comprehensive coverage and performance validation.

Impact

With this change, we can now write e2e tests and microbenchmarks.

Test Plan

Native end-to-end tests. Future enhancements will include adding SpeedTest and ConnectorTest to the Velox repository.

Contributor checklist

[ ] Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
[ ] PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
[ ] Documented new properties (with its default value), SQL syntax, functions, or other functionality.
[ ] If release notes are required, they follow the release notes guidelines.
[ ] Adequate tests were added if applicable.
[ ] CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

TPC-DS Connector Changes
* Add TPC-DS connector for Presto C++. :pr:`23067`

Resolves: https://github.com/prestodb/presto/issues/22361

linux-foundation-easycla[bot] commented 3 months ago

The committers listed above are authorized under a signed CLA.

:white_check_mark: login: pdabre12 / name: Pratik Joseph Dabre (2fafdcefebbbdb9036285fdb8dc758af88a81b63, 9dba5983dc68802c9e390b0d8999bb9db015960c, b895ea738d2360ddba5a7757d0afe5267d67ced4, 5ff51eff9d187f7f8249523d754a3c4f0724ce00)

steveburnett commented 2 months ago

Consider updating the Use Cases in the Presto C++ section of the Presto doc to add TPC-DS to the supported connectors.

steveburnett commented 1 month ago

Consider a release note entry similar to the following:

== RELEASE NOTES ==

TPC-DS Connector Changes
* Add TPC-DS connector for Presto C++. :pr:`23067`

pdabre12 commented 1 month ago

@aditi-pandit @yingsu00 This PR is now ready for review. Please take a look. Thanks.

pdabre12 commented 1 month ago

FYI: @czentgr

pdabre12 commented 1 month ago

@aditi-pandit @czentgr. Thanks for the review. Stalling this PR until we figure out the licensing.

pdabre12 commented 1 month ago

@czentgr @aditi-pandit The licensing issue is resolved. Addressed your comments , can you have another look? Thank you.

Will address the failing header checks soon.

aditi-pandit commented 2 weeks ago

@pdabre12 Some comments. ~I think we can merge the external portion first in a separate commit as they are copied externally. That makes it easy to review the new bits.~ I see that it is already a separate commit.

@aditi-pandit, @tdcmeehan I think we should have a RFC for this work. What do you think? The RFC can be simple.

@pdabre12 @majetideepak : Small RFC is okay. Let's bring this to Native worker working group next session so that it gets immediate attention from others.

prestodb / presto