[Plugin]BlazingSQL and Rapids dataframe support

[kumare3]

Why would this plugin be helpful to the Flyte community It would be great to have a simple plugin that would allow users to simply write BlazingSQL queries in their workflows to manipulate data-frames on a GPU as part of their Flyte workflows.

An example of writing a query from the User point of view could be

task = BlazingSQL(
     input_schema=FlyteSchema,
    outputs=...,
    query=""" SELECT count(*) FROM taxi GROUP BY year(key)""",
    use_gpus=False, # True by default
)

Type of Plugin

• [x] Python/Java interface only plugin
• [ ] Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
• [ ] Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
• [ ] Customized Plugin using native kubernetes constructs
• [ ] Other

[DasithEdirisinghe]

@kumare3 can you assign me to this issue. and refer me to some more resources relevant to this issue

[samhita-alla]

Hi, @DasithEdirisinghe! Thanks for looking into this. :)

This plugin has to be a task template plugin and on the same lines as that of SQLAlchemy plugin.

During plugin development, follow the directory structure used for SQLAlchemy plugin. To test the plugin, refer to https://docs.flyte.org/projects/flytekit/en/latest/contributing.html#plugin-development.

[kumare3]

@DasithEdirisinghe are you working on this?

[DasithEdirisinghe]

@kumare3 yes I started working

[kumare3]

@kumare3 yes I started working

Cool I am assigning it to you! Thank you

[samhita-alla]

@DasithEdirisinghe, two more days till Hacktoberfest ends! Would you mind creating a PR in the next day or two?

[github-actions[bot]]

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

[github-actions[bot]]

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏

[github-actions[bot]]

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏