apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.47k stars 1.28k forks source link

Feature Request : Support DELETE segment operation via SQL #13476

Open piby180 opened 4 months ago

piby180 commented 4 months ago

We already have INSERT INTO operation in Pinot

INSERT INTO events
FROM FILE 's3://marks-st-cloud-bucket/events/'
OPTION(
  taskName=events-task,
  includeFileNamePattern=glob:**/*.json,
  input.fs.className=org.apache.pinot.plugin.filesystem.S3PinotFS,
  input.fs.prop.accessKey=AKIARCOCT6DWLUB7F77Z,
  input.fs.prop.secretKey=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,
  input.fs.prop.region=eu-west-2
);

It would be nice to support operation for deleing segments via SQL as well.

Something like this to delete one segment by name

DELETE FROM events
WHERE $segmentName = "<segment_name>"

Or delete segments which matches name pattern

DELETE FROM events
WHERE $segmentName LIKE "<segment_name_pattern>"
sullis commented 3 months ago

I am taking a look at this issue