Pandaproxy features meta-issue

[BenPope]

Pandaproxy is a work in progress, below is a list of features that we are aware of, and whether they have been implemented.

Endpoints

• [x] GET /topics - Get a list of topics
• [ ] GET /topics/{topic} - Get data about a specific topic
• [ ] GET /topics/{topic}/partitions - Get data about the partitions of a topic
• [ ] GET /topics/{topic}/partitions/partition_id} - Get data about a single partition
• [ ] GET /topics/{topic}/partitions/partition_id}/offsets - Get data about he offsets of a partition
• [x] POST /topics/{topic} - Produce records to a topic
  • [x] Binary - application/vnd.kafka.binary.v2+json
  • [x] JSON - application/vnd.kafka.v2+json
  • [ ] Avro - application/vnd.kafka.avro.v2+json
  • [ ] Protobuf - application/vnd.kafka.protobuf.v2+json
• [ ] POST /topics/{topic}/partitions/{partition_id} - Produce records to a specific partition
  • [ ] Binary - application/vnd.kafka.binary.v2+json
  • [ ] JSON - application/vnd.kafka.v2+json
  • [ ] Avro - application/vnd.kafka.avro.v2+json
  • [ ] Protobuf - application/vnd.kafka.protobuf.v2+json
• [x] POST /consumers/{consumer_group} - Create a consumer
• [x] DELETE /consumers/{consumer_group}/instances/{instance} - Remove a consumer
• [x] GET /consumers/{consumer_group}/instances/{instance}/offsets - Get committed offsets for the consumer
• [x] POST /consumers/{consumer_group}/instances/{instance}/offsets - Commit offsets for the consumer
• [ ] GET /consumers/{consumer_group}/instances/{instance}/subscription - Get the list of subscribed topics for the group
• [x] POST /consumers/{consumer_group}/instances/{instance}/subscription - Subscribe the group to topics
• [ ] DELETE /consumers/{consumer_group}/instances/{instance}/subscription - Unsubscribe the group from all topics
• [ ] POST /consumers/{consumer_group}/instances/{instance}/assignments - Manually assign partitions to the consumer
• [ ] GET /consumers/{consumer_group}/instances/{instance}/assignments - Get the manually assigned partitions for the consumer
• [ ] POST /consumers/{consumer_group}/instances/{instance}/positions - Override the fetch offsets for the consumer
• [x] POST /consumers/{consumer_group}/instances/{instance}/positions/beginning - Set the fetch offsets to the beginning for the consumer
• [ ] POST /consumers/{consumer_group}/instances/{instance}/positions/end - Set the fetch offsets to the end for the consumer
• [x] GET /consumers/{consumer_group}/instances/{instance}/records - Fetch records for the consumer
  • [x] Binary - application/vnd.kafka.binary.v2+json
  • [x] #1789
  • [ ] Avro - application/vnd.kafka.avro.v2+json
  • [ ] Protobuf - application/vnd.kafka.protobuf.v2+json
• [x] GET /brokers - Get a list of brokers #1920

Endpoints on SwaggerHub

JIRA Link: CORE-609

[emaxerrno]

@BenPope should it be a nested list

one per type

• [x] GET /brokers
  • [x] JSON
  • [x] AVRO
  • [ ] Protobuf

[john-livelyvideo]

What about Fetch Offsets by Timestamp?

https://kafka.js.org/docs/next/admin#a-name-fetch-topic-offsets-a-fetch-topic-offsets

[john-livelyvideo]

^Or is redpanda missing this functionality?

[BenPope]

^Or is redpanda missing this functionality?

Pretty sure it's supported by Redpanda. I've been working on the Schema Registry recently, but I'll probably get to add some features to Pandaproxy next month.

[john-livelyvideo]

For a partition, when doing event sourcing type stuff, and deltas have been merged into a snapshot_delta, it's nice to be able to cleanup the deltas.

https://kafka.js.org/docs/admin#a-name-delete-topic-records-a-delete-topic-records

[khionu]

Are there any plans to support topic creation/mutation via Pandaproxy?

[Brotzka]

Is Pandaproxy still under active development?

I wonder if it would be something we could use in production..

[mattschumpert]

Hi, yes it is supported in production, though it can notably increase latency when compared with many Kafka clients such as librdkafka. We are in the process of publishing documentation for this feature, please bear with us

@Brotzka

[shazamkash]

Any updates on the documentation for this feature?

@mattschumpert

[tlopesPT]

Is there a way to pass headers using the post to topic endpoint?

[flyaruu]

So if I understand correctly, the whole concept of partitions assigned to different consumers within a consumer group isn't implemented at all, right?

[BenPope]

So if I understand correctly, the whole concept of partitions assigned to different consumers within a consumer group isn't implemented at all, right?

Consumer groups are supported with the range assignment strategy.

[deniskrumko]

Is there a way to pass headers using the post to topic endpoint?

Same question, very interested if that's possible rn?