Deleting a stream via the REST API can block forever

[mihaitodor]

Steps to reproduce:

1. Start Connect:

> redpanda-connect streams

2. Create a simple stream via the REST API with a failing output which blocks messages forever:

> curl http://localhost:4195/streams/bar -X POST --data-binary @- <<EOF
input:
  broker:
    inputs:
    - generate:
        count: 1
        interval: 1s
        mapping: root = ""

output:
  retry:
    output:
      http_client:
        url: localhost:8080
EOF

3. Try to delete the stream via the REST API:

> curl http://localhost:4195/streams/bar -X DELETE

Note that the curl command will hang and cancelling it via Ctrl+C will produce the following in the Connect log:

WARN Failed to send message: localhost:8080: Post "localhost:8080": unsupported protocol scheme "localhost"  @service=benthos label="" path=root.output stream=bar
ERRO Encountered error whilst attempting to shut down gracefully: context canceled  @service=benthos stream=bar
INFO Some components prevented forced termination as they were either blocked from delivering data or from acknowledging delivered data within the shutdown timeout. This could potentially cause duplicate messages to be delivered on the next run.  @service=benthos stream=bar
ERRO Streams CRUD Error: context canceled          @service=benthos

No more logs are produced after this, but trying to recreate the stream returns an error message saying "Stream already exists". Running the DELETE command again succeeds and now the stream can be recreated.

[SathveeganYogendrarajah2000]

I also encounter the same issue when attempting to DELETE, PUT, or PATCH a stream via the REST API.

However, unlike the original poster, I’m unable to delete the stream at all, and it remains in a stuck state. Running the DELETE command again doesn’t resolve the issue for me.

Is there any alternate solution or workaround to successfully delete or modify the stream?