AWS CloudWatch/Kinesis input restart issue

[danotorrey]

When editing and saving the settings for an existing AWS Kinesis/CloudWatch input, the new settings are not effective until the server restarts.

Steps to reproduce the issue:

1) Create a new AWS Kinesis/Cloudwatch input. 2) Verify that the input is receiving messages. 3) Attempt to change a setting on the input (change the key/secret, or change the stream name). 4) You should see that the change is saved, but is not actually effective (eg. still using the old credentials or stream). 5) Restarting the server always seems to fix the issue.

[lingpri]

I'm looking into this issue.

[lingpri]

@danotorrey - I'm thinking, that we change the AWS Kinesis Input stream name to a dropdown ?. Please advice. Please see the skeleton code I'm thinking of including in the AWSInput. Also, I need to write two new methods getConfiguredStreamNameForRegion() and getAllStreamNamesForRegion() in the AWSService. Please keep me posted on your thoughts.

  request.addField(new DropdownField(
                    KinesisTransport.CK_KINESIS_STREAM_NAME,
                    "Kinesis Stream Name",
                    AWSService.getConfiguredStreamNameForRegion(),
                    AWSService.getAllStreamNamesForRegion(),
                    "The Kinesis stream name, the input is running in.",
                    ConfigurationField.Optional.NOT_OPTIONAL));

[danotorrey]

@lingpri I like this idea, but it is not currently possible with the generated edit page form. Currently, when the form is about to display, and API call is executed to the backend. The call then returns the entire form content (fields, labels, description, and choices). The problem is that, a user could enter a different API key (without saving), and there is no way to run custom Javascript to retrieve the list of streams for the current account until a save and edit is performed.

There has been talk of improving the generated form to support more dynamic content, so perhaps you can continue the discussion on this with the team.

[lingpri]

@danotorrey - I see what you are saying. I'm thinking, If we are looking for a field level validation, can we change the workflow to take the same wizard like interface, while creating the input . I can talk to @kyleknighted to see what he thinks.

There seems to be an another related issue, https://github.com/Graylog2/graylog-plugin-integrations/issues/344. Possibly need to revisit.

[waab76]

Agree that the Edit interface for this Input should be updated to use the wizard that we use at input creation and both things (fixing the restart issue and updating the user interface for edit) should be done at the same time.