Input does not start after ContentPack ReST creation

gianluca-valentini commented 3 years ago

Expected Behavior

When I create a contentPack using graylog ReST API and apply it in a new and empty environment, the input should start without error

Current Behavior

When I create a contentPack using graylog ReST API and apply it in a new and empty environment, the input start fails. The log tell me that the port is already in use (it is a new installation). Restarting the application the input become magically green

Possible Solution

Maybe something change in ReST input creation with content pack and UI one

Steps to Reproduce (for bugs)

Create a stream and an input
Export it using content pack
Import content pack using rest client
The input does not start and the log describe the error (port already in use

Context

Your Environment

ContentPack creation and installation work fine but I have this problem on the input view

Looking the Graylog log I see:

If I create another input using the UI it works fine

I'm sure that the 12201 is not used. If I restart the application it become green

Graylog Version: Applicance: Graylog 4.0.7+c3e766c on graylog (Private Build 1.8.0_292 on Linux 4.15.0-142-generic)
Java Version: openjdk version "1.8.0_292"
Elasticsearch Version: 7.10.2
MongoDB Version: 4.0.24
Operating System: ubuntu 18.04
Browser version:

gianluca-valentini commented 3 years ago

I add other additional info currently I’m using graylog OVA 4.0.7. Now I have an empty environment where I delete all input

using the command ss-tulpn I see

Then I create and install my content-pack (without parameters but with required inputs) Content pack is uploaded with success but looking the input I see

running ss -tulpn

When I try to start the input I have and error:

where the log say

2021-05-19T13:04:51.627Z WARN [AbstractTcpTransport] Failed to start channel for input BeatsInput{title=AX-Beat, type=org.graylog.plugins.beats.BeatsInput, nodeId=null}
io.netty.channel.unix.Errors$NativeIoException: bind(…) failed: Address already in use
2021-05-19T13:04:51.627Z ERROR [InputLauncher] The [org.graylog.plugins.beats.BeatsInput] input with ID <60a50c20a010fd30e2204d1a> misfired. Reason: bind(…) failed: Address already in use.
org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: io.netty.channel.unix.Errors$NativeIoException: bind(…) failed: Address already in use
at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_292]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
Caused by: org.graylog2.plugin.inputs.MisfireException: io.netty.channel.unix.Errors$NativeIoException: bind(…) failed: Address already in use
at org.graylog2.plugin.inputs.transports.AbstractTcpTransport.launch(AbstractTcpTransport.java:199) ~[graylog.jar:?]
at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
… 7 more
Caused by: io.netty.channel.unix.Errors$NativeIoException: bind(…) failed: Address already in use
2021-05-19T13:04:51.631Z INFO [InputStateListener] Input [Beats (deprecated)/60a50c20a010fd30e2204d1a] is now FAILED

At the end the strange, very strange, situation is that is I restart Graylog, I don’t have input error anymore

Other point of this strange scenario is that if I delete all and run running ss -tulpn again

So the input ports seem to be used again

rantonmo commented 2 years ago

I'm facing the same problem using the docker images on a Kubernetes cluster (AKS). In my case, the content pack includes three inputs but only one is failing.