search

thingsboard / thingsboard

Open-source IoT Platform - Device management, data collection, processing and visualization.
https://thingsboard.io
Apache License 2.0
17.4k stars 5.14k forks source link

[Bug] Failed to Upgrade Thingsboard CE from 3.2.2 to 3.3: Error creating bean with name MqttTransportService #5125

Closed iwas108 closed 3 years ago

iwas108 commented 3 years ago

Describe the bug Upgrade from 3.2.2 to 3.3 on Ubuntu Server 12.04 resulting in failed to start thingsboard service.

Your Server Environment

Your Client Environment

Desktop (please complete the following information):

Your Device

To Reproduce Steps to reproduce the behavior:

  1. Login to Server via SSH
  2. Visit https://thingsboard.io/docs/user-guide/install/upgrade-instructions/
  3. Do upgrade from 3.2.2 to 3.3
  4. See error

Expected behavior After executing final step of sudo service thingsboard start it should start the thingsboard service version 3.3, but it failed with error.

Screenshots

2021-08-21 23:45:04,179 [main] INFO  o.t.server.common.data.ResourceUtils - Reading key store from class path lwm2mserver.jks
2021-08-21 23:45:09,023 [main] INFO  o.t.s.a.service.DefaultActorService - Initializing actor system.
2021-08-21 23:45:09,085 [main] INFO  o.t.s.a.service.DefaultActorService - Actor system initialized.
2021-08-21 23:45:10,738 [main] INFO  o.t.s.c.t.a.JsonConverterConfig - JSON max string value length = 0
2021-08-21 23:45:10,746 [main] INFO  o.t.s.c.t.a.JsonConverterConfig - JSON type cast enabled = true
2021-08-21 23:45:10,999 [main] INFO  o.t.s.t.mqtt.MqttTransportService - Setting resource leak detector level to DISABLED
2021-08-21 23:45:11,007 [main] INFO  o.t.s.t.mqtt.MqttTransportService - Starting MQTT transport...
2021-08-21 23:45:11,160 [main] WARN  o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - ca
ncelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'MqttTransportService': Invocation of init me
thod failed; nested exception is java.net.BindException: Address already in use
2021-08-21 23:45:11,172 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_rule_engine.notifications.prita
2021-08-21 23:45:11,197 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_rule_engine.notifications.prita
2021-08-21 23:45:11,199 [main] WARN  o.s.c.a.CommonAnnotationBeanPostProcessor - Destroy method on bean with name 'defaultTbRuleEngineConsumerService' threw a
n exception: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to clean channel shutdown; protocol method: #method<channel.close>(repl
y-code=200, reply-text=OK, class-id=0, method-id=0)
2021-08-21 23:45:11,202 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_core
2021-08-21 23:45:11,211 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_usage_stats
2021-08-21 23:45:11,219 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_ota_package
2021-08-21 23:45:11,227 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_core.notifications.prita
2021-08-21 23:45:11,234 [main] INFO  o.t.s.a.service.DefaultActorService - Stopping actor system.
2021-08-21 23:45:11,236 [main] INFO  o.t.s.a.service.DefaultActorService - Actor system stopped.
2021-08-21 23:45:11,237 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_transport.api.requests
2021-08-21 23:45:11,272 [sql-queue-0-attributes-15-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [Attributes] Queue polling was interrupted
2021-08-21 23:45:11,273 [sql-queue-1-attributes-16-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [Attributes] Queue polling was interrupted
2021-08-21 23:45:11,273 [sql-queue-2-attributes-17-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [Attributes] Queue polling was interrupted
2021-08-21 23:45:11,274 [sql-queue-3-attributes-18-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [Attributes] Queue polling was interrupted
2021-08-21 23:45:11,281 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_transport.notifications.prita
2021-08-21 23:45:11,285 [main] INFO  o.t.s.q.c.AbstractTbQueueConsumerTemplate - unsubscribe topic and stop consumer tb_transport.api.responses.prita
2021-08-21 23:45:11,316 [tb-queue-request-template-tb_transport.api.responses.prita-12-thread-1] WARN  o.t.s.q.c.DefaultTbQueueRequestTemplate - Failed to obt
ain and process responses from queue. Going to sleep 5139193ns
com.rabbitmq.client.AlreadyClosedException: channel is already closed due to clean channel shutdown; protocol method: #method<channel.close>(reply-code=200, r
eply-text=OK, class-id=0, method-id=0)
        at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:228)
        at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:371)
        at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:365)
        at com.rabbitmq.client.impl.ChannelN.basicAck(ChannelN.java:1175)
        at org.thingsboard.server.queue.rabbitmq.TbRabbitMqConsumerTemplate.doCommit(TbRabbitMqConsumerTemplate.java:98)
        at org.thingsboard.server.queue.common.AbstractTbQueueConsumerTemplate.commit(AbstractTbQueueConsumerTemplate.java:157)
        at org.thingsboard.server.queue.common.DefaultTbQueueRequestTemplate.fetchAndProcessResponses(DefaultTbQueueRequestTemplate.java:116)
        at org.thingsboard.server.queue.common.DefaultTbQueueRequestTemplate.mainLoop(DefaultTbQueueRequestTemplate.java:101)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
2021-08-21 23:45:11,346 [sql-queue-0-ts latest-6-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS Latest] Queue polling was interrupted
2021-08-21 23:45:11,347 [sql-queue-1-ts latest-7-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS Latest] Queue polling was interrupted
2021-08-21 23:45:11,347 [sql-queue-2-ts latest-8-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS Latest] Queue polling was interrupted
2021-08-21 23:45:11,348 [sql-queue-3-ts latest-9-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS Latest] Queue polling was interrupted
2021-08-21 23:45:11,348 [sql-queue-0-ts-2-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS] Queue polling was interrupted
2021-08-21 23:45:11,349 [sql-queue-1-ts-3-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS] Queue polling was interrupted
2021-08-21 23:45:11,349 [sql-queue-2-ts-4-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS] Queue polling was interrupted
2021-08-21 23:45:11,351 [sql-queue-3-ts-5-thread-1] INFO  o.t.s.dao.sql.TbSqlBlockingQueue - [TS] Queue polling was interrupted
2021-08-21 23:45:11,378 [main] INFO  o.s.o.j.LocalContainerEntityManagerFactoryBean - Closing JPA EntityManagerFactory for persistence unit 'default'
2021-08-21 23:45:11,389 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown initiated...
2021-08-21 23:45:11,420 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown completed.
2021-08-21 23:45:11,456 [main] INFO  o.a.catalina.core.StandardService - Stopping service [Tomcat]
2021-08-21 23:45:11,472 [main] WARN  o.a.c.loader.WebappClassLoaderBase - The web application [ROOT] appears to have started a thread named [AMQP Connection 127.0.0.1:5672] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.base@11.0.11/java.net.SocketInputStream.socketRead0(Native Method)
 java.base@11.0.11/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
 java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:168)
 java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:140)
 java.base@11.0.11/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
 java.base@11.0.11/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
 java.base@11.0.11/java.io.DataInputStream.readUnsignedByte(DataInputStream.java:293)
 com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
 com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164)
 com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:602)
 java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
2021-08-21 23:45:11,474 [main] WARN  o.a.c.loader.WebappClassLoaderBase - The web application [ROOT] appears to have started a thread named [pool-2-thread-38] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
 java.base@11.0.11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
 java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
 java.base@11.0.11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
 java.base@11.0.11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
 java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
 java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
 java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
2021-08-21 23:45:11,550 [main] INFO  o.s.b.a.l.ConditionEvaluationReportLoggingListener -

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-08-21 23:45:11,611 [main] ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'MqttTransportService': Invocation of init method failed; nested exception is java.net.BindException: Address already in use
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:415)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1791)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1247)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1236)
        at org.thingsboard.server.ThingsboardServerApplication.main(ThingsboardServerApplication.java:38)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.net.BindException: Address already in use
        at java.base/sun.nio.ch.Net.bind0(Native Method)
        at java.base/sun.nio.ch.Net.bind(Net.java:455)
        at java.base/sun.nio.ch.Net.bind(Net.java:447)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
        at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
        at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
        at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
        at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
        at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

Additional context Here is my thingsboard.conf

export JAVA_OPTS="$JAVA_OPTS -Dplatform=deb -Dinstall.data_dir=/usr/share/thingsboard/data"
export JAVA_OPTS="$JAVA_OPTS -Xlog:gc*,heap*,age*,safepoint=debug:file=/var/log/thingsboard/gc.log:time,uptime,level,tags:filecount=10,filesize=10M"
export JAVA_OPTS="$JAVA_OPTS -XX:+IgnoreUnrecognizedVMOptions -XX:+HeapDumpOnOutOfMemoryError"
export JAVA_OPTS="$JAVA_OPTS -XX:-UseBiasedLocking -XX:+UseTLAB -XX:+ResizeTLAB -XX:+PerfDisableSharedMem -XX:+UseCondCardMark"
export JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC -XX:MaxGCPauseMillis=500 -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled -XX:MaxTenuringThreshold=10"
export LOG_FILENAME=thingsboard.out
export LOADER_PATH=/usr/share/thingsboard/conf,/usr/share/thingsboard/extensions
export SQL_DATA_FOLDER=/usr/share/thingsboard/data/sql

export HTTP_BIND_ADDRESS=127.0.0.1
export HTTP_BIND_PORT=8080

# DB Configuration
export DATABASE_ENTITIES_TYPE=sql
export DATABASE_TS_TYPE=sql
export SPRING_JPA_DATABASE_PLATFORM=org.hibernate.dialect.PostgreSQLDialect
export SPRING_DRIVER_CLASS_NAME=org.postgresql.Driver
export SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:5432/thingsboard
export SPRING_DATASOURCE_USERNAME=postgres
export SPRING_DATASOURCE_PASSWORD=password
export SPRING_DATASOURCE_MAXIMUM_POOL_SIZE=5
# Specify partitioning size for timestamp key-value storage. Allowed values: DAYS, MONTHS, YEARS, INDEFINITE.
export SQL_POSTGRES_TS_KV_PARTITIONING=MONTHS

export TB_QUEUE_TYPE=rabbitmq
export TB_QUEUE_RABBIT_MQ_USERNAME=thingsboard
export TB_QUEUE_RABBIT_MQ_PASSWORD=password
export TB_QUEUE_RABBIT_MQ_HOST=localhost
export TB_QUEUE_RABBIT_MQ_PORT=5672

export MQTT_SSL_ENABLED=true
export MQTT_BIND_PORT=8883
export MQTT_SSL_KEY_STORE_TYPE=PKCS12
export MQTT_SSL_KEY_STORE=mqttserver.p12
export MQTT_SSL_KEY_STORE_PASSWORD=password
export MQTT_SSL_KEY_PASSWORD=password
export MQTT_SSL_PROTOCOL=TLSv1.2

# Update ThingsBoard memory usage and restrict it to 256MB in /etc/thingsboard/conf/thingsboard.conf
export JAVA_OPTS="$JAVA_OPTS -Xms256M -Xmx256M"
TKRishikesh commented 3 years ago

Can you try removing/commenting out the below line from your thingsboard.conf ?

export MQTT_BIND_PORT=8883

This will start the server. But I am not sure if your secure MQTT will work after this change. But I can see that even without the above configuration, the thingsboard process is getting attached to the ports 1883 and 8883, so you can give it a try.

Longer Version

I am facing a similar issue in Thingsboard 3.3 This was on a fresh instance of thingsboard, all was fine until I tried to install a self-signed certificate to secure the MQTT connection by following the steps available here

Stack trace:

ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'MqttTransportService': Invocation of init method failed; nested exception is java.net.BindException: Address already in use
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:415)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1791)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1247)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1236)
    at org.thingsboard.server.ThingsboardServerApplication.main(ThingsboardServerApplication.java:38)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
    at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.net.BindException: Address already in use
    at java.base/sun.nio.ch.Net.bind0(Native Method)
    at java.base/sun.nio.ch.Net.bind(Net.java:455)
    at java.base/sun.nio.ch.Net.bind(Net.java:447)
    at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
    at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
    at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
    at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
    at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
    at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
    at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:829)

I had configured to use the port number 8883 as suggested in the documentation. I can also verify that there is no process listening to that port before starting the thignsboard service

But after the above error log, I can see the process created by Thingsboard still attached to that port

$  lsof -i :8883
COMMAND   PID        USER               FD     TYPE   DEVICE      SIZE/OFF    NODE NAME
java   12046.   thingsboard   65u     IPv6    65043     0t0          TCP *:8883 (LISTEN)

Then only I figured out that thingboard is attaching to the ports 1883 and 8883 by default without any external configuration. Hint : v3.3 Release notes Ability to launch MQTT and MQTTS simultaneously in one transport instance

It looks like the same process tried to attach the port again which resulted in the server crashing, but strangely if the MQTT_BIND_PORT is set to 1883 there is no problem in server startup.

FYI : I am still struggling to set up a secure MQTT

iwas108 commented 3 years ago

Commenting out,

export MQTT_BIND_PORT=8883

Fixed the problem, thingsboard service started successfuly with SSL enabled.

Closing this issue, thankyou verymuch!

icebob commented 2 years ago

Thanks @iwas108, it was the solution. But later I found the instruction in the upgrade guide:

export MQTT_SSL_ENABLED=true
export MQTT_SSL_BIND_ADDRESS=0.0.0.0
export MQTT_SSL_BIND_PORT=8883

export MQTT_BIND_ADDRESS=0.0.0.0
export MQTT_BIND_PORT=1883

So change from MQTT_BIND_PORT=8883 to MQTT_SSL_BIND_PORT=8883 is necessary at upgrading to 3.3.x