search

apache / rocketmq-docker

Apache RocketMQ Docker
https://rocketmq.apache.org/
Apache License 2.0
683 stars 249 forks source link

求助 单机通过docker-compose部署5.1.3 #88

Open cyjaysong opened 1 year ago

cyjaysong commented 1 year ago

使用 docker-compose 在docker 部署5.1.3的namesrv 和 broker 用于开发测试，都启动正常了，但是sdk愣是连接不上，搜了一下，需要启动代理，但是在broker启动时添加了 --enable-proxy 还是不能正常连接，有朋友可以指导一下吗？或者有朋友有现成的docker-compose.yml文件么

francisoliverlee commented 1 year ago

can you share you docker-compose.yaml ?

cyjaysong commented 1 year ago

@francisoliverlee 

version: "3"
services:
  #Service for nameserver
  namesrv:
    container_name: RocketMQ-namesrv
    image: apache/rocketmq:5.1.3
    command: sh mqnamesrv
    restart: always
    ports:
      - 9876:9876
    volumes:
      - ./namesrv/logs:/home/rocketmq/logs

  #Service for broker
  broker:
    container_name: RocketMQ-broker
    links:
      - namesrv
    image: apache/rocketmq:5.1.3
    command: sh mqbroker --enable-proxy -c /opt/rocketmq/conf/broker.conf
    restart: always
    ports:
      - 8080:8080
      - 10909:10909
      - 10911:10911
      - 10912:10912
    environment:
      - NAMESRV_ADDR=namesrv:9876
    volumes:
      - ./broker/logs:/home/rocketmq/logs
      - ./broker/store:/home/rocketmq/store
      - ./broker/conf:/opt/rocketmq/conf
huangzulin commented 1 year ago

set broker dir permission @CyJaySong

this issues

cyjaysong commented 1 year ago

set broker dir permission @CyJaySong

this issues

broker 正常启动了的只是，客户端无法链接

peng-huang-ch commented 1 year ago

docker-compose.yml

version: "3"
services:
  #Service for nameserver
  rmqnamesrv:
    image: apache/rocketmq:5.1.3
    container_name: rmqnamesrv
    ports:
      - 9876:9876
    environment:
      JAVA_OPT_EXT: "-server -Xms512M -Xmx512M -Xmn128m"
    volumes:
      - ./data/namesrv/logs:/home/rocketmq/logs
    command: sh mqnamesrv
    restart: unless-stopped

  rmqbroker:
    image: apache/rocketmq:5.1.3
    container_name: rmqbroker
    depends_on:
      - rmqnamesrv
    ports:
      - 10909:10909
      - 10911:10911
      - 10912:10912
    environment:
      NAMESRV_ADDR: "rmqnamesrv:9876"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms512M -Xmx512M -Xmn128m"
    volumes:
      - ./data/broker/logs:/home/rocketmq/logs
      - ./data/broker/store:/home/rocketmq/store
      - ./conf/broker.conf:/opt/conf/broker.conf
    command: sh mqbroker -c /opt/conf/broker.conf 

  rmqproxy:
    image: apache/rocketmq:5.1.3
    container_name: rmqproxy
    depends_on:
      - rmqnamesrv
    ports:
      - 8081:8081
    environment:
      NAMESRV_ADDR: "rmqnamesrv:9876"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms512M -Xmx512M -Xmn128m"
    volumes:
      - ./data/proxy/logs:/home/rocketmq/logs
      - ./data/proxy/store:/home/rocketmq/store
      - ./conf/proxy.conf:/opt/conf/proxy.conf
    command: sh mqbroker -c /opt/conf/proxy.conf --enable-proxy

  dashbord:
    image: apacherocketmq/rocketmq-dashboard:1.0.0
    ports:
      - 18080:8080
    environment:
        JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Duser.timezone='Asia/Shanghai'"
    depends_on:
      - rmqnamesrv

broker.conf & proxy.conf

# 所属集群名字
brokerClusterName=rmqnamesrv

# broker 名字，注意此处不同的配置文件填写的不一样，如果在 broker-a.properties 使用: broker-a,
# 在 broker-b.properties 使用: broker-b
brokerName=broker-a

# 0 表示 Master，> 0 表示 Slave
brokerId=0

# nameServer地址，分号分割
#namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
namesrvAddr=rmqnamesrv:9876

# 启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句 producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.31.149

# 在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4

# 是否允许 Broker 自动创建 Topic，建议线下开启，线上关闭 ！！！这里仔细看是 false，false，false
autoCreateTopicEnable=true

# 是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true

# Broker 对外服务的监听端口
listenPort=10911

# 删除文件时间点，默认凌晨4点
deleteWhen=04

# 文件保留时间，默认48小时
fileReservedTime=120

# commitLog 每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

# ConsumeQueue 每个文件默认存 30W 条，根据业务情况调整
mapedFileSizeConsumeQueue=300000

# destroyMapedFileIntervalForcibly=120000
# redeleteHangedFileInterval=120000
# 检测物理文件磁盘空间
diskMaxUsedSpaceRatio=99
# 存储路径
# storePathRootDir=/home/ztztdata/rocketmq-all-4.1.0-incubating/store
# commitLog 存储路径
# storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
# 消费队列存储
# storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
# 消息索引存储路径
# storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
# checkpoint 文件存储路径
# storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
# abort 文件存储路径
# abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
# 限制的消息大小
maxMessageSize=65536

# flushCommitLogLeastPages=4
# flushConsumeQueueLeastPages=2
# flushCommitLogThoroughInterval=10000
# flushConsumeQueueThoroughInterval=60000

# Broker 的角色
# - ASYNC_MASTER 异步复制Master
# - SYNC_MASTER 同步双写Master
# - SLAVE
brokerRole=ASYNC_MASTER

# 刷盘方式
# - ASYNC_FLUSH 异步刷盘
# - SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

# 发消息线程池数量
# sendMessageThreadPoolNums=128
# 拉消息线程池数量
# pullMessageThreadPoolNums=12

messageDelayLevel=1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h

启动日志：

 docker compose logs --tail=1000 rmqnamesrv 

rmqnamesrv  | OpenJDK 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
rmqnamesrv  | OpenJDK 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
rmqnamesrv  | The Name Server boot success. serializeType=JSON, address 0.0.0.0:9876

docker compose logs --tail=1000 rmqproxy 

rmqproxy  | OpenJDK 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
rmqproxy  | OpenJDK 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
rmqproxy  | Thu Sep 14 12:24:26 UTC 2023 rocketmq-proxy startup successfully

使用 dashbord 可以正常的发送消息，但是使用 producer 会报错

current message type not match with topic accept message types.

似乎 RocketMQ 5.0 引入了 TopicMessageType 的概念，但是调用了 updateTopic 好像还是有问题

sh mqadmin updateTopic -n 127.0.0.1:9876 -t txs  -c rmqnamesrv -a +message.type=NORMAL

希望有人能帮忙解决下，😄

zhyblack commented 1 year ago

我使用的官方例子也出现了这个问题，解决了 1、play-docker.sh + sh mqbroker --enable-proxy 2、-p 8081:8081 说是 --enable-proxy 会自动启用 8081端口 3、我使用的是阿里云 所以安全组需要放行 8081 然后就搞定了 Consume message successfully, messageId={}7F00000100065910E4409199EB2F0000 程序生产者&后台发送消息都没问题

huiliaoning commented 10 months ago

Reference

遇到了同样的问题

mi1chstrasse commented 9 months ago

连接的服务应该是 8081 端口

leizhiyuan commented 9 months ago

8081 是grpc 的端口，需要放开。 private Integer grpcServerPort = 8081;

huangsunting commented 7 months ago

官方文档比较坑，甚至没有docker的部署示例。。。 rocketmq5.x引入了proxy，需要开放一下端口，不然访问不了。

1.先简单安装一下，8081是grpc访问端口，8080是remoting访问（如果需要） docker run -d --name rmqbroker --net rmq_net -p 10911:10911 -p 10909:10909 -p 8081:8081 --privileged=true apache/rocketmq:5.2.0 sh mqbroker -n rmqnamesrv:9876 --enable-proxy

2.拷贝文件后，删除容器 docker cp rmqbroker:/home/rocketmq/rocketmq-5.2.0/conf /你的目录/docker/rocketmq/broker/conf

3.重新安装，endpoints=localhost:8081 docker run -d --name rmqbroker --net rmq_net -p 10911:10911 -p 10909:10909 -p 8081:8081 -p 8080:8080 --privileged=true -v 你的目录/docker/rocketmq/broker/logs:/home/rocketmq/logs -v 你的目录/docker/rocketmq/broker/store:/home/rocketmq/store -v 你的目录/docker/rocketmq/broker/conf:/home/rocketmq/rocketmq-5.2.0/conf -e "MAX_HEAP_SIZE=1024M" apache/rocketmq:5.2.0 sh mqbroker -n rmqnamesrv:9876 --enable-proxy -c /home/rocketmq/rocketmq-5.2.0/conf/broker.conf

jasonjiang9527 commented 1 month ago

官方文档比较坑，甚至没有docker的部署示例。。。 rocketmq5.x引入了proxy，需要开放一下端口，不然访问不了。

1.先简单安装一下，8081是grpc访问端口，8080是remoting访问（如果需要） docker run -d --name rmqbroker --net rmq_net -p 10911:10911 -p 10909:10909 -p 8081:8081 --privileged=true apache/rocketmq:5.2.0 sh mqbroker -n rmqnamesrv:9876 --enable-proxy

2.拷贝文件后，删除容器 docker cp rmqbroker:/home/rocketmq/rocketmq-5.2.0/conf /你的目录/docker/rocketmq/broker/conf

3.重新安装，endpoints=localhost:8081 docker run -d --name rmqbroker --net rmq_net -p 10911:10911 -p 10909:10909 -p 8081:8081 -p 8080:8080 --privileged=true -v 你的目录/docker/rocketmq/broker/logs:/home/rocketmq/logs -v 你的目录/docker/rocketmq/broker/store:/home/rocketmq/store -v 你的目录/docker/rocketmq/broker/conf:/home/rocketmq/rocketmq-5.2.0/conf -e "MAX_HEAP_SIZE=1024M" apache/rocketmq:5.2.0 sh mqbroker -n rmqnamesrv:9876 --enable-proxy -c /home/rocketmq/rocketmq-5.2.0/conf/broker.conf

work for me ,其实就是不要挂载broker.conf,因为有权限问题.启动容器后用docker desktop进容器直接加127的配置重启就好了