apache / hertzbeat

Apache HertzBeat(incubating) is a real-time monitoring system with agentless, performance cluster, prometheus-compatible, custom monitoring and status page building capabilities.
https://hertzbeat.apache.org/
Apache License 2.0
5.23k stars 911 forks source link

[BUG] Found more than one migration with version 160 (Docker Compose mysql-victoria version) #2206

Open andyg2 opened 1 week ago

andyg2 commented 1 week ago

Is there an existing issue for this?

Current Behavior

Upon running docker compose up -d I'm getting an error in 'compose-hertzbeat' stating there's multiple migration files.

/opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/h2/V160__update_column.sql (SQL) /opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/mysql/V160__update_column.sql (SQL)

Despite my best efforts to locate these I haven't been able to find them or experiment with removing one of them.

Expected Behavior

Hertzbeat handles the multiple files gracefully, or improved documentation for remediation.

Steps To Reproduce

  1. Put mysql connector mysql-connector-java-8.0.25.jar into ./ext-lib
  2. Create a volume ./ext-lib:/opt/hertzbeat/ext-lib
  3. Remove docker container and image
  4. run docker compose up -d

Environment

HertzBeat version(s): 1.6.0
Windows 10 Pro Version 22H2 (19045.4529)
Docker Desktop 4.31.1 (153621) (newest version available)

Debug logs

2024-07-04 22:25:27.143 [main] ERROR org.springframework.boot.SpringApplication Line:851 - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Found more than one migration with version 160
Offenders:
-> /opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/h2/V160__update_column.sql (SQL)
-> /opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/mysql/V160__update_column.sql (SQL)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1786)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:522)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1231)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:949)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:624)
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:334)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
    at org.apache.hertzbeat.manager.Manager.main(Manager.java:44)
Caused by: org.flywaydb.core.api.FlywayException: Found more than one migration with version 160
Offenders:
-> /opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/h2/V160__update_column.sql (SQL)
-> /opt/hertzbeat/file:/opt/hertzbeat/apache-hertzbeat-1.6.0.jar!/db/migration/mysql/V160__update_column.sql (SQL)
    at org.flywaydb.core.internal.resolver.CompositeMigrationResolver.checkForIncompatibilities(CompositeMigrationResolver.java:94)
    at org.flywaydb.core.internal.resolver.CompositeMigrationResolver.doFindAvailableMigrations(CompositeMigrationResolver.java:127)
    at org.flywaydb.core.internal.resolver.CompositeMigrationResolver.resolveMigrations(CompositeMigrationResolver.java:114)
    at org.flywaydb.core.internal.resolver.CompositeMigrationResolver.resolveMigrations(CompositeMigrationResolver.java:120)
    at org.flywaydb.core.internal.info.MigrationInfoServiceImpl.refresh(MigrationInfoServiceImpl.java:86)
    at org.flywaydb.core.internal.command.DbValidate.lambda$validate$0(DbValidate.java:112)
    at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:59)
    at org.flywaydb.core.internal.command.DbValidate.validate(DbValidate.java:105)
    at org.flywaydb.core.Flyway.doValidate(Flyway.java:418)
    at org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:162)
    at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:209)
    at org.flywaydb.core.Flyway.migrate(Flyway.java:151)
    at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1833)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782)
    ... 18 common frames omitted

Anything else?

image

version: "3.7"

networks:
  hertzbeat:
    driver: bridge

services:
  mysql:
    image: mysql:8
    container_name: compose-mysql
    hostname: mysql
    restart: always
    healthcheck:
      test: ['CMD-SHELL', 'mysqladmin ping -h 127.0.0.1 --silent']
      interval: 3s
      retries: 5
      start_period: 3m
    ports:
      - "3306"
    environment:
      TZ: Asia/Shanghai
      MYSQL_ROOT_PASSWORD: 123456
    volumes:
      - ./dbdata/mysqldata:/var/lib/mysql/
      - ./conf/sql:/docker-entrypoint-initdb.d/
    networks:
      - hertzbeat

  victoria-metrics:
    image: victoriametrics/victoria-metrics:v1.95.1
    container_name: compose-victoria-metrics
    hostname: victoria-metrics
    restart: always
    healthcheck:
      test: ["CMD", "wget", "-q", "-O", "-", "http://victoria-metrics:8428/-/healthy"]
      interval: 10s
      retries: 5
      timeout: 5s
      start_period: 30s
    environment:
      TZ: Asia/Shanghai
    ports:
      - "8428"
    volumes:
      - ./dbdata/tsdbdata:/victoria-metrics-data
    networks:
      - hertzbeat

  hertzbeat:
    image: tancloud/hertzbeat
    container_name: compose-hertzbeat
    hostname: hertzbeat
    restart: always
    environment:
      TZ: Asia/Shanghai
      LANG: zh_CN.UTF-8
    depends_on:
      mysql:
        condition: service_healthy
      victoria-metrics:
        condition: service_healthy
    volumes:
      - ./ext-lib:/opt/hertzbeat/ext-lib
      - ./conf/application.yml:/opt/hertzbeat/config/application.yml
      - ./conf/sureness.yml:/opt/hertzbeat/config/sureness.yml
      - ./log:/opt/hertzbeat/logs
      - ./conf/enable-mysql-native-password.cnf:/etc/mysql/conf.d/enable-mysql-native-password.cnf
    ports:
      - "1157:1157"
      - "1158:1158"
    networks:
      - hertzbeat
tomsun28 commented 1 week ago

hi, got it, we will try to reproduce this. Are you deploying a completely new version, or upgrading from an older version?

andyg2 commented 1 week ago

Are you deploying a completely new version, or upgrading from an older version?

It's a completely new version.


Side note - possibly related - maybe expected: I've logged in to mysql and the hertzbeat database is empty (no tables)

sh-5.1# mysql -uroot -p123456; Server version: 8.4.1 MySQL Community Server - GPL mysql> show databases;

+--------------------+
| Database           |
+--------------------+
| hertzbeat          |
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+

mysql> use hertzbeat;

Database changed

mysql> show tables;

Empty set (0.00 sec)
gjjjj0101 commented 1 week ago

Hi, Can you provide detail content of ./conf/application.yml. I tried to reproduce the bug in your steps, but failed to reproduce. I guess your application.yml miss the flyway configuration. So I deleted the flyway configuration in the latest application.yml and successfully reproduced the bug you mentioned. I hope this helps you. You can provide more information so we can help you solve this problem further.

andyg2 commented 1 week ago

Hi, Can you provide detail content of ./conf/application.yml. I tried to reproduce the bug in your steps, but failed to reproduce. I guess your application.yml miss the flyway configuration. So I deleted the flyway configuration in the latest application.yml and successfully reproduced the bug you mentioned. I hope this helps you. You can provide more information so we can help you solve this problem further.

Sure, it's was 100% untouched from the 1.6.0 release. I see in the master branch there's numerous changes including flyway. Should I be cloning the master branch instead of using the releases?

Currently though I've added the flyway config from the master branch and HertzBeat seems to have started up ok and I can at least log in.

...
...
...
  mail:
    # Attention: this is mail server address.
    # 请注意此为邮件服务器地址:qq邮箱为 smtp.qq.com qq企业邮箱为 smtp.exmail.qq.com
    host: smtp.qq.com
    username: example@qq.com
    # Attention: this is not email account password, this requires an email authorization code
    # 请注意此非邮箱账户密码 此需填写邮箱授权码
    password: xxqzvuqbnqvbbdac
    port: 465
    default-encoding: UTF-8
    properties:
      mail:
        smtp:
          socketFactoryClass: javax.net.ssl.SSLSocketFactory
          ssl:
            enable: true

  # - Added flyway config (from master)
  flyway:
    enabled: true
    clean-disabled: true
    baseline-on-migrate: true
    baseline-version: 1
    locations:
      - classpath:db/migration/{vendor}
  # // Added from master branch

warehouse:
  store:
...
...
...

There were some warnings output by HertzBeat though (below) - are these anything to be concerned about? If not feel free to close this issue, your direction was very much appreciated.

  _   _           _       ____             _
 | | | | ___ _ __| |_ ___| __ )  ___  __ _| |_
 | |_| |/ _ \ '__| __|_  /  _ \ / _ \/ _` | __|        Profile: prod
 |  _  |  __/ |  | |_ / /| |_) |  __/ (_| | |_         Name: hertzbeat10 Port: 1157 Pid: 10
 |_| |_|\___|_|   \__/___|____/ \___|\__,_|\__|        https://hertzbeat.apache.org/ 

[EL Info]: 2024-07-05 17:26:22.513--ServerSession(1688861863)--EclipseLink, version: Eclipse Persistence Services - 4.0.2.v202306161219
[EL Warning]: 2024-07-05 17:26:22.929--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Key column 'alertDefineId' doesn't exist in table
Error Code: 1072
Call: CREATE INDEX index_alert_define_monitor ON hzb_alert_define_monitor_bind (alertDefineId)
Query: DataModifyQuery(sql="CREATE INDEX index_alert_define_monitor ON hzb_alert_define_monitor_bind (alertDefineId)")
[EL Warning]: 2024-07-05 17:26:23.035--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'index_incident_component'
Error Code: 1061
Call: CREATE INDEX index_incident_component ON hzb_status_page_incident_component_bind (component_id)
Query: DataModifyQuery(sql="CREATE INDEX index_incident_component ON hzb_status_page_incident_component_bind (component_id)")
[EL Warning]: 2024-07-05 17:26:23.214--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Key column 'monitorId' doesn't exist in table
Error Code: 1072
Call: CREATE INDEX INDEX_hzb_param_monitorId ON hzb_param (monitorId)
Query: DataModifyQuery(sql="CREATE INDEX INDEX_hzb_param_monitorId ON hzb_param (monitorId)")
[EL Warning]: 2024-07-05 17:26:23.297--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'index_tag_monitor'
Error Code: 1061
Call: CREATE INDEX index_tag_monitor ON hzb_tag_monitor_bind (monitor_id)
Query: DataModifyQuery(sql="CREATE INDEX index_tag_monitor ON hzb_tag_monitor_bind (monitor_id)")
[EL Warning]: 2024-07-05 17:26:23.389--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'index_collector_monitor'
Error Code: 1061
Call: CREATE INDEX index_collector_monitor ON hzb_collector_monitor_bind (monitor_id)
Query: DataModifyQuery(sql="CREATE INDEX index_collector_monitor ON hzb_collector_monitor_bind (monitor_id)")
[EL Warning]: 2024-07-05 17:26:23.549--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'monitor_query_index'
Error Code: 1061
Call: CREATE INDEX monitor_query_index ON hzb_monitor (host)
Query: DataModifyQuery(sql="CREATE INDEX monitor_query_index ON hzb_monitor (host)")
[EL Warning]: 2024-07-05 17:26:23.755--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Key column 'monitorId' doesn't exist in table
Error Code: 1072
Call: CREATE INDEX push_query_index ON hzb_push_metrics (monitorId)
Query: DataModifyQuery(sql="CREATE INDEX push_query_index ON hzb_push_metrics (monitorId)")
[EL Warning]: 2024-07-05 17:26:23.796--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Key column 'monitorId' doesn't exist in table
Error Code: 1072
Call: CREATE INDEX history_query_index ON hzb_history (monitorId)
Query: DataModifyQuery(sql="CREATE INDEX history_query_index ON hzb_history (monitorId)")
[EL Warning]: 2024-07-05 17:26:23.825--ServerSession(1688861863)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.2.v202306161219): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: Key column 'monitorId' doesn't exist in table
Error Code: 1072
Call: ALTER TABLE hzb_param ADD CONSTRAINT UNQ_hzb_param_0 UNIQUE (monitorId, field)
Query: DataModifyQuery(sql="ALTER TABLE hzb_param ADD CONSTRAINT UNQ_hzb_param_0 UNIQUE (monitorId, field)")
gjjjj0101 commented 1 week ago

Yes, I find this problem in v1.6.0. The configuration file for starting heartzbeat via docker compose has problems. This problem has been fixed in the current master branch. You can cd to script/docker-compose/hertzbeat-mysql-victoria-metrics directory, configure conf/application.yml in this path and add external jar to ext-lib. Then execute docker compose up -d to start the service.