search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
984 stars 76 forks source link

Controller start fail #412

Open youzuoyong opened 4 months ago

youzuoyong commented 4 months ago

LINSTOR, Module Controller Version: 1.25.1 (a9dc5e08a0c66ae505db5b70bba9ab78bddec9a6) Build time: 2024-06-21T07:31:30+00:00 Java Version: 11 Java VM: Debian, Version 11.0.23+9-post-Debian-1deb11u1 Operating system: Linux, Version 5.15.0-97-generic Environment: aarch64, 64 processors, 30688 MiB memory reserved for allocations

System components initialization in progress

Loading configuration file "/etc/linstor/linstor.toml" 08:15:25.101 [main] INFO LINSTOR/Controller - SYSTEM - ErrorReporter DB version 1 found. 08:15:25.106 [main] INFO LINSTOR/Controller - SYSTEM - Log directory set to: '/var/log/linstor-controller' 08:15:25.180 [main] INFO LINSTOR/Controller - SYSTEM - Database type is SQL 08:15:25.181 [Main] INFO LINSTOR/Controller - SYSTEM - Loading API classes started. 08:15:26.289 [Main] INFO LINSTOR/Controller - SYSTEM - API classes loading finished: 1107ms 08:15:26.290 [Main] INFO LINSTOR/Controller - SYSTEM - Dependency injection started. 08:15:26.331 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.modularcrypto.FipsCryptoModule" 08:15:26.332 [Main] INFO LINSTOR/Controller - SYSTEM - Extension module "com.linbit.linstor.modularcrypto.FipsCryptoModule" is not installed 08:15:26.333 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.modularcrypto.JclCryptoModule" 08:15:26.363 [Main] INFO LINSTOR/Controller - SYSTEM - Dynamic load of extension module "com.linbit.linstor.modularcrypto.JclCryptoModule" was successful 08:15:26.364 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" 08:15:26.366 [Main] INFO LINSTOR/Controller - SYSTEM - Dynamic load of extension module "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" was successful 08:15:28.937 [Main] INFO LINSTOR/Controller - SYSTEM - Dependency injection finished: 2645ms 08:15:28.940 [Main] INFO LINSTOR/Controller - SYSTEM - Cryptography provider: Using default cryptography module 08:15:29.780 [Main] INFO LINSTOR/Controller - SYSTEM - Initializing authentication subsystem 08:15:30.527 [Main] INFO LINSTOR/Controller - SYSTEM - SpaceTrackingService: Instance added as a system service 08:15:30.531 [Main] INFO LINSTOR/Controller - SYSTEM - Starting service instance 'TimerEventService' of type TimerEventService 08:15:30.532 [Main] INFO LINSTOR/Controller - SYSTEM - Initializing the database connection pool 08:15:30.533 [Main] INFO LINSTOR/Controller - SYSTEM - SQL database connection URL is "jdbc:h2:/var/lib/linstor/linstordb" 08:15:30.944 [Main] INFO LINSTOR/Controller - SYSTEM - SQL database is H2 08:15:31.201 [Main] INFO org.flywaydb.core.internal.license.VersionPrinter - Flyway Community Edition 7.15.0 by Redgate 08:15:31.203 [Main] INFO org.flywaydb.core.internal.database.base.BaseDatabaseType - Database: jdbc:h2:/var/lib/linstor/linstordb (H2 1.4) 08:15:31.345 [Main] INFO org.flywaydb.core.internal.command.DbValidate - Successfully validated 74 migrations (execution time 00:00.087s) 08:15:31.434 [Main] INFO org.flywaydb.core.internal.command.DbMigrate - Current version of schema "LINSTOR": 2019.11.12.01.01 08:15:31.435 [Main] WARN org.flywaydb.core.internal.command.DbMigrate - outOfOrder mode is active. Migration of schema "LINSTOR" may not be reproducible. 08:15:31.441 [Main] INFO org.flywaydb.core.internal.command.DbMigrate - Migrating schema "LINSTOR" to version "2019.11.21.01.01 - Unifies resources and snapshots" 08:15:31.468 [Main] ERROR org.flywaydb.core.internal.command.DbMigrate - Migration of schema "LINSTOR" to version "2019.11.21.01.01 - Unifies resources and snapshots" failed! Please restore backups and roll back database and code! 08:15:31.610 [Main] ERROR LINSTOR/Controller - SYSTEM - Database initialization error [Report number 6685089B-00000-000000]

08:15:31.615 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutdown in progress 08:15:31.618 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'EbsStatusPoll' of type EbsStatusPoll 08:15:31.619 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'EbsStatusPoll' to complete shutdown 08:15:31.620 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'ScheduleBackupService' of type ScheduleBackupService 08:15:31.620 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'ScheduleBackupService' to complete shutdown 08:15:31.621 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'SpaceTrackingService' of type SpaceTrackingService 08:15:31.621 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'SpaceTrackingService' to complete shutdown 08:15:31.622 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'TaskScheduleService' of type TaskScheduleService 08:15:31.623 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'TaskScheduleService' to complete shutdown 08:15:31.624 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'DatabaseService' of type DatabaseService 08:15:31.634 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'DatabaseService' to complete shutdown 08:15:31.635 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'TimerEventService' of type TimerEventService 08:15:31.637 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'TimerEventService' to complete shutdown 08:15:31.637 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutdown complete time="2024-07-03T08:15:31Z" level=fatal msg="failed to run" err="exit status 20" time="2024-07-03T08:46:26Z" level=info msg="running k8s-await-election" version=refs/tags/v0.3.1 time="2024-07-03T08:46:26Z" level=info msg="not running with leader election" time="2024-07-03T08:46:26Z" level=info msg="starting command '/usr/bin/entry.sh' with arguments: '[startController]'" LINSTOR, Module Controller Version: 1.25.1 (a9dc5e08a0c66ae505db5b70bba9ab78bddec9a6) Build time: 2024-06-21T07:31:30+00:00 Java Version: 11 Java VM: Debian, Version 11.0.23+9-post-Debian-1deb11u1 Operating system: Linux, Version 5.15.0-97-generic Environment: aarch64, 64 processors, 30688 MiB memory reserved for allocations

System components initialization in progress

Loading configuration file "/etc/linstor/linstor.toml" 08:46:29.201 [main] INFO LINSTOR/Controller - SYSTEM - ErrorReporter DB version 1 found. 08:46:29.208 [main] INFO LINSTOR/Controller - SYSTEM - Log directory set to: '/var/log/linstor-controller' 08:46:29.284 [main] INFO LINSTOR/Controller - SYSTEM - Database type is SQL 08:46:29.285 [Main] INFO LINSTOR/Controller - SYSTEM - Loading API classes started. 08:46:30.300 [Main] INFO LINSTOR/Controller - SYSTEM - API classes loading finished: 1014ms 08:46:30.301 [Main] INFO LINSTOR/Controller - SYSTEM - Dependency injection started. 08:46:30.344 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.modularcrypto.FipsCryptoModule" 08:46:30.345 [Main] INFO LINSTOR/Controller - SYSTEM - Extension module "com.linbit.linstor.modularcrypto.FipsCryptoModule" is not installed 08:46:30.346 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.modularcrypto.JclCryptoModule" 08:46:30.372 [Main] INFO LINSTOR/Controller - SYSTEM - Dynamic load of extension module "com.linbit.linstor.modularcrypto.JclCryptoModule" was successful 08:46:30.373 [Main] INFO LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" 08:46:30.374 [Main] INFO LINSTOR/Controller - SYSTEM - Dynamic load of extension module "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" was successful 08:46:32.866 [Main] INFO LINSTOR/Controller - SYSTEM - Dependency injection finished: 2564ms 08:46:32.868 [Main] INFO LINSTOR/Controller - SYSTEM - Cryptography provider: Using default cryptography module 08:46:33.715 [Main] INFO LINSTOR/Controller - SYSTEM - Initializing authentication subsystem 08:46:34.473 [Main] INFO LINSTOR/Controller - SYSTEM - SpaceTrackingService: Instance added as a system service 08:46:34.478 [Main] INFO LINSTOR/Controller - SYSTEM - Starting service instance 'TimerEventService' of type TimerEventService 08:46:34.479 [Main] INFO LINSTOR/Controller - SYSTEM - Initializing the database connection pool 08:46:34.481 [Main] INFO LINSTOR/Controller - SYSTEM - SQL database connection URL is "jdbc:h2:/var/lib/linstor/linstordb" 08:46:34.858 [Main] INFO LINSTOR/Controller - SYSTEM - SQL database is H2 08:46:35.095 [Main] INFO org.flywaydb.core.internal.license.VersionPrinter - Flyway Community Edition 7.15.0 by Redgate 08:46:35.106 [Main] INFO org.flywaydb.core.internal.database.base.BaseDatabaseType - Database: jdbc:h2:/var/lib/linstor/linstordb (H2 1.4) 08:46:35.367 [Main] ERROR LINSTOR/Controller - SYSTEM - Database initialization error [Report number 66850FE4-00000-000000]

08:46:35.371 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutdown in progress 08:46:35.379 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'EbsStatusPoll' of type EbsStatusPoll 08:46:35.379 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'EbsStatusPoll' to complete shutdown 08:46:35.380 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'ScheduleBackupService' of type ScheduleBackupService 08:46:35.380 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'ScheduleBackupService' to complete shutdown 08:46:35.380 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'SpaceTrackingService' of type SpaceTrackingService 08:46:35.381 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'SpaceTrackingService' to complete shutdown 08:46:35.381 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'TaskScheduleService' of type TaskScheduleService 08:46:35.381 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'TaskScheduleService' to complete shutdown 08:46:35.382 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'DatabaseService' of type DatabaseService 08:46:35.385 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'DatabaseService' to complete shutdown 08:46:35.386 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutting down service instance 'TimerEventService' of type TimerEventService 08:46:35.387 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Waiting for service instance 'TimerEventService' to complete shutdown 08:46:35.388 [Thread-2] INFO LINSTOR/Controller - SYSTEM - Shutdown complete time="2024-07-03T08:46:35Z" level=fatal msg="failed to run" err="exit status 20"

ERROR REPORT 66850FE4-00000-000000

============================================================

Application: LINBIT? LINSTOR Module: Controller Version: 1.25.1 Build ID: a9dc5e08a0c66ae505db5b70bba9ab78bddec9a6 Build time: 2024-06-21T07:31:30+00:00 Error time: 2024-07-03 08:46:35 Node: node134

============================================================

Reported error:

Description: Database initialization error

Category: LinStorException Class name: SystemServiceStartException Class canonical name: com.linbit.SystemServiceStartException Generated at: Method 'initialize', Source file 'DbConnectionPoolInitializer.java', Line #69

Error message: Database initialization error

ErrorContext:

Call backtrace:

Method                                   Native Class:Line number
initialize                               N      com.linbit.linstor.dbcp.DbConnectionPoolInitializer:69
startSystemServices                      N      com.linbit.linstor.core.ApplicationLifecycleManager:87
start                                    N      com.linbit.linstor.core.Controller:361
main                                     N      com.linbit.linstor.core.Controller:609

Caused by:

Category: RuntimeException Class name: FlywayValidateException Class canonical name: org.flywaydb.core.api.exception.FlywayValidateException Generated at: Method 'execute', Source file 'Flyway.java', Line #177

Error message: Validate failed: Migrations have failed validation Detected failed migration to version 2019.11.21.01.01 (Unifies resources and snapshots). Please remove any half-completed changes then run repair to fix the schema history. Need more flexibility with validation rules? Learn more: https://rd.gt/3AbJUZE

Call backtrace:

Method                                   Native Class:Line number
execute                                  N      org.flywaydb.core.Flyway$1:177
execute                                  N      org.flywaydb.core.Flyway$1:170
execute                                  N      org.flywaydb.core.Flyway:586
migrate                                  N      org.flywaydb.core.Flyway:170
migrate                                  N      com.linbit.linstor.dbcp.DbConnectionPool:220
initialize                               N      com.linbit.linstor.dbcp.DbConnectionPoolInitializer:61
startSystemServices                      N      com.linbit.linstor.core.ApplicationLifecycleManager:87
start                                    N      com.linbit.linstor.core.Controller:361
main                                     N      com.linbit.linstor.core.Controller:609

END OF ERROR REPORT.

How will I repair this issue？