[BUG] creating DB unable to open database file: no such file or directory

[AzFlin7]

Bug Report

Error creating DB unable to open database file: no such file or directory

Description

I was going to run the CDK-node on the AWS ubuntu(22.04) instance, but I have faced some errors.

To Reproduce

Steps to reproduce the behavior:

1. Download cdk-v0.4.0-beta6-amd64.deb and Source code
2. Unzip and move usr/bin/cdk-node to cdk-0.4.0-beta6 folder.
3. "docker build -t cdk ."
4. Customize test.config.toml, test.genesis.json, docker-compose.yml in the test folder.
5. "docker compose up"

Expected behavior

Running cdk-sequence-sender, cdk-aggregator without any issue.

Screenshots

Environment (please complete the following information):

• OS: ubuntu22.04
• Version cdk-0.4.0-beta6

[goran-ethernal]

Can you share the cdk-node .toml config file?

[AzFlin7]

Yes. of course. The important links and addresses were hide.

[Common]
IsValidiumMode = false
ContractVersions = "banana"

[SequenceSender]
WaitPeriodSendSequence = "15s"
LastBatchVirtualizationTimeMaxWaitPeriod = "10s"
L1BlockTimestampMargin = "30s"
MaxTxSizeForL1 = 131072
L2Coinbase = "0x000"
PrivateKey = {Path = "./sequencer.keystore", Password = "xxx"}
SequencesTxFileName = "sequencesender.json"
GasOffset = 80000
WaitPeriodPurgeTxFile = "60m"
MaxPendingTx = 1
RPCURL = "https://rep-url.com"
GetBatchWaitInterval = "10s"
    [SequenceSender.StreamClient]
        Server = "xxx"
    [SequenceSender.EthTxManager]
        FrequencyToMonitorTxs = "1s"
        WaitTxToBeMined = "2m"
        GetReceiptMaxTime = "250ms"
        GetReceiptWaitInterval = "1s"
        PrivateKeys = [
            {Path = "./sequencer.keystore", Password = "xxx"},
        ]
        ForcedGas = 0
        GasPriceMarginFactor = 1
        MaxGasPriceLimit = 0
        StoragePath = "ethtxmanager.db"
        ReadPendingL1Txs = false
        SafeStatusL1NumberOfBlocks = 5
        FinalizedStatusL1NumberOfBlocks = 10
            [SequenceSender.EthTxManager.Etherman]
                URL = "http://xxx.com"
                MultiGasProvider = false
                L1ChainID = 11155111
                HTTPHeaders = []

[Aggregator]
Host = "0.0.0.0"
Port = 50081
RetryTime = "5s"
VerifyProofInterval = "10s"
TxProfitabilityCheckerType = "acceptall"
TxProfitabilityMinReward = "1.1"
ProofStatePollingInterval = "5s"
SenderAddress = "0x000"
CleanupLockedProofsInterval = "2m"
GeneratingProofCleanupThreshold = "10m"
BatchProofSanityCheckEnabled = true
ForkId = 9
GasOffset = 0
WitnessURL = "https://rpc-url.com"
UseL1BatchData = true
SettlementBackend = "l1"
AggLayerTxTimeout = "5m"
AggLayerURL = "https://agglayer.com"
MaxWitnessRetrievalWorkers = 2
SyncModeOnlyEnabled = false
UseFullWitness = false
SequencerPrivateKey = {}
    [Aggregator.DB]
        Name = "aggregator_db"
        User = "aggregator_user"
        Password = "aggregator_password"
        Host = "127.0.0.1"
        Port = "5432"
        EnableLog = false   
        MaxConns = 200
    [Aggregator.Log]
        Environment = "production" # "production" or "development"
        Level = "info"
        Outputs = ["stderr"]
    [Aggregator.StreamClient]
        Server = "x.x.x.x:6900"
    [Aggregator.EthTxManager]
        FrequencyToMonitorTxs = "1s"
        WaitTxToBeMined = "2m"
        GetReceiptMaxTime = "250ms"
        GetReceiptWaitInterval = "1s"
        PrivateKeys = [
            {Path = "/pk/aggregator.keystore", Password = "xxx"},
        ]
        ForcedGas = 0
        GasPriceMarginFactor = 1
        MaxGasPriceLimit = 0
        StoragePath = ""
        ReadPendingL1Txs = false
        SafeStatusL1NumberOfBlocks = 0
        FinalizedStatusL1NumberOfBlocks = 0
            [Aggregator.EthTxManager.Etherman]
                URL = ""
                L1ChainID = 11155111
                HTTPHeaders = []
    [Aggregator.Synchronizer]
        [Aggregator.Synchronizer.SQLDB]
            Name = "aggregator_syncer_db"
            User = "aggregator_syncer_db_user"
            Password = "aggregator_syncer_password"
            Host = "127.0.0.1"
            Port = "5432"
            EnableLog = false
            MaxConns = 10
        [Aggregator.Synchronizer.Synchronizer]
            SyncInterval = "10s"
            SyncChunkSize = 1000
            GenesisBlockNumber = 7076564
            SyncUpToBlock = "finalized"
            BlockFinality = "finalized"
        [Aggregator.Synchronizer.Etherman]
            [Aggregator.Synchronizer.Etherman.Validium]
                Enabled = false
                TrustedSequencerURL = ""
                DataSourcePriority = ["trusted", "external"]
                [Aggregator.Synchronizer.Etherman.Validium.Translator]
                    FullMatchRules = []

[goran-ethernal]

Can you add this part to your config just to see if it will start:

[ReorgDetectorL1]
DBPath = "/tmp/cdk/reorgdetectorl1.sqlite"
CheckReorgInterval = 2s

[AzFlin7]

Thanks for your kind advise. Btw, can you tell me where I can add this part in config.toml file?

[goran-ethernal]

On the same level as SequenceSender or Aggregator. For example:

[Common]
IsValidiumMode = false
ContractVersions = "banana"

[ReorgDetectorL1]
DBPath = "/tmp/cdk/reorgdetectorl1.sqlite"
CheckReorgInterval = 2s

[SequenceSender]

[AzFlin7]

@goran-ethernal I still have the same errors. Btw, when I use the original docker-compose.yml, there is app/cdk not found error. So I edited the docker-compose.yml as follows:

etworks:
  default:
    name: cdk

services:
  cdk-sequence-sender:
    container_name: cdk-sequence-sender
    restart: no
    image: cdk
    build: .
    volumes:
      - ./config.toml:/app/config.toml
      - ./genesis.json:/app/genesis.json
      - ./sequencer.keystore:/app/keystore/sequencer.keystore
      - ../bin/cdk-node:/app/cdk-node
    command:
      - "/bin/sh"
      - "-c"
      - "/app/cdk-node run --cfg /app/config.toml --custom-network-file /app/genesis.json --components sequence-sender"

  cdk-aggregator:
    container_name: cdk-aggregator
    image: cdk
    ports:
      - 50081:50081
      - 9093:9091 # needed if metrics enabled
    environment:
      - CDK_AGGREGATOR_DB_HOST=cdk-aggregator-db
      - CDK_AGGREGATOR_SENDER_ADDRESS=0x000
    volumes:
      - ./config.toml:/app/config.toml
      - ./genesis.json:/app/genesis.json
      - ./aggregator.keystore:/pk/aggregator.keystore
      - ../bin/cdk-node:/app/cdk-node
    command:
      - "/bin/sh"
      - "-c"
      - "/app/cdk-node run --cfg /app/config.toml --custom-network-file /app/genesis.json  --components aggregator"
    depends_on:
      cdk-aggregator-db:
        condition: service_started
      cdk-l1-sync-db:
        condition: service_started

  cdk-aggregator-db:
    container_name: cdk-aggregator-db
    image: postgres:15
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G
    ports:
      - 5434:5432
    environment:
      - POSTGRES_USER=aggregator_user
      - POSTGRES_PASSWORD=aggregator_password
      - POSTGRES_DB=aggregator_db
    command:
      - "postgres"
      - "-N"
      - "500"

  cdk-l1-sync-db:
    container_name: cdk-l1-sync-db
    image: postgres:15
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G
    ports:
      - 5436:5432
    environment:
      - POSTGRES_USER=aggregator_syncer_user
      - POSTGRES_PASSWORD=aggregator_syncer_password
      - POSTGRES_DB=aggregator_syncer_db
    command:
      - "postgres"
      - "-N"
      - "500"

In this, I removed the network params too. After update the docker-compose.yml, I faced the current errors. Thanks

[goran-ethernal]

That docker-compose might be obsolete.

Can you use the Make command to build a docker image. In the root folder of the cdk repo, you have a Make file. You can use the build-docker command to build a docker image for cdk node:

make build-docker

[AzFlin7]

That docker-compose might be obsolete.

Can you use the Make command to build a docker image. In the root folder of the cdk repo, you have a Make file. You can use the build-docker command to build a docker image for cdk node:

make build-docker

I already build docker at first.

[goran-ethernal]

That docker compose file is definitely obsolete. For example, we do not use Postgress database anymore. Which components do you need?

If you need sequence-sender, then I would suggest changing the docker-compose to create a folder in the docker container where the reorg detector db will be created, and then change the .toml file to use that folder in the ReorgDetector DBPath. Or, use the /app/reorgdetectorl1.sqlite as the DBPath, since the /app/ folder definitely exists in the docker container.

[AzFlin7]

That docker compose file is definitely obsolete. For example, we do not use Postgress database anymore. Which components do you need?

If you need sequence-sender, then I would suggest changing the docker-compose to create a folder in the docker container where the reorg detector db will be created, and then change the .toml file to use that folder in the ReorgDetector DBPath. Or, use the /app/reorgdetectorl1.sqlite as the DBPath, since the /app/ folder definitely exists in the docker container.

Thanks @goran-ethernal Can you please explain in more detail? I need sequence-sender and aggregator too. Where can I get reorgdetectorl1.sqlite file? Actually, I am little confused. So it will be working if I change the db image to sqlite for aggregator and sequencer-sync db? And also when I set the keystore files and reorgdetectorl1.sqlite file's path in config.toml file, the paths will be the file artifact path in container or absolute path? Btw, is there any chance to have a short call with you? Actually, I'd love to ask some additional questions. This is my personal email: mark.soosaar77@gmail.com Telegram: @marksoosaar

I look forward to hear from you soon. Warm regards

[AzFlin7]

@goran-ethernal Would you like answer my questions?