search

singlestore-labs / singlestoredb-dev-image

The SingleStoreDB Dev Container is the fastest way to develop with SingleStore on your laptop or in a CI/CD environment.
Apache License 2.0
41 stars 16 forks source link

Error 2374 due to CPU core quantity #5

Closed bitclaw closed 1 year ago

bitclaw commented 1 year ago

Describe the bug SingleStore docker container does not start successfully on a laptop/computer that has more than 4 cores.

To Reproduce Steps to reproduce the behavior:

  1. Run docker compose up
  2. See the following error:
$ docker compose up
[+] Running 3/3
 ⠿ Network traffic-copter_default          Created                                                                                                        0.1s
 ⠿ Volume "traffic-copter_data"            Created                                                                                                        0.0s
 ⠿ Container traffic-copter-singlestore-1  Created                                                                                                        2.8s
Attaching to traffic-copter-singlestore-1
traffic-copter-singlestore-1  | Starting SingleStore nodes...
traffic-copter-singlestore-1  | {}
traffic-copter-singlestore-1  | 
traffic-copter-singlestore-1  | real    0m7.204s
traffic-copter-singlestore-1  | user    0m0.580s
traffic-copter-singlestore-1  | sys 0m0.317s
traffic-copter-singlestore-1  | Configuring SingleStore nodes...
traffic-copter-singlestore-1  | Error 2373: Code generation for new statements is disabled because the total number of license units of capacity used on all leaf nodes is 4, which is above the limit of 2 for the SingleStore free license. To purchase more capacity, contact team@memsql.com. Or, adjust the size or number of leaf nodes to bring the number of license units used to 2 or less. See https://docs.memsql.com/redir/capacity-limit-error for more information.
traffic-copter-singlestore-1 exited with code 1

See below the docker-compose.yaml file I am using to test this new image:

version: '3.8'

services:
  singlestore:
    image: ghcr.io/singlestore-labs/singlestoredb-dev:latest
    ports:
      - 3306:3306
      - 8080:8080
    volumes:
      # persist data between runs:
      - data:/data
      - ./init.sql:/init.sql
    environment:
      # use the LICENSE_KEY environment variable set in the terminal:
      - SINGLESTORE_LICENSE=${SINGLESTORE_LICENSE}
      - ROOT_PASSWORD=root

volumes:
  data:

Expected behavior

I expect the SingleStore container to start successfully.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

$ nproc
32

Additional context

Additionally, SingleStore GitHub Action has never worked correctly. The code for this article is here.

I had to use instead docker-compose on the GitHub action with the image singlestore/cluster-in-a-box:latest. The developer who wrote that GitHub action should have placed a sanity check for the MySQL connection.

See also SingleStore Error 2374

To finalize, the image singlestore/cluster-in-a-box:latest works fine with docker compose. See the below docker compose configuration:

version: '3.8'

services:
  singlestore:
    image: singlestore/cluster-in-a-box:latest
    ports:
      - 3306:3306
      - 8080:8080
    volumes:
      # persist data between runs:
      - db-data:/var/lib/memsql
      - ./init.sql:/init.sql
    environment:
      # use the LICENSE_KEY environment variable set in the terminal:
      - LICENSE_KEY=${SINGLESTORE_LICENSE}
      - START_AFTER_INIT=Y
      - ROOT_PASSWORD=password_here

volumes:
  db-data:

And docker compose CLI output showing that the container started successfully:

$ docker compose up
[+] Running 1/0
 ⠿ Container tc-singlestore-1  Recreated                                                                                                      0.1s
Attaching to tc-singlestore-1
tc-singlestore-1  | Starting Cluster
tc-singlestore-1  | {}
tc-singlestore-1  | ==> /var/lib/memsql/1b482754-cbe1-4336-bfaf-811ea6789926/tracelogs/memsql.log <==
tc-singlestore-1  | 03651764 2022-09-30 16:01:43.925   INFO: Completed recovery of system databases.  Beginning recovery of user databases.
tc-singlestore-1  | 03656876 2022-09-30 16:01:43.930   INFO: Initializing cluster management.
tc-singlestore-1  | 03656926 2022-09-30 16:01:43.930   INFO: Consensus node started...
tc-singlestore-1  | 03657583 2022-09-30 16:01:43.931   INFO: Thread 115029: Fn: Initializing Replication Management Thread
tc-singlestore-1  | 03657734 2022-09-30 16:01:43.931   INFO: Starting the accept loop.
tc-singlestore-1  | 03670562 2022-09-30 16:01:43.944   INFO: Query information_schema.'SELECT @@memsql_id' submitted for asynchronous compilation
tc-singlestore-1  | 03746448 2022-09-30 16:01:44.019   INFO: Query information_schema.'SELECT node_id FROM INFORMATION_SCHEMA.lmv_nodes' submitted for asynchronous compilation
tc-singlestore-1  | 03771281 2022-09-30 16:01:44.044   INFO: Query information_schema.'SELECT @@memsql_version' submitted for asynchronous compilation
tc-singlestore-1  | 03876453 2022-09-30 16:01:44.149   INFO: Query information_schema.'SELECT @@memsql_id' submitted 215 milliseconds ago, queued for 12 milliseconds, compiled asynchronously in 203 milliseconds
tc-singlestore-1  | 03880903 2022-09-30 16:01:44.154   INFO: Query information_schema.'SELECT @@memsql_version' submitted 130 milliseconds ago, queued for 22 milliseconds, compiled asynchronously in 108 milliseconds
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/memsql/5b30dc09-100d-4ef3-9fc4-2b5c28919aa5/tracelogs/memsql.log <==
tc-singlestore-1  | 03650171 2022-09-30 16:01:43.926   INFO: Completed recovery of system databases.  Beginning recovery of user databases.
tc-singlestore-1  | 03655365 2022-09-30 16:01:43.931   INFO: Initializing cluster management.
tc-singlestore-1  | 03655413 2022-09-30 16:01:43.931   INFO: Consensus node started...
tc-singlestore-1  | 03655974 2022-09-30 16:01:43.931   INFO: Thread 115029: Fn: Initializing Replication Management Thread
tc-singlestore-1  | 03656188 2022-09-30 16:01:43.932   INFO: Starting the accept loop.
tc-singlestore-1  | 03668121 2022-09-30 16:01:43.944   INFO: Query information_schema.'SELECT @@memsql_id' submitted for asynchronous compilation
tc-singlestore-1  | 03785955 2022-09-30 16:01:44.061   INFO: Query information_schema.'SELECT node_id FROM INFORMATION_SCHEMA.lmv_nodes' submitted for asynchronous compilation
tc-singlestore-1  | 03798763 2022-09-30 16:01:44.074   INFO: Query information_schema.'SELECT @@memsql_version' submitted for asynchronous compilation
tc-singlestore-1  | 03905646 2022-09-30 16:01:44.181   INFO: Query information_schema.'SELECT @@memsql_id' submitted 246 milliseconds ago, queued for 11 milliseconds, compiled asynchronously in 235 milliseconds
tc-singlestore-1  | 03909953 2022-09-30 16:01:44.185   INFO: Query information_schema.'SELECT @@memsql_version' submitted 120 milliseconds ago, queued for 11 milliseconds, compiled asynchronously in 109 milliseconds
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/singlestoredb-studio/studio.log <==
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/memsql/memsql_exporter.log <==
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/memsql/1b482754-cbe1-4336-bfaf-811ea6789926/tracelogs/memsql.log <==
tc-singlestore-1  | 03991436 2022-09-30 16:01:44.264   INFO: Query information_schema.'SELECT node_id FROM INFORMATION_SCHEMA.lmv_nodes' submitted 258 milliseconds ago, queued for 16 milliseconds, compiled asynchronously in 242 milliseconds
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/memsql/5b30dc09-100d-4ef3-9fc4-2b5c28919aa5/tracelogs/memsql.log <==
tc-singlestore-1  | 04018928 2022-09-30 16:01:44.294   INFO: Query information_schema.'SELECT node_id FROM INFORMATION_SCHEMA.lmv_nodes' submitted 243 milliseconds ago, queued for 13 milliseconds, compiled asynchronously in 230 milliseconds
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/singlestoredb-studio/studio.log <==
tc-singlestore-1  | 2022/09/30 16:01:44 env.go:90 Log Opened
tc-singlestore-1  | 2022/09/30 16:01:44 server.go:74 Listening on 0.0.0.0:8080
tc-singlestore-1  | 2022/09/30 16:01:44 server.go:93 HTTPS configuration was not detected, serving with HTTP
tc-singlestore-1  | 
tc-singlestore-1  | ==> /var/lib/memsql/memsql_exporter.log <==
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Arguments: [@/opt/singlestoredb-server-7.8.13-f8fec5f0db/memsql_exporter/conf/memsql_exporter_ma.args]" source="memsql_exporter.go:621"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Starting memsql_exporter (version=7.8.13, branch=, revision=f8fec5f)" source="memsql_exporter.go:636"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Build context (go=devel +2a0dd053ec Fri Feb 5 19:53:15 2021 +0000, user=, date=)" source="memsql_exporter.go:637"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Read DSN from /etc/memsql/memsql_exporter.cnf" source="memsql_exporter.go:656"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Enabled scrapers:" source="memsql_exporter.go:671"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.show_status_extended" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.global_status_gauge_vars" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.show_variables" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.show_workload_management_status" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.distributed.partitions" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.events" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.mv_activities" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.tables" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.processlist" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.tablestats" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.memory.counters" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.mv_activities_extended_cumulative" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.system_info" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.pipeline_batches" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.info_schema.mv_nodes" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --collect.global_status_counting_vars" source="memsql_exporter.go:675"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Enabled cluster scrapers:" source="memsql_exporter.go:683"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_sysinfo_net" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_global_variables" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_workload_management_status" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_sysinfo_mem" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_sysinfo_disk" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_activities" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.tables" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_processlist" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.tablestats" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.events" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.distributed.partitions" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_global_status" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_sysinfo_cpu" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_activities_extended_cumulative" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.system_info" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.pipeline_batches" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --cluster-collect.info_schema.mv_nodes" source="memsql_exporter.go:687"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Enabled samplers:" source="memsql_exporter.go:694"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.activity_extended" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.pipeline_batches" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.cluster_info" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.events" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.node" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg=" --sample.activity" source="memsql_exporter.go:698"
tc-singlestore-1  | time="2022-09-30T16:01:44Z" level=info msg="Listening on :9104" source="memsql_exporter.go:755"
carlsverre commented 1 year ago

Thanks for this bug report! I think the limit is 16 cores rather than 4, but either way this problem needs to be resolved. Will figure out a solution today.

carlsverre commented 1 year ago

Also the github actions issue will be resolved in the blog. I will remove the example project since this repo contains working examples. Can you switch over to following this tutorial for github actions: https://github.com/singlestore-labs/singlestoredb-dev-image#github-actions

bitclaw commented 1 year ago

@carlsverre Thanks! Great work on this image!

carlsverre commented 1 year ago

This issue should be resolved in v0.0.9 which I am testing + releasing right now.

The github actions blog post and corresponding github repo will be resolved later today.

Thanks for the bug report! Please re-open this issue if 0.0.9 does not solve the problem.

carlsverre commented 1 year ago

Note: due to some other issues I am releasing v0.1.0 instead and bundling everything up. Should be out in the next hour or so.