search

m3db / m3

M3 monorepo - Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Graphite Compatible, Metrics Platform
https://m3db.io/
Apache License 2.0
4.73k stars 451 forks source link

Cannot run single node m3db with default configuration #3433

Open aimtsou opened 3 years ago

aimtsou commented 3 years ago

General Issues

I am following the documentation(https://m3db.io/docs/quickstart/binaries/): 1) I have downloaded the binaries and extracted them to a directory 2) I fixed the system variables 3) I got the sample configuration file and run the m3dbnode

{"level":"warn","ts":1618921322.172541,"msg":"could not load cache from file","file":"/var/lib/m3kv/_kv_default_env_m3db_embedded.json","error":"error opening cache file /var/lib/m3kv/_kv_default_env_m3db_embedded.json: open /var/lib/m3kv/_kv_default_env_m3db_embedded.json: no such file or directory"}

{"level":"error","ts":1618921335.5360305,"msg":"error initializing namespaces values, retrying in the background","key":"/namespaces","error":"initializing value error (key='/namespaces'): init watch timeout"} {"level":"info","ts":1618921335.7031667,"msg":"received kv update","version":1,"key":"/placement"} {"level":"info","ts":1618921335.7035007,"msg":"election manager opened successfully"} {"level":"info","ts":1618921335.7037265,"msg":"election state changed from follower to leader"}

I am not sure why I cannot create a placement.

Please provide the following information along with a description of the issue that you're experiencing:

  1. What service is experiencing the issue? (M3Coordinator, M3DB, M3Aggregator, etc)

m3db

  1. What is the configuration of the service? Please include any YAML files, as well as namespace / placement configuration (with any sensitive information anonymized if necessary).

Default yaml (https://github.com/m3db/m3/blob/master/src/dbnode/config/m3dbnode-local-etcd.yml)

  1. How are you using the service? For example, are you performing read/writes to the service via Prometheus, or are you using a custom script?

Without prometheus, trying with curl curl http://localhost:7201/api/v1/services/m3db/placement

gibbscullen commented 3 years ago

@aimtsou -- at which step of the quickstart guide are you having trouble with?

aimtsou commented 3 years ago

@gibbscullen:

I confirm after retrying that the documentation works. My mistake because I did not check if the process was running before trying the curl but for some reason the process gets killed(I have not made it yet a system process).

{"level":"warn","ts":1618931148.4730375,"msg":"skipping namespace removals and updates (except schema updates and runtime options), restart the process if you want changes to take effect"} {"level":"info","ts":1618931148.474379,"msg":"dynamic namespace registry updated to version","version":2} {"level":"info","ts":1618931148.4744046,"msg":"received update from kv namespace watch"} {"level":"info","ts":1618931148.4746268,"msg":"refreshed cluster namespaces","added":[],"updated":["default"],"removed":[]} Killed

praisethejack commented 3 years ago

@gibbscullen : I also try to deploy M3 follow https://m3db.io/docs/quickstart/binaries/ I had the same problem. 

[root@node2 m3]# ./m3dbnode -f ./m3dbnode-local-etcd.yml
2021/05/14 08:41:14 Go Runtime version: go1.13.8
2021/05/14 08:41:14 Build Version:      v1.1.0
2021/05/14 08:41:14 Build Revision:     468ab1c4b
2021/05/14 08:41:14 Build Branch:       HEAD
2021/05/14 08:41:14 Build Date:         2021-02-03-16:57:21
2021/05/14 08:41:14 Build TimeUnix:     1612371441
{"level":"info","ts":1620952874.8099675,"msg":"setting MutexProfileFraction: 0"}
{"level":"info","ts":1620952874.810015,"msg":"setting BlockProfileRate: 0"}
{"level":"info","ts":1620952874.8100953,"msg":"raised rlimit no file fds limit","required":false,"sysNROpenValue":3000000,"noFileMaxValue":3000000,"noFileCurrValue":3000000}
{"level":"info","ts":1620952874.8102643,"msg":"setting MutexProfileFraction: 0"}
{"level":"info","ts":1620952874.8102984,"msg":"setting BlockProfileRate: 0"}
{"level":"info","ts":1620952874.8118398,"msg":"tracing disabled for m3query; set `tracing.backend` to enable"}
{"level":"info","ts":1620952874.8141813,"msg":"tracing disabled; set `tracing.backend` to enable"}
{"level":"info","ts":1620952874.8142583,"msg":"resolving seed node configuration","hostID":"m3db_local","seedNodeHostIDs":["m3db_local"]}
{"level":"info","ts":1620952874.8142838,"msg":"seed node, starting etcd server"}
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-05-14 08:41:14.814811 I | embed: name = m3db_local
2021-05-14 08:41:14.814834 I | embed: data dir = /var/lib/m3db/etcd
2021-05-14 08:41:14.814851 I | embed: member dir = /var/lib/m3db/etcd/member
2021-05-14 08:41:14.814865 I | embed: heartbeat = 100ms
2021-05-14 08:41:14.814879 I | embed: election = 1000ms
2021-05-14 08:41:14.814893 I | embed: snapshot count = 100000
2021-05-14 08:41:14.814916 I | embed: advertise client URLs = http://127.0.0.1:2379
2021-05-14 08:41:14.814934 I | embed: initial advertise peer URLs = http://127.0.0.1:2380
2021-05-14 08:41:14.814953 I | embed: initial cluster = 
2021-05-14 08:41:14.824510 I | etcdserver: restarting member b71f75320dc06a6c in cluster 1c45a069f3a1d796 at commit index 22
raft2021/05/14 08:41:14 INFO: b71f75320dc06a6c switched to configuration voters=()
raft2021/05/14 08:41:14 INFO: b71f75320dc06a6c became follower at term 8
raft2021/05/14 08:41:14 INFO: newRaft b71f75320dc06a6c [peers: [], term: 8, commit: 22, applied: 0, lastindex: 22, lastterm: 8]
2021-05-14 08:41:14.829539 W | auth: simple token is not cryptographically signed
2021-05-14 08:41:14.841281 I | etcdserver: starting server... [version: 3.4.13, cluster version: to_be_decided]
raft2021/05/14 08:41:14 INFO: b71f75320dc06a6c switched to configuration voters=(13195394291058371180)
2021-05-14 08:41:14.842379 I | etcdserver/membership: added member b71f75320dc06a6c [http://127.0.0.1:2380] to cluster 1c45a069f3a1d796
2021-05-14 08:41:14.842484 N | etcdserver/membership: set the initial cluster version to 3.4
2021-05-14 08:41:14.842514 I | etcdserver/api: enabled capabilities for version 3.4
2021-05-14 08:41:14.844814 I | embed: listening for peers on [::]:2380
{"level":"info","ts":1620952874.9862587,"msg":"successfully created new cache dir","path":"/var/lib/m3kv","mode":0}
{"level":"info","ts":1620952874.9864328,"msg":"successfully loaded cache from file","file":"/var/lib/m3kv/_kv_default_env_m3db_embedded.json"}
{"level":"info","ts":1620952875.0512884,"msg":"creating namespaces watcher","clusters":1}
{"level":"info","ts":1620952875.0515127,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.node.namespaces","revision":0}
{"level":"warn","ts":1620952875.2074301,"msg":"max index query IDs concurrency was not set, falling back to default value"}
{"level":"warn","ts":1620952875.2075589,"msg":"host doesn't support HugeTLB, proceeding without it"}
{"level":"info","ts":1620952875.2385128,"msg":"set thrift bytes pool alloc size","size":2048}
{"level":"info","ts":1620952875.2385685,"msg":"bytes pool configured","capacity":16,"size":524288,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2385857,"msg":"bytes pool configured","capacity":32,"size":262144,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2385945,"msg":"bytes pool configured","capacity":64,"size":131072,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2386003,"msg":"bytes pool configured","capacity":128,"size":65536,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2386057,"msg":"bytes pool configured","capacity":256,"size":65536,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2386115,"msg":"bytes pool configured","capacity":1440,"size":16384,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2386172,"msg":"bytes pool configured","capacity":4096,"size":8192,"refillLowWaterMark":0.3,"refillHighWaterMark":0.6}
{"level":"info","ts":1620952875.2386599,"msg":"bytes pool init start"}
{"level":"info","ts":1620952875.7343376,"msg":"bytes pool init end"}
{"level":"info","ts":1620952875.9741464,"msg":"creating dynamic config service client with m3cluster"}
{"level":"info","ts":1620952875.9747343,"msg":"successfully created new cache dir","path":"/var/lib/m3kv","mode":493}
{"level":"info","ts":1620952875.974803,"msg":"successfully loaded cache from file","file":"/var/lib/m3kv/_kv_default_env_m3db_embedded.json"}
{"level":"info","ts":1620952876.3236039,"msg":"node tchannelthrift: listening","address":"0.0.0.0:9000"}
{"level":"info","ts":1620952876.3241925,"msg":"node httpjson: listening","address":"0.0.0.0:9002"}
{"level":"info","ts":1620952876.324254,"msg":"waiting for dynamic topology initialization, if this takes a long time, make sure that a topology/placement is configured"}
{"level":"info","ts":1620952876.324272,"msg":"adding a watch","service":"m3db","env":"default_env","zone":"embedded","includeUnhealthy":true}
{"level":"info","ts":1620952876.3243678,"msg":"successfully created new cache dir","path":"/var/lib/m3kv","mode":493}
{"level":"info","ts":1620952876.3244379,"msg":"successfully loaded cache from file","file":"/var/lib/m3kv/m3db_embedded.json"}
{"level":"info","ts":1620952876.3244958,"msg":"creating etcd watch at revision","watch_key":"_sd.placement/default_env/m3db","revision":0}
raft2021/05/14 08:41:16 INFO: b71f75320dc06a6c is starting a new election at term 8
raft2021/05/14 08:41:16 INFO: b71f75320dc06a6c became candidate at term 9
raft2021/05/14 08:41:16 INFO: b71f75320dc06a6c received MsgVoteResp from b71f75320dc06a6c at term 9
raft2021/05/14 08:41:16 INFO: b71f75320dc06a6c became leader at term 9
raft2021/05/14 08:41:16 INFO: raft.node: b71f75320dc06a6c elected leader b71f75320dc06a6c at term 9
2021-05-14 08:41:16.626615 I | etcdserver: published {Name:m3db_local ClientURLs:[http://127.0.0.1:2379]} to cluster 1c45a069f3a1d796
2021-05-14 08:41:16.626661 I | embed: ready to serve client requests
2021-05-14 08:41:16.627678 N | embed: serving insecure client requests on [::]:2379, this is strongly discouraged!
{"level":"info","ts":1620952876.639591,"msg":"initial namespace value received"}
{"level":"info","ts":1620952876.6403859,"msg":"refreshed cluster namespaces","added":["default_unaggregated"],"updated":[],"removed":[]}
{"level":"warn","ts":1620952876.640396,"msg":"dynamic namespace registry received older version, skipping","version":1}
{"level":"info","ts":1620952876.6404378,"msg":"namespace has non-ready staging state status","namespace":"default_unaggregated","status":"unknown"}
{"level":"info","ts":1620952876.640349,"msg":"initial topology / placement value received"}
{"level":"info","ts":1620952876.6405873,"msg":"configuring downsampler to use with aggregated cluster namespaces","numAggregatedClusterNamespaces":0}
{"level":"info","ts":1620952876.6583068,"msg":"no m3msg server configured"}
{"level":"info","ts":1620952876.658349,"msg":"using registered interrupt handler"}
{"level":"info","ts":1620952876.658368,"msg":"starting API server","address":"[::]:7201"}
{"level":"info","ts":1620952876.6757576,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env//namespaces","revision":0}
{"level":"info","ts":1620952876.6820033,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.client.bootstrap-consistency-level","revision":0}
{"level":"info","ts":1620952876.6826081,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.client.read-consistency-level","revision":0}
{"level":"info","ts":1620952876.682986,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.client.write-consistency-level","revision":0}
{"level":"info","ts":1620952876.8225446,"msg":"successfully updated topology","numHosts":1}
{"level":"info","ts":1620952878.2371209,"msg":"cluster tchannelthrift: listening","address":"0.0.0.0:9001"}
{"level":"info","ts":1620952878.2372875,"msg":"cluster httpjson: listening","address":"0.0.0.0:9003"}
{"level":"info","ts":1620952878.2374518,"msg":"cluster database initializing topology"}
{"level":"info","ts":1620952878.2374585,"msg":"cluster database resolving topology"}
{"level":"info","ts":1620952878.2374637,"msg":"cluster database resolved topology"}
{"level":"info","ts":1620952878.2607145,"msg":"creating namespaces watch"}
{"level":"info","ts":1620952878.2608252,"msg":"waiting for dynamic namespace registry initialization, if this takes a long time, make sure that a namespace is configured"}
{"level":"info","ts":1620952878.2609024,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.node.namespaces","revision":0}
{"level":"info","ts":1620952878.2635598,"msg":"initial namespace value received"}
{"level":"info","ts":1620952878.2638268,"msg":"resolving namespaces with namespace watch"}
{"level":"info","ts":1620952878.2639859,"msg":"updating database namespaces","adds":"[default_unaggregated]","updates":"[]","removals":"[]"}
{"level":"info","ts":1620952878.266283,"msg":"set namespace runtime index options","namespace":"default_unaggregated","namespace":"default_unaggregated","writeIndexingPerCPUConcurrency":null,"flushIndexingPerCPUConcurrency":null}
{"level":"info","ts":1620952878.2663238,"msg":"namespace forward indexing not enabled","namespace":"default_unaggregated","namespace":"default_unaggregated","enabled":false,"threshold":0,"probability":0}
{"level":"info","ts":1620952878.280406,"msg":"using registered interrupt handler"}
{"level":"info","ts":1620952878.2809634,"msg":"bootstrap prepare"}
{"level":"info","ts":1620952878.3105178,"msg":"bootstrap started","numShards":64}
{"level":"info","ts":1620952878.311187,"msg":"bootstrap range starting","bootstrapper":"base","namespace":"default_unaggregated","numShards":64,"dataFrom":1620864000,"dataTo":1620950400,"dataRange":86400,"indexFrom":1620864000,"indexTo":1620950400,"indexRange":86400}
{"level":"info","ts":1620952878.3113015,"msg":"bootstrap from source requested","bootstrapper":"filesystem","namespace":"default_unaggregated","numShards":64,"dataRange":86400,"dataFrom":1620864000,"dataTo":1620950400,"indexRange":86400,"indexFrom":1620864000,"indexTo":1620950400}
{"level":"info","ts":1620952878.4687495,"msg":"bootstrap from source ready after availability query","bootstrapper":"filesystem","namespace":"default_unaggregated","numShards":64,"dataRange":86400,"dataFrom":1620864000,"dataTo":1620950400,"indexRange":86400,"indexFrom":1620864000,"indexTo":1620950400}
{"level":"info","ts":1620952878.4687836,"msg":"bootstrap from source hook begin started","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.4688077,"msg":"bootstrap from source started","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.47316,"msg":"bootstrapping time series data start","bootstrapper":"filesystem","cachePolicy":"lru"}
{"level":"info","ts":1620952878.4774213,"msg":"bootstrapping time series data success","bootstrapper":"filesystem","cachePolicy":"lru","took":0.004269006}
{"level":"info","ts":1620952878.4774542,"msg":"bootstrapping index metadata start","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.497617,"msg":"bootstrapping index metadata success","bootstrapper":"filesystem","took":0.020162565}
{"level":"info","ts":1620952878.4976323,"msg":"bootstrap from source hook end started","bootstrapper":"filesystem","took":0.028848}
{"level":"info","ts":1620952878.5264068,"msg":"bootstrap from source completed","bootstrapper":"filesystem","took":0.028848}
{"level":"info","ts":1620952878.5298169,"msg":"bootstrapping from source completed successfully","bootstrapper":"filesystem","took":0.028848,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":86400,"dataRangeFulfilled":86400,"indexRangeRequested":86400,"indexRangeFulfilled":86400,"numIndexBlocks":24}
{"level":"info","ts":1620952878.5298429,"msg":"bootstrap from source requested","bootstrapper":"commitlog","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5298548,"msg":"bootstrap from source ready after availability query","bootstrapper":"commitlog","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5298626,"msg":"bootstrap from source hook begin started","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.5298839,"msg":"bootstrap from source started","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.529903,"msg":"read snapshots start","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.5333507,"msg":"read snapshots done","bootstrapper":"commitlog","took":0.003447681}
{"level":"info","ts":1620952878.5335026,"msg":"read commit logs start","bootstrapper":"commitlog"}
{"level":"error","ts":1620952878.5340533,"msg":"opting to skip commit log due to corruption","bootstrapper":"commitlog","error":"/var/lib/m3db/commitlogs/commitlog-0-4909.db: EOF"}
{"level":"info","ts":1620952878.5344412,"msg":"returning not-unfulfilled: ReturnUnfulfilledForCorruptCommitLogFiles is false","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.534458,"msg":"read commit logs done","bootstrapper":"commitlog","took":0.000954521,"datapointsRead":0,"datapointsSkippedNotBootstrappingNamespace":0,"datapointsSkippedNotBootstrappingShard":0,"datapointsSkippedShardNoLongerOwned":0}
{"level":"info","ts":1620952878.534476,"msg":"bootstrap from source hook end started","bootstrapper":"commitlog","took":0.004612817}
{"level":"info","ts":1620952878.5586698,"msg":"bootstrap from source completed","bootstrapper":"commitlog","took":0.004612817}
{"level":"info","ts":1620952878.5587566,"msg":"bootstrapping from source completed successfully","bootstrapper":"commitlog","took":0.004612817,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.5587733,"msg":"bootstrap from source requested","bootstrapper":"peers","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5587995,"msg":"bootstrap from source ready after availability query","bootstrapper":"peers","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5588074,"msg":"bootstrap from source hook begin started","bootstrapper":"peers"}
{"level":"info","ts":1620952878.5588222,"msg":"bootstrap from source started","bootstrapper":"peers"}
{"level":"info","ts":1620952878.5588527,"msg":"bootstrap from source hook end started","bootstrapper":"peers","took":0.000044825}
{"level":"info","ts":1620952878.5815299,"msg":"bootstrap from source completed","bootstrapper":"peers","took":0.000044825}
{"level":"info","ts":1620952878.5815785,"msg":"bootstrapping from source completed successfully","bootstrapper":"peers","took":0.000044825,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.5816052,"msg":"bootstrap from source requested","bootstrapper":"uninitialized_topology","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5816267,"msg":"bootstrap from source ready after availability query","bootstrapper":"uninitialized_topology","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.5816364,"msg":"bootstrap from source hook begin started","bootstrapper":"uninitialized_topology"}
{"level":"info","ts":1620952878.581652,"msg":"bootstrap from source started","bootstrapper":"uninitialized_topology"}
{"level":"info","ts":1620952878.5816612,"msg":"bootstrap from source hook end started","bootstrapper":"uninitialized_topology","took":0.000024833}
{"level":"info","ts":1620952878.6040175,"msg":"bootstrap from source completed","bootstrapper":"uninitialized_topology","took":0.000024833}
{"level":"info","ts":1620952878.6040664,"msg":"bootstrapping from source completed successfully","bootstrapper":"uninitialized_topology","took":0.000024833,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.604185,"msg":"bootstrap range completed","bootstrapper":"base","namespace":"default_unaggregated","numShards":64,"dataFrom":1620864000,"dataTo":1620950400,"dataRange":86400,"indexFrom":1620864000,"indexTo":1620950400,"indexRange":86400,"took":0.292972734,"numIndexBlocks":24}
{"level":"info","ts":1620952878.6042104,"msg":"bootstrap range starting","bootstrapper":"base","namespace":"default_unaggregated","numShards":64,"dataFrom":1620950400,"dataTo":1620954000,"dataRange":3600,"indexFrom":1620950400,"indexTo":1620954000,"indexRange":3600}
{"level":"info","ts":1620952878.6042511,"msg":"bootstrap from source requested","bootstrapper":"filesystem","namespace":"default_unaggregated","numShards":64,"dataRange":3600,"dataFrom":1620950400,"dataTo":1620954000,"indexRange":3600,"indexFrom":1620950400,"indexTo":1620954000}
{"level":"info","ts":1620952878.6046379,"msg":"bootstrap from source ready after availability query","bootstrapper":"filesystem","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.604649,"msg":"bootstrap from source hook begin started","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.6046748,"msg":"bootstrap from source started","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.6047864,"msg":"bootstrapping time series data start","bootstrapper":"filesystem","cachePolicy":"lru"}
{"level":"info","ts":1620952878.6048,"msg":"bootstrapping time series data success","bootstrapper":"filesystem","cachePolicy":"lru","took":0.000014112}
{"level":"info","ts":1620952878.6048064,"msg":"bootstrapping index metadata start","bootstrapper":"filesystem"}
{"level":"info","ts":1620952878.604815,"msg":"bootstrapping index metadata success","bootstrapper":"filesystem","took":0.000008319}
{"level":"info","ts":1620952878.6048205,"msg":"bootstrap from source hook end started","bootstrapper":"filesystem","took":0.000171004}
{"level":"info","ts":1620952878.63076,"msg":"bootstrap from source completed","bootstrapper":"filesystem","took":0.000171004}
{"level":"info","ts":1620952878.6314726,"msg":"bootstrapping from source completed successfully","bootstrapper":"filesystem","took":0.000171004,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.6315103,"msg":"bootstrap from source requested","bootstrapper":"commitlog","namespace":"default_unaggregated","numShards":64,"dataRange":3600,"dataFrom":1620950400,"dataTo":1620954000,"indexRange":3600,"indexFrom":1620950400,"indexTo":1620954000}
{"level":"info","ts":1620952878.6316953,"msg":"bootstrap from source ready after availability query","bootstrapper":"commitlog","namespace":"default_unaggregated","numShards":64,"dataRange":3600,"dataFrom":1620950400,"dataTo":1620954000,"indexRange":3600,"indexFrom":1620950400,"indexTo":1620954000}
{"level":"info","ts":1620952878.6317065,"msg":"bootstrap from source hook begin started","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.6317246,"msg":"bootstrap from source started","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.631739,"msg":"read snapshots start","bootstrapper":"commitlog"}
{"level":"info","ts":1620952878.6326494,"msg":"read snapshots done","bootstrapper":"commitlog","took":0.000910248}
{"level":"info","ts":1620952878.6326642,"msg":"bootstrap from source hook end started","bootstrapper":"commitlog","took":0.000957622}
{"level":"info","ts":1620952878.6584594,"msg":"bootstrap from source completed","bootstrapper":"commitlog","took":0.000957622}
{"level":"info","ts":1620952878.659064,"msg":"bootstrapping from source completed successfully","bootstrapper":"commitlog","took":0.000957622,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":3600,"dataRangeFulfilled":3600,"indexRangeRequested":3600,"indexRangeFulfilled":3600,"numIndexBlocks":0}
{"level":"info","ts":1620952878.659084,"msg":"bootstrap from source requested","bootstrapper":"peers","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.6591103,"msg":"bootstrap from source ready after availability query","bootstrapper":"peers","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.6591182,"msg":"bootstrap from source hook begin started","bootstrapper":"peers"}
{"level":"info","ts":1620952878.6591344,"msg":"bootstrap from source started","bootstrapper":"peers"}
{"level":"info","ts":1620952878.659145,"msg":"bootstrap from source hook end started","bootstrapper":"peers","took":0.000027041}
{"level":"info","ts":1620952878.685654,"msg":"bootstrap from source completed","bootstrapper":"peers","took":0.000027041}
{"level":"info","ts":1620952878.6857057,"msg":"bootstrapping from source completed successfully","bootstrapper":"peers","took":0.000027041,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.6857278,"msg":"bootstrap from source requested","bootstrapper":"uninitialized_topology","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.68574,"msg":"bootstrap from source ready after availability query","bootstrapper":"uninitialized_topology","namespace":"default_unaggregated","numShards":64,"dataRange":0,"indexRange":0}
{"level":"info","ts":1620952878.6857474,"msg":"bootstrap from source hook begin started","bootstrapper":"uninitialized_topology"}
{"level":"info","ts":1620952878.6857626,"msg":"bootstrap from source started","bootstrapper":"uninitialized_topology"}
{"level":"info","ts":1620952878.685773,"msg":"bootstrap from source hook end started","bootstrapper":"uninitialized_topology","took":0.000025038}
{"level":"info","ts":1620952878.7132056,"msg":"bootstrap from source completed","bootstrapper":"uninitialized_topology","took":0.000025038}
{"level":"info","ts":1620952878.7132437,"msg":"bootstrapping from source completed successfully","bootstrapper":"uninitialized_topology","took":0.000025038,"namespace":"default_unaggregated","numShards":64,"dataRangeRequested":0,"dataRangeFulfilled":0,"indexRangeRequested":0,"indexRangeFulfilled":0,"numIndexBlocks":0}
{"level":"info","ts":1620952878.7132807,"msg":"bootstrap range completed","bootstrapper":"base","namespace":"default_unaggregated","numShards":64,"dataFrom":1620950400,"dataTo":1620954000,"dataRange":3600,"indexFrom":1620950400,"indexTo":1620954000,"indexRange":3600,"took":0.109048597,"numIndexBlocks":0}
{"level":"info","ts":1620952878.7133126,"msg":"bootstrap succeeded, marking namespaces complete","numShards":64,"bootstrapDuration":0.432345597}
{"level":"info","ts":1620952878.7133317,"msg":"bootstrap marking all shards as bootstrapped","namespace":"default_unaggregated","namespace":"default_unaggregated","numShards":64}
{"level":"info","ts":1620952879.012588,"msg":"bootstrap index with bootstrapped index segments","namespace":"default_unaggregated","numIndexBlocks":24}
{"level":"info","ts":1620952879.0149784,"msg":"bootstrap success","numShards":64,"bootstrapDuration":0.432345597}
{"level":"info","ts":1620952879.0149984,"msg":"bootstrapped"}
{"level":"info","ts":1620952879.0162363,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.node.cluster-new-series-insert-limit","revision":0}
{"level":"info","ts":1620952879.0168374,"msg":"creating etcd watch at revision","watch_key":"_kv/default_env/m3db.node.encoders-per-block-limit","revision":0}
{"level":"warn","ts":1620952883.2992258,"msg":"corrupt commitlog file during cleanup, marking file for deletion","comment":"partial/corrupt files are expected as result of a restart (this is ok)","error":"/var/lib/m3db/commitlogs/commitlog-0-4909.db: EOF","path":"/var/lib/m3db/commitlogs/commitlog-0-4909.db"}
{"level":"error","ts":1620952886.6760092,"msg":"error initializing namespaces values, retrying in the background","key":"/namespaces","error":"initializing value error (key='/namespaces'): init watch timeout"}
{"level":"info","ts":1620952886.8910687,"msg":"received kv update","version":1,"key":"/placement"}
{"level":"info","ts":1620952886.8915358,"msg":"election manager opened successfully"}
{"level":"info","ts":1620952886.8920674,"msg":"election state changed from follower to leader"}

It looks like commitLog corruption caused NS initialization to fail. Status is "UNKNOWN"

[root@node2 ~]# curl http://localhost:7201/api/v1/services/m3db/namespace
{"registry":{"namespaces":{"default_unaggregated":{"bootstrapEnabled":true,"flushEnabled":true,"writesToCommitLog":true,"cleanupEnabled":true,"repairEnabled":false,"retentionOptions":{"retentionPeriodNanos":"86400000000000","blockSizeNanos":"3600000000000","bufferFutureNanos":"120000000000","bufferPastNanos":"600000000000","blockDataExpiry":true,"blockDataExpiryAfterNotAccessPeriodNanos":"300000000000","futureRetentionPeriodNanos":"0"},"snapshotEnabled":true,"indexOptions":{"enabled":true,"blockSizeNanos":"3600000000000"},"schemaOptions":null,"coldWritesEnabled":false,"runtimeOptions":null,"cacheBlocksOnRetrieve":false,"aggregationOptions":{"aggregations":[{"aggregated":false,"attributes":null}]},"stagingState":{"status":"UNKNOWN"},"extendedOptions":null}}}}

Once I try to call the write interface, M3 will report a lot of errors and terminate

curl -X POST http://localhost:7201/api/v1/json/write -d '{"tags":{"__name__": "abc"},"timestamp":"1620877967","value":1234}'
or
curl http://localhost:9003/writetagged -s -X POST -d '{"namespace": "default","id": "foo","tags": [{"name": "__name__","value": "user_login"},{"name": "city","value": "new_york"},{"name": "endpoint","value": "/request"}],"datapoint": {"timestamp":'"$(date +"%s")"',"value": 42.123456789}}'

{"level":"error","ts":1620954833.8027418,"msg":"write error","rqID":"b3acd24a-f7d0-4899-92a9-5d3758fd9c55","remoteAddr":"127.0.0.1:55566","error":"unaggregated namespace is not yet initialized"}
SIGILL: illegal instruction
PC=0xd01c10 m=2 sigcode=2

Beside, I wonder why the API documentation https://m3db.io/openapi/ doesn't include write and query interfaces. It feels like M3 is there just for Promtheus.