search

questdb / questdb

QuestDB is an open source time-series database for fast ingest and SQL queries
https://questdb.io
Apache License 2.0
14.6k stars 1.18k forks source link

Data cannot be recovered after the server is powered off #2420

Closed pinzi closed 1 year ago

pinzi commented 2 years ago

Describe the bug

Data cannot be recovered after the server is powered off

image image

To reproduce

the server is powered off

Expected Behavior

No response

Environment

- **QuestDB version**:6.4.3
- **OS**:windows server 2019 datacenter
- **Browser**:Google Chrome

Additional context

No response

pswu11 commented 2 years ago

Hi @pinzi could you provide more context about "data cannot be recovered"? Do you mean your instance went offline, and after it's back online you cannot access the data anymore? What steps have you tried?

pinzi commented 2 years ago

The server was restarted due to an unexpected power outage in the computer room.One data table cannot be displayed in http://localhost:9000. After the database service is restarted.I did nothing.

ideoma commented 2 years ago

What will be in the logs if you do select from this table?

pinzi commented 2 years ago

What will be in the logs if you do select from this table?

I can not see it

1660570789446

pswu11 commented 2 years ago

Hi @pinzi powerless could cause database files corruption, to give you a better context: (from our docs, storage model)

By default QuestDB relies on OS-level data durability leaving the OS to write dirty pages to disk. Data durability can be also configured with commit() optionally being able to invoke msync() with a choice of synchronous or asynchronous IO. The msync() calls are made for column files only, so while the sync/async commit modes improve the overall durability, they don't guarantee durability in the face of OS errors or power loss.

As for now, we'd recommend users have regular snapshots. https://questdb.io/docs/reference/sql/snapshot/

ideoma commented 2 years ago

What will be in the logs if you do select from this table?

I can not see it

You can still query it and see what logs will say

ideoma commented 2 years ago

If you can attach the folder content we can have a look

pinzi commented 2 years ago

If you can attach the folder content we can have a look

[0]: Metadata read timeout

pinzi commented 2 years ago

Hi @pinzi powerless could cause database files corruption, to give you a better context: (from our docs, storage model)

By default QuestDB relies on OS-level data durability leaving the OS to write dirty pages to disk. Data durability can be also configured with commit() optionally being able to invoke msync() with a choice of synchronous or asynchronous IO. The msync() calls are made for column files only, so while the sync/async commit modes improve the overall durability, they don't guarantee durability in the face of OS errors or power loss.

As for now, we'd recommend users have regular snapshots. https://questdb.io/docs/reference/sql/snapshot/

Snapshot statements are not supported on Windows OS.