Recovery question

[water5-cmd]

Hi, @thomasten, From the document, I know "when EdgelessDB is moved to another physical host, it enters recovery mode and waits for the master key to be passed over the HTTP REST API." I have a question about this statement: “when EdgelessDB is moved to another physical host”, does it refer to the edgelssdb folder or the executable file copied to another physical machine? By the way, when you execute the following command, then running ./edb also enters recovery mode, why?

// Prerequisite: edb is already running properly, then shut down
root@adminroot:~/edgelessdb/build# make clean
root@adminroot:~/edgelessdb/build# cmake ..
root@adminroot:~/edgelessdb/build# make -j`nproc`
root@adminroot:~/edgelessdb/build# ./edb                         //Not delete the data folder
[erthost] loading enclave ...
[erthost] entering enclave ...
[EDB] 2022/04/12 06:50:52 EdgelessDB v0.2.1 (8c1df2066513153ac9b8e129a0924abe431e3191)
[EDB] 2022/04/12 06:50:52 Failed to initialize master key: cipher: message authentication failed
[EDB] 2022/04/12 06:50:52 Entering recovery mode...
edb failed to retrieve the database encryption key and has entered recovery mode.
You can use the /recover API endpoint to upload the recovery data which was generated when the manifest has been initialized originally.
For more information, consult the documentation.
[EDB] 2022/04/12 06:50:52 HTTP REST API listening on :8080

In other terminal, I have followed the documentation and generated the RSA key pair

// can not get edb_temp.pem
root@adminroot:~/edgelessdb/build# era -c edgelessdb-sgx.json -h localhost:8080 -output-root edb_temp.pem
ERROR: Received an empty quote from host. Is it running in OE Simulation mode?
For testing purposes, you can pass the parameter '-skip-quote' to skip remote attestation.
panic: no quote received

goroutine 1 [running]:
main.main()
    /__w/era/era/cmd/era/main.go:63 +0xd7f

// can not use edb_temp.pem
root@adminroot:~/edgelessdb/build# base64 -d master_key | openssl pkeyutl -inkey recovery-private.pem -decrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 | curl --cacert edb_temp.pem --data-binary @- https://localhost:8080/recover
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

// so, I use curl -k, recovery successful
root@adminroot:~/edgelessdb/build# base64 -d master_key | openssl pkeyutl -inkey recovery-private.pem -decrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 | curl -k --data-binary @- https://localhost:8080/recover
{"status":"success","data":"Recovery successful."}

When you do not execute the make clean command, you will not enter recovery mode.

root@adminroot:~/edgelessdb/build# cmake ..
root@adminroot:~/edgelessdb/build# make -j`nproc`
root@adminroot:~/edgelessdb/build# ./edb                         //Not delete the data folder
[erthost] loading enclave ...
[erthost] entering enclave ...
test into emain
[EDB] 2022/04/12 06:59:46 EdgelessDB v0.2.1 (8c1df2066513153ac9b8e129a0924abe431e3191)
host is: 0.0.0.0[EDB] 2022/04/12 06:59:46 starting up ...
2022-04-12  6:59:46 0 [Note] edb (mysqld 10.5.11-MariaDB-debug-log) starting as process 1010301 ...
2022-04-12  6:59:46 0 [Warning] You need to use --log-bin to make --binlog-format work.
2022-04-12  6:59:46 0 [Note] Initializing built-in plugins
2022-04-12  6:59:46 0 [Note] Initializing plugins specified on the command line
2022-04-12  6:59:46 0 [Note] RocksDB: 4 column families found
2022-04-12  6:59:46 0 [Note] RocksDB: Column Families at start:
2022-04-12  6:59:46 0 [Note]   cf=default
2022-04-12  6:59:46 0 [Note]     write_buffer_size=67108864
2022-04-12  6:59:46 0 [Note]     target_file_size_base=67108864
2022-04-12  6:59:46 0 [Note]   cf=__system__
2022-04-12  6:59:46 0 [Note]     write_buffer_size=67108864
2022-04-12  6:59:46 0 [Note]     target_file_size_base=67108864
2022-04-12  6:59:46 0 [Note]   cf=edg_db_cf
2022-04-12  6:59:46 0 [Note]     write_buffer_size=67108864
2022-04-12  6:59:46 0 [Note]     target_file_size_base=67108864
2022-04-12  6:59:46 0 [Note]   cf=edg_frm_cf
2022-04-12  6:59:46 0 [Note]     write_buffer_size=67108864
2022-04-12  6:59:46 0 [Note]     target_file_size_base=67108864
2022-04-12  6:59:46 0 [Note] RocksDB: Table_store: loaded DDL data for 30 tables
2022-04-12  6:59:47 0 [Note] RocksDB: global statistics using get_sched_indexer_t indexer
2022-04-12  6:59:47 0 [Note] RocksDB: sched_getcpu() failed - global statistics will use thread_id_indexer_t instead
2022-04-12  6:59:47 0 [Note] MyRocks storage engine plugin has been successfully initialized.
2022-04-12  6:59:47 0 [Note] Initializing installed plugins
2022-04-12  6:59:47 0 [Note] Server socket created on IP: '0.0.0.0'.
2022-04-12  6:59:47 1 [Warning] Aborted connection 1 to db: 'unconnected' user: 'root' host: '' (This connection closed normally)
2022-04-12  6:59:47 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '255.0.0.1' (This connection closed normally without authentication)
2022-04-12  6:59:47 0 [Note] Reading of all Master_info entries succeeded
2022-04-12  6:59:47 0 [Note] Added new Master_info '' to hash table
2022-04-12  6:59:47 0 [Note] edb: ready for connections.
Version: '10.5.11-MariaDB-debug-log'  socket: ''  port: 3306  Source distribution
internalPath:/tmp/edb
internalAdress:255.0.0.1
externalPath:/data
externalAdress:
[EDB] 2022/04/12 06:59:47 DB is running.
[EDB] 2022/04/12 06:59:47 HTTP REST API listening on :8080

My guess is that I performed the recovery mode operation in the wrong usage scenario, which caused me to fail to fetch the edb_temp.pem. Is that the reason?

[thomasten]

Hi!

“when EdgelessDB is moved to another physical host”, does it refer to the edgelssdb folder or the executable file copied to another physical machine?

It refers to the data folder. EdgelessDB can only decrypt the data files on the same machine that encrypted it.

By the way, when you execute the following command, then running ./edb also enters recovery mode, why?

make clean removes the SGX signer key private.pem in the build folder, so make will generate a new one. The master key is (among other things) derived from the SGX signer. If you copy build/private.pem before doing make clean and copy it back afterwards (but before building), it should work.

In other terminal, I have followed the documentation and generated the RSA key pair

Regarding era failing, you've encountered a bug here. I can reproduce this. Thanks for reporting.

[thomasten]

The recovery mode bug has been fixed in v0.3.0