This is the edgedb-rust part of the changes to make restore output more verbose when needed, as users with large databases currently just see a blank screen while the restore process is (probably) ongoing and are unsure whether the restore is actually happening or not.
The restore method is called from edgedb-cli but located here, and fortunately it turns out there is already a mechanism for restore output which starts here where the CLI looks for a --verbose flag, and sets the log level to info for the restore module if found:
On the output side these changes will turn the output from this:
// (No output throughout entire restore, then...)
Blocks sent in 9.4168649s
To this:
[2023-09-25T02:18:00Z INFO edgedb::restore] Block 1 processed: 10.04 MB restored
[2023-09-25T02:18:00Z INFO edgedb::restore] Block 20 processed: 198.64 MB restored
...
Database restored in 9.4168649s
The [2023-09-25T02:18:00Z INFO edgedb::restore] part is pretty noisy but it can be changed on the edgedb-cli side to have the env_logger set to only display the arguments, so just the Block 1 processed: 10.04 MB restored per line.
The edgedb-cli PR to make the output less noisy (and add info on the file size) is here https://github.com/edgedb/edgedb-cli/pull/1132 which I'll turn from draft into a real PR if this one is merged.
The other way to do it is what we discussed at devrel which would be to change the restore function to pass back more info to the CLI per block. One way would be splitting into three functions:
fn start_restore() which goes up to line 71
fn process_block() which returns the number of bytes that the CLI would use to generate output,
fn complete_restore() which takes care of the rest after the blocks are processed.
Would also involve passing the guard back and forth a bit. So that way is probably doable but the only advantage would be being able to use a progress bar instead of displaying each line. But since it turns out that the existing structure already has level-based logging and is well thought out (such as setting the level based on the command and only applying to a single module) it feels like overkill. Thoughts?
This is the edgedb-rust part of the changes to make restore output more verbose when needed, as users with large databases currently just see a blank screen while the restore process is (probably) ongoing and are unsure whether the restore is actually happening or not.
The restore method is called from edgedb-cli but located here, and fortunately it turns out there is already a mechanism for restore output which starts here where the CLI looks for a --verbose flag, and sets the log level to info for the restore module if found:
https://github.com/edgedb/edgedb-cli/blob/b7a02e0cf53cf7f279d6125d14bff1ba191150fa/src/log_levels.rs#L27
On the output side these changes will turn the output from this:
To this:
The
[2023-09-25T02:18:00Z INFO edgedb::restore]part is pretty noisy but it can be changed on the edgedb-cli side to have the env_logger set to only display the arguments, so just theBlock 1 processed: 10.04 MB restoredper line.The edgedb-cli PR to make the output less noisy (and add info on the file size) is here https://github.com/edgedb/edgedb-cli/pull/1132 which I'll turn from draft into a real PR if this one is merged.
The other way to do it is what we discussed at devrel which would be to change the restore function to pass back more info to the CLI per block. One way would be splitting into three functions:
fn start_restore() which goes up to line 71 fn process_block() which returns the number of bytes that the CLI would use to generate output, fn complete_restore() which takes care of the rest after the blocks are processed.
Would also involve passing the guard back and forth a bit. So that way is probably doable but the only advantage would be being able to use a progress bar instead of displaying each line. But since it turns out that the existing structure already has level-based logging and is well thought out (such as setting the level based on the command and only applying to a single module) it feels like overkill. Thoughts?