add to gfix -progress switch to show progress of validation operation

[livius2]

Please add to gfix -progress switch to show progression of operation validation of the database if possible.

It can be log into firebird.log by eg. every 10% to not spam too much the log.

Now when we must verify database we do not know where we are in the process...

[mrotteveel]

The progress of what operation?

[livius2]

I mean progress of e.g. -validate -full.

[mrotteveel]

I don't like the suggestion that it is output to firebird.log. If you specify an option like that in a commandline tool, I would expect progress to be output to the standard out (or error stream), not to some log file somewhere else.

[livius2]

But today gfix write log about errors/warnings to firebird.log. Maybe another switch or change in current behavior that gfix always write to std out. Now when we start validation of multiple databases in the same time, log is mixed, so i like your proposition a lot :-)

[hvlad]

Did you considered online validation ?

[livius2]

Did you considered online validation ?

What about?

[mrotteveel]

But today gfix write log about errors/warnings to firebird.log.

That (IMHO) design mistakes where made in the past, does not mean we should perpetuate them. If a user ask for progress, I expect them to want to see it. Otherwise we could just log the progress in firebird.log always, without having to explicitly ask for it.

[livius2]

@mrotteveel I have modified my comment probably when you were writing ;-) Look above "so i like your proposition a lot :-)"

[hvlad]

Did you considered online validation ?

What about?

https://firebirdsql.org/file/documentation/release_notes/html/en/2_5/rnfb25-apiods-api.html#rnfb25-apiods-api-onlinevalidation

[livius2]

@hvlad but what about it? It report progress? Even if so it cannot validate all situation like exclusive access with gfix do? So still gfix progress will help a lot. Yesterday I had to check a very large database and the only thing I could estimate was to take the size of the database and divide it by looking at readings per second from the disk ;-)

[hvlad]

@hvlad but what about it? It report progress?

Try and see. I consider - yes, it is reported progress. But you could have another opinion.

Even if so it cannot validate all situation like exclusive access with gfix do?

It validates most cases. The approach could be extended to the offline validation, if required. Or we can add some events for the trace, like it is done for sweep.

So still gfix progress will help a lot. Yesterday I had to check a very large database and the only thing I could estimate was to take the size of the database and divide it by looking at readings per second from the disk ;-)

gfix way to run validation physically can't report any kind of progress. All gfix does is just attach and detach. All validation activity happens inside attach, by the engine. Online vailidation was implemented as a service that could start task and get results (progress).

[livius2]

Try and see. I consider - yes, it is reported progress. But you could have another opinion.

Do not get my question as some criticism i simply did not catch the context when you wrote about online validation.

It validates most cases. The approach could be extended to the offline validation, if required. Or we can add some events for the trace, like it is done for sweep.

Is there some document about comparision what is checked by online validation vs gfix -validate - full? It will be very helpfull information.

All gfix does is just attach and detach

but it report how many errors, warnings was is it from detach process only?

[AlexPeshkoff]

On 12/19/23 14:55, Vlad Khorsun wrote:

gfix way to run validation physically can't report any kind of progress. All gfix does is just attach and detach. All validation activity happens inside attach, by the engine. Online vailidation was implemented as a service that could start task and get results (progress).

As the simplest (but restricted and bad in all aspects except simplicity) solution we can add progress report at least when gfix is invoked as service. That's true easy. If utility is much wanted to also report progress it's possible via set of callbacks and special OP in remote protocol - approximately same actions like when dbcrypt key is delivered from client during attach. Or change approach of offline validation - run it not on attach but using separate API call with existing attachment.

[aafemt]

If utility is much wanted to also report progress it's possible via set of callbacks and special OP in remote protocol

Can't gfix utility simply use the service instead?

[AlexPeshkoff]

On 12/19/23 15:26, Dimitry Sibiryakov wrote:

If utility is much wanted to also report progress it's possible
via set of callbacks and special OP in remote protocol

Can't gfix simply use the service instead?

When working with fresh enough server - why not?

[aafemt]

Utilities usually are supposed to work with servers of the same version. Ability to work with servers of different versions is just a nice extra feature.