digital-fabric / extralite

Ruby on SQLite
http://www.rubydoc.info/gems/extralite
MIT License
253 stars 8 forks source link

add support for backups #11

Closed sitano closed 1 year ago

sitano commented 1 year ago

add support for backups.

PR lacks the support of 2 features:

if you are willing to add this I can fix those issues and finish the fork. otherwise just close it.

    b = Extralite::Backup.new(@dst, "main", @src, "main")
    ok = SQLITE_OK; ok = b.step(1) while ok == SQLITE_OK
    assert_operator 0, :<, b.pagecount
    assert_equal 0, b.remaining
    b.finish
    raise "omg: #{ok}" if ok != SQLITE_DONE

or

    b = Extralite::Backup.new(@dst, "main", @src, "main")
    begin
      b.step(1)
    end while b.remaining > 0
    assert_operator 0, :<, b.pagecount
    b.finish
noteflakes commented 1 year ago

I get the value of providing some feedback to the user while the backup is running (via sqlite3_backup_remaining), but I think the API can be much simpler. I suggest the following for basic usage:

db = Extralite::Database.new('foo.db')
db.backup('bar.db')

...where Database#backup is a blocking operation that returns when the backup is done. If we need to provide some feedback on progress, we can do the following:

db.backup('bar.db') { |total, remaining| ... }

Any error from the sqlite3 backup API while performing the backup will cause an exception to be raised, just like all other Extralite APIs. I wasn't familiar with this corner of the sqlite3 API. Note also that a backup can also be made using VACUUM INTO. So:

class Extralite::Database
  def backup_using_vacuum_into(fn)
    query("VACUUM INTO '#{fn}'")
  end
end

What do you think?

sitano commented 1 year ago

@ciconia I agree entirely. do you think it is better to implement the improved API in pure C or provide a Ruby-level db.backup() version on top of what we have?

noteflakes commented 1 year ago

I think it would be better to keep the implementation in pure C. I'll try to do this next week, unless you want to do it yourself. Let me know, thanks @sitano.

sitano commented 1 year ago

@ciconia yeah, go ahead. I will happily reuse the results. If you would not have time, I can assist.

are you thinking of implementing a sleep-retry mechanism for backing up live databases? like in the example

if (rc==SQLITE_BUSY || rc==SQLITE_LOCKED) {
  sqlite3_sleep(rand() % 250);
}

I don't know if it makes sense to make it configurable.

noteflakes commented 1 year ago

That's possible, perhaps with a shorter max period (250ms is very long). No need to configure this, for the moment at least.