lucaong / cubdb

Elixir embedded key/value database
Apache License 2.0
556 stars 23 forks source link

Export/Import #39

Closed Hermanverschooten closed 3 years ago

Hermanverschooten commented 3 years ago

What would be the best way to export all data and later import it again. I am working on a nerves app, and would like to give my user the opportunity to make backups of the data.

I am currently using this for the backup: CubDB.select(:db) |> :erlang.term_to_binary()

But I do not find how to re-import this. I thought about emptying the db and then putting all data back, but there seems to be no easy way to empty the db either.

Any help would be appreciated.

lucaong commented 3 years ago

Hi @Hermanverschooten , sorry for the late response.

If you want to backup the whole database, the easiest thing is to just copy the data file. You can get the path to the latest data file by calling CubDB.current_db_file/1.

Importing the backup is just a matter of cleaning up the data folder from any file, and copying or moving the one you backed up in it.

Of course, this only works if you want to backup/restore the whole database, not just selected entries.

lucaong commented 3 years ago

Closing this issue for now, as I think that the question is answered. Feel free to comment on it if necessary.

Hermanverschooten commented 3 years ago

For posterity, this is how i currently do backup/restore:

  def backup(conn, _params) do
    data =
      CubDB.select(:db)
      |> :erlang.term_to_binary()

    conn
    |> send_download({:binary, data}, filename: "backup.bin")
  end
@impl true
  def handle_event("restore", _params, socket) do
    consume_uploaded_entries(socket, :config, fn
      %{path: path}, _entry ->
        log("Creating temporary database")
        File.rm_rf("/tmp/db")
        CubDB.start_link("/tmp/db", name: :db2, auto_file_sync: true, auto_compact: true)
        log("Created")
        log("Reading config")
        txt_config = File.read!(path)
        log("Config read, parsing")

        case :erlang.binary_to_term(txt_config) do
          {:ok, config} ->
            log("Config parsed - OK")
            log("Uploading to temporary database")

            for {k, v} <- config do
              CubDB.put(:db2, k, v)
            end

            log("Upload complete,stopping temporary db")
            CubDB.stop(:db2)
            current_db = Application.get_env(:watchit, :db_path, "/data/db")
            File.rm_rf(current_db <> ".bak")
            File.rename!(current_db, current_db <> ".bak")
            log("Moved current db")
            File.rename!("/tmp/db", current_db)
            log("Move temporary")
            CubDB.stop(:db)
            log("Restarted db to get the changes, waiting")
            wait_for(:db)
            log("Db restarted")

          _ ->
            log("Config not valid, terminating restore")
            CubDB.stop(:db2)
        end
    end)

    {:noreply,  socket}
end

defp wait_for(process) do
    wait_for(nil, process)
end

defp wait_for(nil, process) do
    Process.sleep(100)

    process
    |> Process.whereis()
    |> wait_for(process)
end

defp wait_for(_, _), do: :ok