Is there a way to clean deleted data?

[dangershony]

This thread we found that remove is only a logical operation.

Is there a way to "clean" deleted data from disk?

[dangershony]

https://github.com/stratisproject/StratisBitcoinFullNode/issues/2414 Tracked here

[hhblaze]

tran.RemoveAllKeys("tblName", true); - recreates table with the same name and without data. It is possible to copy the rest of the data (not deleted keys) into the temporary table and then copy data back into the recreated table before removing all keys. Also it is possible to delete table completely Scheme.DeleteTable and then Scheme.RenameTable (that temporary table with the rest of data, rename it to the original name). Second approach can economize time on copying data back, but has a small drawback: some reads can fail, at that moment when file original table-file is deleted (OS level, takes some milliseconds) and table is renamed after that (some microseconds). This problem is solvable by trick - putting small synchronizer on reading from that table into DataAccessLayer, that will delay returning data until rename is finished. Writes at this moment will be synchronized by transaction - so, no worry.

Write me down if you need further investigation.

[hhblaze]

There is also an old feature described in documentation as [20130613] Full tables locking inside of transaction.

This can give for the transaction exclusive access to the table among the other transactions. That can be used to clean data with tran.RemoveAllKeys("tblName", true); and copy the rest of data back from the temporary table.

Note, that if you start to work with the table in terms of "exclusive/shared" access, then all queries to that table must use that term.

[hhblaze]

From discussion year 2013:

Reading-Session decoration can be combined with SynchronizeTables instruction.
But SynchronizeTable has no connection to Reading-Session.

Proc1 - inserting keys:

using (var tran = engine.GetTransaction())
            {
                using (DBSession ses = new DBSession(eDBSessionLockTypes.SHARED, "t1","p*"))
                {
                     //can be added if we want to change one of those tables.
                     tran.SynchronizeTable("t1","p*");

                    tran.Insert..."t1"
                    tran.Insert..."p127"

                    tran.Commit();
                }
            }

Proc2 - reading keys:

using (var tran = engine.GetTransaction())
            {
                using (DBSession ses = new DBSession(eDBSessionLockTypes.SHARED, "t1"))
                {

                    foreach(var row in tran.SelectForward("t1")
                      ....
                }
            }

Proc3 - Removing Keys from t1 with the table recreation:

using (var tran = engine.GetTransaction())
            {
                using (DBSession ses = new DBSession(eDBSessionLockTypes.EXCLUSIVE, "t1"))
                {

                    tran.RemoveKeys("t1",true);
                    tran.Commit();
                }
            }

[hhblaze]

Also, may be useful ref.docu [20130608] Restoring table from the other table.