Open vmx opened 1 year ago
Currently tree-c and tree-r are constructed concurrently since for CC sectors and when the data is present on the sealing machine they utilize the same data. This is important in those cases since the data movement is a significant part of the cost.
In PC2 tree-c is much more expensive than tree-r so in the scenario where the data is remote the potential savings from skipping tree-r is minimal.
We'll add this one to the list for future updates.
Currently tree-c and tree-r are constructed concurrently
Oh that's interesting. I though they would be built sequentially. But thinking about it, it makes sense (given the on disk data layout). Then indeed the TreeR costs are pretty small. I guess you would have a little more memory on the GPU available to do the TreeC, but that probably doesn't really matter.
On a side-channel we agreed on adding a option to not write the TreeR tree files to disk.
When looking at the PC2 API, it looks like you cannot run PC2 without building TreeR. It would be great to have an option that only generates the TreeC and the sector key. The sector key is what currently is called the
sealed-file
.Background on why this would be nice. When doing data sectors (as opposed to CC sectors), it could be that the data files are not accessible from the machine that runs the PC2. Therefore, you would run the PC1 process as you currently do and disable the TreeC building. You would then use the
sealed-file
as sector key and move that to where the data file is. You'd the run the encoding step and TreeC building there, the existingtools/tree_r.cpp
to do that.I know that building TreeC is just a small fraction of the work done during PC1/PC2, but not doing things that are not needed are always a win.
This request is not urgent, but it will eventually be needed. I though I open the issue early on, so that you can schedule it into your work stream easily.