After a .usd file is opened using the USDStage::Open interface, the USD file cannot be edited.

[FeiLongXu]

So how do I incrementally render this usd? Isn't the Reload interface useless?

[spiffmon]

Hi @FeiLongXu - can you please be a bit more specific in what you are trying to do, or are expecting? UsdStage::Reload() (or the corresponding "Reload" command in usdview) checks to see if any of the layers used by the scene have changed in their asset-store, and if so, re-loads them and adjusts the stage accordingly, which should trigger Hydra renderers to re-render. But if nothing has changed in the backing store, there will be no changes.

If you mutate data on the stage, Hydra renderers should also update in response, though it is possible there are bugs. If this is what you are seeing, describing exactly what you are doing, with an example, preferrably, would help us out.

Thanks, --spiff

[FeiLongXu]

My use scenarios are as follows:

1. Use Open to open a .usd file.
2. Start the rendering and wait until the rendering is complete.
3. After rendering, use another program to update the open usd file. In step 3, an error is reported, indicating that the usd file cannot be opened / edit

[spiffmon]

Thank you for the details! Windows specifically prevents one process from editing a file that another process has open for read, I believe. But in this case, that is actually a good thing. The price that we pay for concise, deferred access to big data in USD (specifically usdc) files, which allows us to render with low latency, is that we must keep the files open in the reading/consuming process, and they cannot safely be modified.

The easiest and least satisfying workflow is to make sure you close the stage after rendering it, if you are going to modify it in another process.

Next easiest, but possibly impractical depending on the size of your datasets, is to use usda files instead of usdc in such workflows. Because usda files are read into memory up-front in their entirety, you should not be prevented from editing the file in another process.

The next more involved workflow is to explicitly version your file as you edit a copy of it in your editing process, then update the ArResolverContext in your consumer process to point at the newer version, at which point Reload() should behave as you were hoping.

The most involved, but very slick, is the approach Omniverse takes, which is to have the file managed by a central server that receives edits from editing tools, merges them, and sends out updates to clients like your rendering application.

Hopefully something in there can get you unstuck!

[FeiLongXu]

Oh, thank you very much for your answer. I also found a problem:

1. Open the usda file. (This usda file references other usda files.), modify one of the files, and reload the stage. The result is what I expected, and can be rendered incrementally.
2. Perform the same operation for the usd file. After reload the stage, all the files are considered as dirty data and cannot be rendered incrementally. Sad~~

[spiffmon]

@FeiLongXu , in your second case, it's not (or at least, it shouldn't be) that all the files are considered dirty. It is simply that because your root usd layer is dirty, the composition engine must recompute the entire stage, which means querying each of the other layers for data - but that data is coming from in-memory caches, not from disk.

The other part of the price that you pay (if you think about it) for not reading and caching the entire usd file into memory, is that when its contents on disk change, you cannot perform diffs to see precisely what, if any, data changed - all you can tell the composition engine is that the contents of the file may have arbitrarily changed, so it needs to start over.

A thing we have discussed is providing some means to tell a session that instead of opening usdc files lazily, it should read them in greedily, as it does for usda files. In theory, this would address both of your issues, if we do it right. If you're not operating on giant shot-caches, this could be a reasonable thing to do.

If that seems interesting, please feel free to file an Issue requesting it, though I can't make any promises as to when we might get to it.

[jtran56]

Filed as internal issue #USD-6850

[FeiLongXu]

@jtran56 What does that mean? The problem has been solved? Or will it be resolved in the future?

[spiffmon]

It simply means the issue has been recorded in our internal bug tracking system. It has not yet been resolved.

On Sun, Aug 22, 2021 at 6:32 PM FeiLongXu @.***> wrote:

@jtran56 https://github.com/jtran56 What does that mean? The problem has been solved? Or will it be resolved in the future?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PixarAnimationStudios/USD/issues/1594#issuecomment-903378165, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPOU2DMK3NMOLJJAQ2CVEDT6GQJBANCNFSM5CDDTLGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- --spiffiPhone

[FeiLongXu]

Hello Everyone! Do I think the problem is due to my use error? My code is roughly as follows, omitting some code:

UsdStageRefPtr stage = UsdStage::Open(path);
UsdImagingGLEngine imagingEngine;
const UsdPrim& pseudoRoot = stage->GetPseudoRoot();
while(true) {
    while (!imagingEngine.IsConverged()) {
        imagingEngine.Render(pseudoRoot, renderParams);
    }
    wait(condition);//Other programs notify me that the usd file has changed.
    stage->Reload();
    imagingEngine.Render(pseudoRoot, renderParams);
}

[spiffmon]

Yes! Provided the files your second process is editing are not crate files, that code looks good!

[FeiLongXu]

@spiffmon So how do I open/load a .usd file to achieve the incremental rendering I want?

[spiffmon]

As I mentioned earlier, using the usda file format for the data that wants to be edited in another application simultaneously should make that code above work as intended.

[FeiLongXu]

@spiffmon The size of a file in the usda format is much larger than that in the usd format, which is unfavorable for network transmission. I found a way to do a reload after opening a usd file. After that, reload can be incremented normally. I don't know if it's a bug or if it just needs to be used like this. like this:

UsdStageRefPtr stage = UsdStage::Open(path);
**stage->Reload();**
UsdImagingGLEngine imagingEngine;
const UsdPrim& pseudoRoot = stage->GetPseudoRoot();
while(true) {
    while (!imagingEngine.IsConverged()) {
        imagingEngine.Render(pseudoRoot, renderParams);
    }
    wait(condition);//Other programs notify me that the usd file has changed.
    stage->Reload();
    imagingEngine.Render(pseudoRoot, renderParams);
}

[spiffmon]

I’m unsure what’s going on with the extra Reload() changing behavior, but what you are doing is not safe, because your other programs could be mutating the “shared” file while the renderer is trying to read data out of it. Doing this safely requires two-way communication, with your rendering program signaling to the others that it is going to sleep, and only then can they mutate files that may be used by the renderer, then signal to the renderer that it can resume.

Even so it can be tricky with networked file systems since sometimes it takes awhile for new data to propagate to other nodes in the network. But if you write out new files and atomically move them into place, you can minimize these kinds of problems.

On Thu, Aug 26, 2021 at 2:15 AM FeiLongXu @.***> wrote:

@spiffmon https://github.com/spiffmon The size of a file in the usda format is much larger than that in the usd format, which is unfavorable for network transmission. I found a way to do a reload after opening a usd file. After that, reload can be incremented normally. I don't know if it's a bug or if it just needs to be used like this. like this:

UsdStageRefPtr stage = UsdStage::Open(path); stage->Reload(); UsdImagingGLEngine imagingEngine; const UsdPrim& pseudoRoot = stage->GetPseudoRoot(); while(true) { while (!imagingEngine.IsConverged()) { imagingEngine.Render(pseudoRoot, renderParams); } wait(condition);//Other programs notify me that the usd file has changed. stage->Reload(); imagingEngine.Render(pseudoRoot, renderParams); }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PixarAnimationStudios/USD/issues/1594#issuecomment-906236527, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPOU2BBGM5TOQWS6UVM3ILT6YA37ANCNFSM5CDDTLGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- --spiffiPhone