Fix for editor freeze upon assets reload

[dkondrashkin]

Sometimes editor process becomes unresponsive upon importing *.fbx file just exported from, say, 3ds Max or Blender.

STR: 1) Start editor and open some level with an asset present on it; 2) Open this asset in Blander (import from FBX); 2a) (optional) Introduce some changes to the asset; 3) Export changes to FBX and observe asset being "hot reloaded" into the editor.

Result: Sometimes editor freezes.

What happens: When processing changed asset AssetProcessor issues messages consumed by editor. These messages after being handled end up calling AssetCatalog::AssetChanged(...) which subsequently calls AssetManager::ReloadAsset(...). It turns out, that under certain circumstances LmbrCentral::MeshaAssetHandler (could be any other handler dependent on tick) does not finish asset (say *.cgf) loading in time when another AssetManager::ReloadAsset(...) call happens. This leads to execution of the following code:

auto reloadIter = m_reloads.find(assetId); if (reloadIter != m_reloads.end() && reloadIter->second.GetData()->IsLoading()) { // This asset may already be loading stale data. Queue this requested reload until it's finished. AssetBus::QueueFunction([this, assetId]() {this->ReloadAsset(assetId); }); return; }

Unfortunately when this queued lambda executes upon the next AssetManager::DispatchEvents(...) call (in SystemTick handler) before asset handler is finished with the asset loading a livelock happens - AssetManager::ReloadAsset(...) call is being queued and executed on the same frame over and over again (while in our case LmbrCentral::MeshaAssetHandler waits on the binary semaphore at MeshAssetHandler.cpp@122).

Proposed fix: Queue the whole recursive call (with AssetBus::QueueFunction) on SystemTickBus ensuring that it will be dispatched on the next frame.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[AMZN-alexpete]

Thanks for the fix @dkondrashkin, we're taking a look at this.

[AMZN-alexpete]

Hi @dkondrashkin, this change has been accepted and will be included in a future version of Lumberyard, thanks!