JSON input processing on background thread?

[bernhardrieder]

Hey!

I'm using the TF plugin for my thesis project and was annoyed of a delay during the json input processing and discovered that you don't use a BT thread for the processing like you do for training and setup. Therefore, I changed the code like you implemented it for the training and setup and it works fine! The only problem is that I can't forward arguments to the background thread with ut.run_on_bt() which is the reason why I needed to introduce a new variable with the current input -> which would cause problems if it'd be overriden by a new value before its actually processed. But that isn't a problem in my case since I double check if I'm currently processing a JSON input in my derived API class.

Anyways, is there a reason why you haven't implemented that? If there's no proper reason then I'd suggest to include that in the TF component to avoid game freezes :)

[getnamo]

In my use cases I never found the inference step to take that long, so I thought it would make sense for it to remain on the game thread. It seems my models are too small :D

It shouldn't be too hard to extend run_on_bt to add an extra wildcard parameter, the background function is defined here: https://github.com/getnamo/UnrealEnginePython/blob/master/Content/Scripts/upythread.py and the json_input function as you've found is defined here: https://github.com/getnamo/tensorflow-ue4/blob/master/Content/Scripts/TensorFlowComponent.py#L101 but you'd likely need to have the data resolve back again on the game thread (since you can't control what the user does in blueprint and making/destroying UObjects on background threads is a nono), which would use the same old way as ue.run_on_gt. That callback function already has a wildcard parameter.

I'll mark this as an enhancement and I'll try to get a tickbox option background thread operation for json_input ~ when I do the 4.20 update. If you feel confident enough however consider making a pull request, contributions are always welcome :)

[bernhardrieder]

Cool, seems like it'd easy to implemented! Nah, it's not that big... I'm reloading a new trained model after a while and don't want to interrupt the json input processing so I check that at the beginning of the input and reload the model - which causes a bigger freeze. But there's a regular freeze of 2 seconds which may be because it's a generative model and generates stuff instead of doing a simple prediction.

[getnamo]

Support added in https://github.com/getnamo/tensorflow-ue4/releases/tag/0.10.0. Enabled by default if using multi-threading.

[AZE38]

hello , sorry but i m in ue 4.23 , tensorflow-cpu, and when i want send Json texture or float array 20x20 or more , my game freeze again , even if multi threading is enable . there is any way for solve this ? or it's me who is not using in the right way ?

ps : without verbose log , its better, but there again little freeze

[getnamo]

It's likely your conversion to float array is being called on game thread which may cause the hitch, check what you're using to convert into float array