First Release

[getnamo]

Todo:

• [x] BP <-> Python Communication
• [x] successfully import tensorflow in unreal engine
• [x] Texture conversion to float arrays
• [x] Proof of Concept
• [x] Presentation
• [x] Encapsulate pip site packages and python basic modules in python plugin path
• [x] Encapsulate tensorflow scripts in plugin path
• [x] pip from python working
• [x] automatic pip dependency resolution
• [x] PoC API: Init, SendTensor-> OnResult
• [x] Basic log wrapping for print() -> ue.log()
• [x] basic threading support to make Tensorflow operations non-blocking
• [x] mainthread callbacks with results
• [x] verify running multithreaded calls multiple times works with core TF api
• [x] basic API: Set script, Train, Run & event callbacks: OnInitComplete, OnTrained, OnResult
• [x] update to TF 1.0.0 for stable API
• [x] misc bugfixes
• [x] pull unrealpythonplugin latest master changes
• [x] update SocketIO to 0.5.2
• [x] test external drag and drop
• [x] fix drag and drop problems: pip initialization, uninstall crash
• [x] fix path inconsistencies (likely relative paths)
• [x] confirm https://github.com/getnamo/tensorflow-ue4-examples/issues/6 is fixed
• [x] confirm https://github.com/getnamo/tensorflow-ue4-examples/issues/5 is fixed
• [x] confirm https://github.com/getnamo/tensorflow-ue4-examples/issues/4 is fixed
• [x] fix https://github.com/getnamo/tensorflow-ue4-examples/issues/7
• [x] fix https://github.com/getnamo/tensorflow-ue4-examples/issues/8
• [x] auto-resolve pip if project moved
• [x] auto-resolve pip if we delete site-packages
• [x] documentation update
• [x] second presentation
• [ ] example videos

Current focus: cleanup and fixes

[getnamo]

threads possible with https://github.com/getnamo/UnrealEnginePython#threading-experimental and _thread but its currently breaks our encapsulation loading.

They also only are able to run tensorflow once, the graph seems to get corrupted or nil referenced otherwise.

Performance however is excellent and shows great promise of being able to train networks while in play.

[getnamo]

Log wrapping possible using e.g.

class Logwrapper(object):
    def __init__(self):
        self.terminal = sys.stdout

    def write(self, message):
        ue.log(message)

    def flush(self):
        ue.log("")

and

sys.stdout = Logwrapper()
sys.stderr = Logwrapper()

[getnamo]

pip install from python on background thread appears to be working, need to add pip dependency json file to plugins and auto-resolve on editor start, or add a command/button to resolve dependencies

[getnamo]

pip auto dependency resolution works. Plugins now simply call e.g.

FUnrealEnginePythonModule::Get().AddPythonDependentPlugin("TensorFlow");

In their StartupModule() to both add their Content/Scripts to the python system path and auto resolves the upymodule.json found in that folder

Tensorflow example upymodule.json:

{
    "name" : "tensorflow-ue4",
    "pythonModules": {
        "tensorflow": "0.12.1"
    }
}

[getnamo]

callback api works now with

ue.run_on_gt(function, <optional>args)

which schedules a callback to come back on the game thread. Also added convenience threading functions through upythread.py

import upythread as ut
ut.run_on_bt(function, <optional>callback)

which wraps threading->thread into a one function fire and forget.

[getnamo]

with TF 1.0.0 release, it is time to link up the API to the blueprint layer (basic for now, later: fine control BP).

• [x] Explore new tf.keras submodule Edit: nevermind, it will come out in TF 1.1 as tf.contrib.keras and then as tf.keras in TF1.2. Good timing, it gives focus for pure tf for now.

[getnamo]

TensorFlowComponent is now fully encapsulated with very basic API:

Variables: -TensorFlowModule (name of your python script module with train and runJsonInput) -Multithreaded Training toggle -Verbose log toggle -Train on begin play toggle

Functions: -Train -TensorInput

Callback Events: -OnTraining Complete -OnResults

Test another TF model with this setup and push first release.

Critically we will need a way to save/load the model before non-proof of concept scripts should be tested.

[getnamo]

drag and drop isn't working as expected yet, a bit more work required.

[getnamo]

https://github.com/getnamo/tensorflow-ue4/releases/tag/0.1.2-pre seems to be a working drag and drop release, cleanup first mnist example and release them as a package along with early documentation

[getnamo]

Two more issues should be fixed before closing this issue: -https://github.com/getnamo/tensorflow-ue4-examples/issues/7 -https://github.com/getnamo/tensorflow-ue4-examples/issues/8

[getnamo]

Need to wrap up documentation. TF basics -> beginner mnist in a video.

[getnamo]

Released. Videos will come when qlearning example is ready.