Converts all from numpy to torch. The backprop has been tested local, it is running, but can't verify algorithm correctness.
It is very slow. Reason is the training loop is a 4-nested loop: steps, epochs, batches, and dynamics models. Currently, they are all sequential. And a single CPU core is being used. Some pending actions:
We need parallelize the loop through the dynamics model, they are running sequentially but there is no dependency between them. Each Dynamics model can be a different actor, with its own set of resources. See Ray as an alternative to parallelize through actors: https://docs.ray.io/en/master/actors.html
Switch print calls to log.info or log.debug - so user can control level of verbosity. I've added a few more, they can be changed to the lowest level of logging (logging.debug)
Migrate from command line to python config dictionaries - follow standard used in nupic.research
ps1: My local environment is python 3.7, hence pytorch is limited to 1.71. For that I couldn't use t.ravel() and had to use t.flatten() instead. The difference is ravel() only uses a view(), so it points to the same place in memory, while flatten create a copy. When upgrading the environment we should switch those.
ps2: PR also includes some style fixes to be compliant to PEP8, including variable naming, spacing, copyright notice and import order. I've been applying the style fixes gradually as I work with the different sections of the code.
Converts all from numpy to torch. The backprop has been tested local, it is running, but can't verify algorithm correctness.
It is very slow. Reason is the training loop is a 4-nested loop: steps, epochs, batches, and dynamics models. Currently, they are all sequential. And a single CPU core is being used. Some pending actions:
ps1: My local environment is python 3.7, hence pytorch is limited to 1.71. For that I couldn't use
t.ravel()and had to uset.flatten()instead. The difference is ravel() only uses a view(), so it points to the same place in memory, while flatten create a copy. When upgrading the environment we should switch those.ps2: PR also includes some style fixes to be compliant to PEP8, including variable naming, spacing, copyright notice and import order. I've been applying the style fixes gradually as I work with the different sections of the code.