Add a wrapper and an abr algorithm trained using A2C

[sagar-pa]

The wrapper in C++ simply creates a sub-process to which it passes back-and-forth the information about the last chunk and the corresponding best action to take. All of the code is similar to Pensieve, but includes more features from Puffer, is trained using a different strategy, and is implemented using PyTorch.

Notes for using the code:

Adding the abr algorithm in settings.yml:

abr: tara
abr_config:
  test_path: /path/to/third_party/abr_rl_test/test.py
  model_path: /path/to/third_party/abr_rl_test/models/tara_model.pt
cc: bbr

Virtual Environment setup: Although there is no setup.py, to run the code, create a python environment with python >= 3.7, install torch >= 1.8.1, and edit the shebang in third_party/abr_rl_test/test.py to point to the environment created.

EDIT: For heart's content, I've gone ahead and edited the environment variables by default in the python code using os. PyTorch should only execute with 1 thread per viewer.

[francisyyan]

Hi Sagar,

Thanks for sending the pull request. 1) Did you forget to make the file abr_rl_test/test.py executable (i.e., with file permission 755) in your Git repo? python_ipc.cc tries to execute test.py without explicitly calling python, and it fails to do so until I set the executable flag on test.py. 2) Could you give your ABR scheme a less general name ? deep_rl sounds too general. 3) Please merge your commits into a single one, and overwrite this PR or submit a new PR (multiple commits are fine per se but one commit makes more sense in your case).

Best, Francis

[keithw]

I'm a little confused by the PR -- the submodule URL (https://github.com/sagar-pa/abr_rl_test.git) seems to be a 404?

[sagar-pa]

Done. The settings have changed b/c of the name change, and I've update my original comment to reflect those changes. I'm sorry, I should have done that from the beginning.

@keithw The submodule URL leads to a 404 because it is a private repo for the time being. I have read your comment on the Google Group post and can add you to the repo for review, but if you feel that still leaves the code too secretive, we can try to work out a solution that feels right over email (along with my advisor).

[francisyyan]

I updated the submodule pointer in the PR to your most recent commit in https://github.com/sagar-pa/abr_rl_test.git, and have merged it into the master branch.