[WIP] Synthetic simulation, environment, and dataset for RTB

[aiueola]

Type of change

• [x] New feature
• [ ] Bugfix
• [ ] hotfix
• [ ] Other

Description

• What has been changed?

Implemented synthetic simulation, environment, and dataset modules for Reinforcement Learning (RL) in Real-Time Bidding (RTB).

• How the logic works?

Please refer to the description in the following files. Also, feel free to ask any questions.

1. Constrained MDP definition in RL: _gym/env/rtb.py
2. How environment interacts with an RL agent and how each auction outcome is calculated: _gym/env/rtb.py
3. Ground-truth winning function and CTR/CVR definition: _gym/simulator/function.py
4. Parameters in the simulator and the environment: _gym/simulator/rtb_synthetic.py and _gym/env/rtb.py, respectively
5. Usage of the environment and the dataset modules: _gym/env/rtb.py and _gym/dataset/synthetic.py, respectively.
6. Quickstart code (rough version): examples/quickstart/rtb_synthetic.ipynb

Checklist

• [ ] pass unit test (or unnecessary)
• [x] no errors on newly made test cases
• [x] no errors on existing test cases
• [x] applied black formatter
• [x] no errors on flake8
• [x] no warnings
• [x] work in progress

Comments

• Please refer requirements here.
• Work in progress, haven't yet perform error checks and debugging. I will also add some references later.
• Appreciate any kind of suggestions including more reasonable naming of the parameters and the description, efficient calculation procedures, coding styles, etc.

[k-kawakami213]

@aiueola (cc @kojikawamura ) こちら動作検証ってどこかに書かれています？プルリクみたのですが，よく分からず... 実際にシミュレーション含めて動かしてみたいです．

[aiueola]

@k-kawakami213 (cc: @kojikawamura) ありがとうございます！

先程requirementsを追加したので，そちら見ていただけたらと思います．

Please refer requirements here.

あと，かなりラフな感じになっていますが，ipynbにdataset moduleの使い方を書きました．

6. Quickstart code (rough version): examples/quickstart/rtb_synthetic.ipynb

もう少し詳細な引数の定義などについてはこちらを見ていただけたら嬉しいです．

5. Usage of the environment and the dataset modules: _gym/env/rtb.py and _gym/dataset/synthetic.py, respectively.

よろしくお願いします！