search

takuseno / d3rlpy

An offline deep reinforcement learning library
https://takuseno.github.io/d3rlpy
MIT License
1.33k stars 243 forks source link

[REQUEST] support of new algorithms #45

Open weiguowilliam opened 3 years ago

weiguowilliam commented 3 years ago

This repo is really great and helpful! I enjoy working with that. Thank you for your great work:)

I find your repo so potential and I wonder if you have a plan to support some new algorithms such as Deep Variational Reinforcement Learning or if it possible to provide some API to use the algorithms implemented by others under the framework of d3rlpy?

Thank you!

takuseno commented 3 years ago

@weiguowilliam Thank you for the request! And, I appreciate your comment! I'm considering that d3rlpy continuously supports new algorithms if it's useful. I'll take a look at a paper you mention. Also, I'm looking for a way to provide a way to implement a new algorithm on user sides.

Thank you!

JayanthRR commented 3 years ago

Hi @takuseno for this wonderful repository. I want to know if you have any update on enabling users to implement custom algorithms? If there is no way currently, can you provide me pointers on what can be done from the user side to modify any existing algorithm? Thank you.

takuseno commented 3 years ago

@JayanthRR Hi, thanks for your comment. Unfortunately exposing APIs for custom algorithms is not high priority issue now. But, you can implement your own algorithms on your end. The algorithm classes have two-level abstractions:

high-level

https://github.com/takuseno/d3rlpy/blob/f7c7e8e77282500e3d5602dc220b3efdcb901425/d3rlpy/algos/sac.py#L24

low-level

https://github.com/takuseno/d3rlpy/blob/f7c7e8e77282500e3d5602dc220b3efdcb901425/d3rlpy/algos/torch/sac_impl.py#L34

You can implement new ones by referring these parts.

JayanthRR commented 3 years ago

Thanks for the pointers. I am sorry I have a very basic question. Once I implement any algorithm, how can I build it or use it? Here is what I did. I installed the requirements.txt (without installing the d3rlpy api), and tried to import from d3rlpy.algos import CQL but I got an error as follows 

ModuleNotFoundError: No module named 'd3rlpy.dataset'

What is the procedure to implement an algorithm and build it? Do I have to install with setup.py everytime I make changes to my code? Thanks in advance.

takuseno commented 3 years ago

The error is raised because dataset.pyx, which is Cython module, is not compiled. All you have to do is:

$ git clone https://github.com/takuseno/d3rlpy
$ cd d3rlpy
$ pip install -e .
$ # you're ready to go

Once you install the modules via -e option, you don't have to pip install any more.