search

JuliaReinforcementLearning / ReinforcementLearning.jl

A reinforcement learning package for Julia
https://juliareinforcementlearning.org
Other
582 stars 114 forks source link

Next Release Plan (v0.11) #614

Closed findmyway closed 7 months ago

findmyway commented 2 years ago

Goal

Improve the interactions between ReinforcementLearning.jl and other ecosystems in Julia.

Why is it important?

In the early days of developing this package, the main goal is to reproduce some popular (deep) RL algorithms. It's still important to keep adding new emerging algorithms into this package. But as an engineer, I always think the higher impact is achieved only when users really apply those powerful RL algorithms to the problems they are interested in. In recent years, many important packages across different domains were developed in Julia and the whole ecosystem improved a lot. Although the interfaces defined in this package are loose and flexible, people are still unsure how to use this package due to lacking concrete examples. Adding more examples and removing some restricted assumptions will greatly encourage more people to try this package. On the other hand, doing so will also improve the quality of this package.

Potential breaking changes

The most important change would be decoupling the training data generation and policy optimization. The state is assumed to be a tensor by default in many cases. This is the main blocking issue when interacting with many other packages. Besides, the async training pipeline will not only improve the performance of existing algorithms on a single node but also provide the foundation of large scale training in future releases (possibly in v0.12)

Key issues to be addressed

Following are some of the existing issues on the top of my mind. Please raise new ones if you wish to be addressed in the next release.

Environments

Refactor Existing Policies

Add New Policies

Training pipeline

Documentation

Utils

Timeline

I'm not sure I can fix them all. But at least I'll take a deep look into them and then tag a new release at the end of this quarter (around the end of June 2022).

findmyway commented 2 years ago

@JuliaRegistrator register

JuliaRegistrator commented 2 years ago

Registration pull request created: JuliaRegistries/General/61734

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.10.1 -m "<description of version>" 2e1de3e5b6b8224f50b3d11bba7e1d2d72c6ef7c
git push origin v0.10.1
findmyway commented 2 years ago

@JuliaRegistrator register subdir=src/ReinforcementLearningBase

JuliaRegistrator commented 2 years ago

Registration pull request created: JuliaRegistries/General/63479

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningBase-v0.10.0 -m "<description of version>" b58c7c4e7c7ecb141df135023dd284cab487c6e0
git push origin ReinforcementLearningBase-v0.10.0
findmyway commented 2 years ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 2 years ago

Registration pull request created: JuliaRegistries/General/63531

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.7.0 -m "<description of version>" 06aa63e06f255654e5ae967ec4737cafd4d77908
git push origin ReinforcementLearningEnvironments-v0.7.0
findmyway commented 2 years ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 2 years ago

Registration pull request updated: JuliaRegistries/General/63532

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.0 -m "<description of version>" 0f00ba42a00aacbf7cd8a31ab6b6e3c4cf14c1ef
git push origin ReinforcementLearningCore-v0.9.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningBase

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/78892

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningBase-v0.11.0 -m "<description of version>" 107ceec40976d8023bc2b17a82861e4a240b7dee
git push origin ReinforcementLearningBase-v0.11.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningBase

JuliaRegistrator commented 1 year ago

Registration pull request updated: JuliaRegistries/General/78892

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningBase-v0.11.0 -m "<description of version>" b38b9f30c5750bc0d522f423d910750efb6f507b
git push origin ReinforcementLearningBase-v0.11.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/78898

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.1 -m "<description of version>" a6216b0b6e21fda86b72611228698455ce515b22
git push origin ReinforcementLearningCore-v0.9.1
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 1 year ago

Registration pull request updated: JuliaRegistries/General/78898

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.1 -m "<description of version>" d49bdd226ccc7e89ec0ebf8eaffee245a5e94346
git push origin ReinforcementLearningCore-v0.9.1
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/78919

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.7.1 -m "<description of version>" bbce4f10c1863b3cae4a98f0a0b1595c34606342
git push origin ReinforcementLearningEnvironments-v0.7.1
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 1 year ago

Registration pull request updated: JuliaRegistries/General/78919

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.7.1 -m "<description of version>" 9b1c2737044736d642d54b10917b038cd5f0bd73
git push origin ReinforcementLearningEnvironments-v0.7.1
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningExperiments

JuliaRegistrator commented 1 year ago

Error while trying to register: Version 0.1.4 already exists

jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningZoo

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/78985

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningZoo-v0.6.0 -m "<description of version>" 589fa9efc8130372bbde0f2733dbaf3ab68f15e0
git push origin ReinforcementLearningZoo-v0.6.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningExperiments

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/78987

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningExperiments-v0.2.0 -m "<description of version>" d40dde1d4804965265eb9eeabab2fec500f2f490
git push origin ReinforcementLearningExperiments-v0.2.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningExperiments

JuliaRegistrator commented 1 year ago

Registration pull request updated: JuliaRegistries/General/78987

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningExperiments-v0.2.0 -m "<description of version>" 25d9349c9fe20956419ea6341baf01a7237361bf
git push origin ReinforcementLearningExperiments-v0.2.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore branch=patch-v10.1.1

JuliaRegistrator commented 1 year ago

Error while trying to register: Version 0.8.12 already exists

jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/79948

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.2 -m "<description of version>" dae8437888ca48701b686a5f67e35d20dc7ab375
git push origin ReinforcementLearningCore-v0.9.2
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/82795

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.3 -m "<description of version>" 18714fc5cc13cab9c77e31f01c4acb5d7d1d97a4
git push origin ReinforcementLearningCore-v0.9.3
JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/82796

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.7.2 -m "<description of version>" 18714fc5cc13cab9c77e31f01c4acb5d7d1d97a4
git push origin ReinforcementLearningEnvironments-v0.7.2
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningBase

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/82803

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningBase-v0.11.1 -m "<description of version>" af2b158b9276c6960bf152839aa571572f4ed88a
git push origin ReinforcementLearningBase-v0.11.1
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/82814

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.9.4 -m "<description of version>" 9961fe66bda28cebe47933a40db3fd3f27b38a2f
git push origin ReinforcementLearningCore-v0.9.4
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 1 year ago

Registration pull request updated: JuliaRegistries/General/82796

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.7.2 -m "<description of version>" 9961fe66bda28cebe47933a40db3fd3f27b38a2f
git push origin ReinforcementLearningEnvironments-v0.7.2
jeremiahpslewis commented 1 year ago

Note: I've closed all issues older than +/- 1 year without review, if something needs to be reopened, feel free!

jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningBase

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/84074

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningBase-v0.12.0 -m "<description of version>" 395f23fa94e1f5d5dba342c19feb7b4f892030d1
git push origin ReinforcementLearningBase-v0.12.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningCore

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/84075

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningCore-v0.10.0 -m "<description of version>" 395f23fa94e1f5d5dba342c19feb7b4f892030d1
git push origin ReinforcementLearningCore-v0.10.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments

JuliaRegistrator commented 1 year ago

Registration pull request created: JuliaRegistries/General/84076

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a ReinforcementLearningEnvironments-v0.8.0 -m "<description of version>" 395f23fa94e1f5d5dba342c19feb7b4f892030d1
git push origin ReinforcementLearningEnvironments-v0.8.0
jeremiahpslewis commented 1 year ago

@JuliaRegistrator register subdir=src/ReinforcementLearningEnvironments