search

h2r / pomdp-py

A framework to build and solve POMDP problems. Documentation: https://h2r.github.io/pomdp-py/
MIT License
216 stars 50 forks source link

An implementation of CC-POMCP #68

Open troiwill opened 6 months ago

troiwill commented 6 months ago

This pull request implements the cost-constrained POMCP (CC-POMCP) algorithm and its dependencies. The algorithm is in algorithms/ccpomcp.p*, while the dependencies are in the framework/generalization.p* and utils/cvec.p* files. This pull request also proposes a generic model, called a ResponseModel, and a corresponding output, called a Response. The name "response" comes from the notion of independent and dependent variables, where a response (reward, cost, etc.) depends on the interaction with the real or simulated environment. Thus, a response model is a wrapper for more specific models, such as reward and cost models (and any others that will follow in the future). By extension, a response is a wrapper for the reward, cost, etc.

The framework/generalization.p* files contain

The pull request has the following:

  1. Implementation of the CCPOMCP and its dependencies,
  2. Some updates to pre-existing algorithms to reduce copied-and-pasted code,
  3. Some updates to the code for rocksample,
  4. Implementation of the rocksample from the CC-POMCP paper,
  5. Test script for Vector operations (in test_util_vector_ops.py), and
  6. Passes for the test_all.py script.
zkytony commented 6 months ago

Thanks for the effort. Much better organized than before. Will take a pass soon. One thing - the CIs are failing. Could you fix this first? I just merged this PR that will make CI actions trigger when you update the source branch of this PR. Please update your branch with the latest changes in main.

troiwill commented 5 months ago

I merged the changes from main into ccpomcp. Let me know if I need to update anything.

zkytony commented 5 months ago

Great. Looks like CIs passed except for pre-commit. I will review the code soon. ~I should also enable CIs for all branches..~. It should be enabled, after you rebase/merge latest changes in main. @troiwill