search

JuliaAcademy / Decision-Making-Under-Uncertainty

Decision making under uncertainty using the POMDPs.jl ecosystem taught by Robert Moss
228 stars 67 forks source link

Creating a Project.toml and Manifest.toml to clone environment #1

Closed 00krishna closed 3 years ago

00krishna commented 3 years ago

Hello. I just though it might be nice to include a Project.toml and Manifest.toml in the root of the repo. That way any dependencies for the notebooks are resolved once the environment is instantiated. Just a suggestion.

Looking forward to the rest of the videos.

mossr commented 3 years ago

Hey Krishna (it was fun chatting over YouTube chat)!

Pluto actually bakes the Manifest.toml/Project.toml into the notebook .jl file itself! See here for an example: https://github.com/JuliaAcademy/Decision-Making-Under-Uncertainty/blob/master/notebooks/1-MDPs.jl#L1630

This should take care of things (aside from the Pluto.jl requirement to open the notebooks themselves). I'm hesitant to turn this repo in to a standard Julia package as I think it's more of a collection of notebooks than a using DecisionMakingUnderUncertainty type package.

00krishna commented 3 years ago

Oh yes, you know I had not seen this feature before. Yeah, I just tried it out and the dependencies seemed to install just fine. It is a nice new feature--someone on julia slack just explained it to me :). Haha.

I just have to figure out if there is a way to load the dependencies in an environment so I don't have to keep reinstalling them each time. But otherwise you are totally correct. This is working as it is, so no need to add a separate Project.toml and Manifest.toml when you don't need it.

And yes, it was really nice talking to you at the premier. Adding this material to JuliaAcademy is invaluable. These notebooks are really helpful. I know it takes a lot of time and effort to build these things, so I really appreciate your sharing that with the entire community. I was trying to teach some of these RL ideas to some students while learning it myself. Trying to explain value iteration and policy iteration was tough with just static charts or tables :). I am looking forward to the premier of the next class, lesson 3 :).

I can close this issue.

mossr commented 3 years ago

Thank you for the kind words 🤗

The nice thing is Pluto will not re-download everything from scratch. So you don't have to worry about that (it may look like it but if you have a package locally already, then it will just update the Julia registry (which takes some time) then notice you already have it and not have to download it again).

Funny you mention teaching value iteration, I've been putting together a notebook for my own teaching on the value iteration algorithm itself (implemented from scratch but works on the Grid World MDP shown in the Julia Academy course). Note that this is a work-in-progress and the implementation will be somewhat slow compared to DiscreteValueIteration for the purpose of education, but here's the notebook:

00krishna commented 3 years ago

@mossr Oh this is really excellent. I can totally use these. Thanks so much for the links.

I have not had a chance to watch the latest videos that were released. But do you know of any examples where POMDP uses a continuous reward function? I was trying to figure out a way to incorporate a differential equation as the basis for assigning rewards based upon actions/states. I saw an example of this in ReinforcementLearning.jl, but not in POMDP.jl. If you know of any good links, or if you have a notebook or video with this, please pass along the information. Thanks again for your efforts on creating all of this great content.

mossr commented 3 years ago

The deep RL example in my lectures talk about the pendulum swing-up problem that has a continuous reward function:

00krishna commented 3 years ago

@mossr you are a star. That is great. It always helps to have some code to start with. Thanking you is getting to be a habit :).