search

OLA2022-Group-12 / OLA2022-Project

Project for the Online Learning Applications class at Politecnico di Milano a.a. 2021-2022
2 stars 0 forks source link

Uncertain graph learner #23

Closed barskern closed 2 years ago

barskern commented 2 years ago

I've now implemented the graph learner using TS sampling as a primer. The way it works is that every interaction it will check to see which edges were activated, if an edge was activated it will increase the corresponding alpha value for that edge, making the estimation more certain that the graph weight is higher. If a product was visited but the secondary product was not chosen, the beta parameter for that edge will be increased to correspondingly move the estimation down.

I'm not fully sure this is the way to go, but thats how I think it must be done. All feedback is more than welcome! 😄

raul-singh commented 2 years ago

It looks good. I'll look into it as soon as I can and will make a proper review.

raul-singh commented 2 years ago

Still haven't look deeply into the code, but I tried your learner in a notebook and I think the "progress" function is not working well. Here you can see that the output is messy. Moreover, the functions prints new stuff at every iteration, leading to hundreds of outputs, which are not even labeled. I made a little screenshot here. image

barskern commented 2 years ago

Still haven't look deeply into the code, but I tried your learner in a notebook and I think the "progress" function is not working well. Here you can see that the output is messy. Moreover, the functions prints new stuff at every iteration, leading to hundreds of outputs, which are not even labeled. I made a little screenshot here. image

Hihi, this is how its supposed to be (though not the overlapping part). The titling and labling could be better as it is hard to understand without a knowledge of how they are put together. Anyways, it's essentially a adjacency matrix where there is a graph at position (x,y) if product y is secondary to product x. Does that make sense?

Also in a notebook however I think this type of iterative usage doesn't work that well. When running at the command line plt.show(block=true) means that the execution waits for you to close the figure, so you get then after each other. Maybe I need to reconsider that.

barskern commented 2 years ago

The learner looks good, however I found it to perform a bit poorly. Honestly everything looks well done, so I think the culprit could be the calculate_aggregated_budget_value function (maybe it was the same problem we had with the clairvoyant). When I tested the graphless, I saw it having a mean reward of around 1400, while the latest alphaless from the alphaless branch have a consistent 2200 mean reward. I don't want to slow down the deployment of this learner (since we are kind of in a rush). I just want to say that I'm quite happy to see it working and I'd already be satisfied with the other changes I requested, I just think that after merging it is worth to spend some time to understand why there is such discrepancy of performance, if we have time obviously. Other than that, really good work! Everything is implemented in a simple yet clean and understandable way!

I think maybe you are on to it regarding the calculate_aggregated_budget_value. I also think that the graph influence code needs a closer inspection as it is still somewhat foreign to me. That's my further goal I suppose, to figure out what is going on there. Further I suppose the alphaless learner is not utilizing the information from the graph yet neither, so maybe that is simply a source of challenges and doesn't provide that much value. I am not certain.