search

facebookresearch / Pearl

A Production-ready Reinforcement Learning AI Agent Library brought by the Applied Reinforcement Learning team at Meta.
MIT License
2.6k stars 157 forks source link

Error when running the Single Item Recommender System Notebook #79

Closed davera-017 closed 5 months ago

davera-017 commented 6 months ago

Hi, I'm trying to run the notebook on how to use Pearl for recommender systems, but when I run the online_learning() function I keep getting the same error, which I copy below:

    [161](Pearl/pearl/pearl_agent.py:161) if isinstance(safe_action_space, DiscreteActionSpace):
--> [162](Pearl/pearl/pearl_agent.py:162)     self._latest_action = safe_action_space.actions_batch[int(action.item())]
    [163](Pearl/pearl/pearl_agent.py:163) else:
    [164](Pearl/pearl/pearl_agent.py:164)     self._latest_action = action

RuntimeError: a Tensor with 100 elements cannot be converted to Scalar

On the other hand, I'm having a hard time understanding how the environment is being built. Could someone please explain further how they are creating the RecEnv object?

rodrigodesalvobraz commented 6 months ago

I am debugging this and will let you know as soon as possible.

rodrigodesalvobraz commented 6 months ago

Update: we've identified the bug and are currently working on a solution. Should be out today or tomorrow.

rodrigodesalvobraz commented 5 months ago

Update: started the fix but it had a wider range than initially expected. It might take a few days to get everything set correctly.

rodrigodesalvobraz commented 5 months ago

Update: we've fixed the bug but we don't see the same learning behavior as previously observed, so now we are working on identifying the cause of that.

rodrigodesalvobraz commented 5 months ago

Glad to let you know this has been finally fixed! It took a few iterations and the removal of a couple of issues.