As training progresses, sampling time becomes longer and longer

YifeiZhou02 / ArCHer

Research Code for "ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL"

https://yifeizhou02.github.io/archer.io/

105 stars 13 forks source link

As training progresses, sampling time becomes longer and longer #13

Open destinyyzy opened 2 months ago

destinyyzy commented 2 months ago

As training progresses, sampling takes longer and longer. Have you encountered this problem during training? If so, what is the solution?

YifeiZhou02 commented 2 months ago

Hi, thanks for your interest in our work. Which environment are you using? If it is Webshop, there is an known issue and its fix as in #6

destinyyzy commented 1 month ago

Yes, I encountered this problem in Webshop. I used the full amount of items and 10,000 instructions. I tried the method provided in #6, but it didn't help.

YifeiZhou02 commented 1 month ago

I see, I also encountered this issue when I ran the experiments for the paper but was not able to solve it. This seems to be an issue related WebShop that processes html requests very slowly when there are a lot of instructions.