100 Webshop？？？？

[xiaxiaxiatengxi]

Excuse me, is your webshop environment currently trained with only 2000~2100 products for testing? And are these 100 products the only ones included in the search engine?"

[YifeiZhou02]

No, we have all the products but we only used the 2000 - 2100 instructions. I'm not sure about how WebShop server is implemented but it seems it will get slower when we use more instructions, even if all products are included in the search engine. We did not do any filtering in terms of the products but only used a subset of the instructions with the default set of all products. Hope it helps.

[xiaxiaxiatengxi]

Thank you very much for your reply. As you mentioned that you haven't modified the Webshop environment, then your Webshop setup should be training with 100 instructions and test them. (for instruction ids numbered 2000 to 2100), and the search engine product database only contain 1000 products. This is because the Webshop's utils.py allow modifying the scales of the product database, and the default is set to 1000.

[xiaxiaxiatengxi]

In webshop, utils.py https://github.com/princeton-nlp/WebShop/blob/master/web_agent_site/utils.py

Could you please confirm if this is the setup you're using? DEFAULT_ATTR_PATH = join(BASE_DIR, '../data/items_ins_v2_1000.json') DEFAULT_FILE_PATH = join(BASE_DIR, '../data/items_shuffle_1000.json') Would it be possible for you to check? This setup has a significant impact on the results.

[YifeiZhou02]

yes, I only used the default json downloaded from webshop. I did not do any modifications to the json file

[xiaxiaxiatengxi]

Thank you for your response, this information is really important to me.！！！