search

taolusi / chisp

scripts and baselines for CSpider: Chinese semantic parsing and text-to-SQL challenge
https://taolusi.github.io/CSpider-explorer/
162 stars 18 forks source link

computer configuration #13

Open Candy621 opened 2 years ago

Candy621 commented 2 years ago

If we want to run this code, how much computer memory do we need?

taolusi commented 2 years ago

We run our code on a machine with 32G memory and 2*1080Ti GPU.

Candy621 commented 2 years ago

Thanks for your reply. However, I failed with 32G memory and CPU to train the 9 models at the same time due to the MemoryError. So I wonder if you run the code to get the 9 models at the same time with 32G and 2 GPU? Than can I train the 9 models separately? For example, I use 32G memory and CPU to train the col model, then I use the same resource to train the next model, until I obtain the whole models.

taolusi commented 2 years ago

In fact, we do not train all the 9 models at the same time. We train the col and where model saparetely with some other models at the same time. I suggest that you can try such a way.

Candy621 commented 2 years ago

Yep! So in this way, the 9 modules are independent and are not depend on each other, is that right? And I have another question, the training data and dev_data both include: db_id, query_toks_no_value, question_toks, query_toks, question, sql and query(7 parts). So I want to know that which parts will be contained when creating testing data? Thanks for your help in advance~

taolusi commented 2 years ago
  1. It is right that all modeuls are independent.
  2. The dataframe is identical for the train, dev and test data.