Closed Anand1405 closed 9 months ago
Hi @Anand1405 ,
Yes, by default it does data parallelism. If you use 32 workers on a trn1.32xlarge
instance, you will be able to use all the cores as well.
Tensor parallelism is supported, but not for GPT-2. We support it for GPTNeo
, GPTNeoX
and LLama
(v1 and v2).
How do I use tensor parallelism for the same task if I switch to gpt-neo, as by default it does data parallelism?
I am writing the doc here: #339 .
When I use BertTokenizer the script runs fine, but with no tokenizer name passed it uses GPT2Tokenizer as default and the produces error. Is there any way that I can use GPT2Tokenizer for the task with the same script?
Can you share the command line please?
I am using optimum neuron run_qa.py to fine-tune GPT2, by looking at the output it seems like it does data parallelism. Kindly confirm what kind of parallelism is done? If I enter 8 as batch size it shows total batch size as 16 as I am using it in trn1.2xlarge, so does this mean it is using data parallelism? If it is data parallelism then how to shift to tensor parallelism? Also can we use it for trn1.32xlarge with the utilisation of all cores?