yangheng95 / PyABSA

Sentiment Analysis, Text Classification, Text Augmentation, Text Adversarial defense, etc.;
https://pyabsa.readthedocs.io
MIT License
902 stars 153 forks source link

Question about acos task on the Chinese dataset #311

Open twotwoiscute opened 1 year ago

twotwoiscute commented 1 year ago

Hi,Thanks for the great work, I have a question on the task of acos I've downloaded the dataset from here and there's a task called acos which I am interested in.

Observation

I try a quick train on all the dataset in acos_datasets and found out the fact that the model does not do well on dataset 505.Chinese_Zhang, the fine-tuned model always output the empty string even the value of both training ands val loss decrease during process.

So I check the paper and realize the dataset paper used is shown below:

Dataset: SemEval 2014,15 and 16 datasets are used for our experimentation. The dataset is used as a benchmark for ABSA tasks and has customer reviews from three domains; laptops (Lapt14), hotels (Hotel15), and restaurants (Rest14, Rest15, and Rest16) which migh bring out the fact that the pretrained model it used in [training script] would not be so good at Chinese. (https://github.com/yangheng95/PyABSA/blob/192a5280eab3c27e03c461d677ddcd5aaff9e3a3/pyabsa/tasks/ABSAInstruction/multitask_train.py#L21)

What I have done

I am new to NLP, I believe there's always the step that deal with tokenizer like the codeAutoTokenizer.from_pretrained before training and here's how I init the model, I downloaded it here directly.

self.tokenizer = AutoTokenizer.from_pretrained(checkpoint)
self.model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)

Question

Is it okay to apply the original pipeline that the model is trained on english I/O directly to the model that takes chinese input I/O? if not what should I do before get into training. Thanks!

twotwoiscute commented 1 year ago

I also try train dataset 505.Chinese_Zhang only, the training process looks to go me. However when it comes to predict, the model still output empty string as described above.

yangheng95 commented 1 year ago

Sorry for my late reply as I am on a vacation. Saddly, the ACOS task is only tested for English, while any other language is enabled, I will release a new version. BTW, Chinese is supported by ATEPC task.

twotwoiscute commented 1 year ago

Thanks for the reply , from your reply I can tell that you’ve known all it takes to get the things done. Can you tell me the pipeline of how to achieve it? I would like to try it on my own.Thanks!