QiaolingChen00
commented
1 year ago Open QiaolingChen00 opened 1 year ago
QiaolingChen00
commented
1 year ago 
CherylQWong
commented
1 year ago 1-4: use "Title" & "Abstract" (truncated into 512 tokens) as input 5 outline: GPT3.5 to summarize the "Abstract" & "Introduction" respectively, then use GPT4 to select the better one 6 research background: use "Introduction" as input
xiaohuihong
commented
1 year ago {
"instruction": "Mark the title of the given paper",
"input": "paper first page",
"output": ""
},
{
"instruction": "List all the authors' names",
"input": "paper first page",
"output": ""
},
{
"instruction": "Mark the keywords of this paper and give their definitions.",
"input": "paper first page",
"output": ""
},
{
"instruction": "Summarize the given introduction to generate the research background of this paper.",
"input": "paper's introduction",
"output": ""
},
{
"instruction": "List all the research methodologies proposed by this paper and summarize their details.",
"input": "paper's methods", ??? use abstract first
"output": ""
},
{
"instruction": "Give a conclusion about this paper's major achievements and breakthroughs.",
"input": "paper's conclusion",
"output": ""
},Please give me some information based on parts of the paper: ''' 1st page ''' Your response should be in the following format: 1.Title: 2.Author names: 3.Keywords: 4.Research Background: 5.Research Methodology: 6.Conclusion:
CherylQWong
commented
1 year ago [ { "instruction": "Please give me some information based on parts of the paper: REWARD DESIGN WITH LANGUAGE MODELS\nMinae Kwon, Sang Michael Xie, Kalesha Bullard†, Dorsa Sadigh\nStanford University, DeepMind†\n{minae, xie, dorsa}@cs.stanford.edu, ksbullard@deepmind.com†\nABSTRACT\nReward design in reinforcement learning (RL) is challenging since specifying human notions of desired behavior may be difficult via reward functions or require many expert demonstrations. Can we instead cheaply design rewards using a natural language inter- face? This paper explores how to simplify reward design by prompting a large language model (LLM) such as GPT-3 as a proxy reward function, where the user provides a tex- tual prompt containing a few examples (few-shot) or a description (zero-shot) of the de- sired behavior. Our approach leverages this proxy reward function in an RL framework. Specifically, users specify a prompt once at the beginning of training. During training, the LLM evaluates an RL agent’s behavior against the desired behavior described by the prompt and outputs a corresponding reward signal. The RL agent then uses this reward to update its behavior. We evaluate whether our approach can train agents aligned with user objectives in the Ultimatum Game, matrix games, and the DEALORNODEAL negotiation task. In all three tasks, we show that RL agents trained with our framework are well-aligned with the user’s objectives and outperform RL agents trained with reward functions learned via supervised learning. Code and prompts can be found here.\n1 INTRODUCTION\nAutonomous agents are becoming increasingly capable with the rise of compute and data. This underscores the importance for human users to be able to control what policies the agents learn and ensure the policies are aligned with their objectives. For instance, imagine training an agent to represent users in a salary negotiation. A working mother fighting for a livable wage may want their agent to be stubborn whereas a new hire looking to develop a good relationship with the company may want their agent to be more versatile.\nCurrently, users specify desired behaviors by 1) designing reward functions or 2) providing large amounts of labeled data. Both approaches are challenging and impractical for different reasons. Designing reward func- tions is not an intuitive way to specify preferences. For instance, it isn’t straightforward how to write a reward function for a “versatile” negotiator. Furthermore, designing reward functions that balance between different objectives — also known as the “reward design problem” — is notoriously difficult because agents are sus- ceptible to reward hacking (Amodei et al., 2016; Hadfield-Menell et al., 2017). On the other hand, one can learn a reward function from labeled examples. However, that is not possible with a single example; we need large amounts of labeled data to capture the nuances of different users’ preferences and objectives, which has shown to be costly (Zhang et al., 2016). Additionally, both approaches do not generalize well to new users who have different objectives — we would have to re-design our reward functions or re-collect data.\nOur aim is to create an easier way for users to communicate their preferences, where the interface is more intuitive than crafting a reward function and where they can cheaply specify their preferences with no more than a few examples. To do this, we leverage large language models (LLMs) that are trained on internet-scale text data and have shown an impressive ability to learn in-context from few or zero examples (Brown et al., 2020). Our key insight is that\nThe scale of data that LLMs have been trained on make them great in-context learners and also allows them to capture meaningful commonsense priors about human behavior. Given a few examples or a description demonstrating the user’s objective, an LLM should be able to provide an accurate instantiation of reward values on a new test example, allowing for easier generalization to new objectives.\nTo this end, we explore how to prompt an LLM as a proxy reward function to train RL agents from user inputs. In our approach, the user specifies an objective with a natural language prompt. Objectives can \nYour response should be in the following format: 1.Title: \n2.Author names: \n3.Keywords: \n4.Research Background: \n5.Research Methodology: \n6.Conclusion:", "input": "", "output": "1. Title: Reward Design with Language Models \n2. Author names: Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh \n3. Keywords: reinforcement learning, natural language interface, large language models, reward design, proxy reward function \n4. Research Background: The paper addresses the challenge of reward design in reinforcement learning, where specifying human notions of desired behavior via reward functions or expert demonstrations can be difficult or impractical. The authors propose leveraging large language models as a proxy reward function, where users provide a textual prompt containing a few examples or a description of the desired behavior. The proxy reward function is then used to train RL agents that are aligned with the user's objectives. \n5. Research Methodology: The authors prompt a large language model such as GPT-3 to provide a reward signal based on the RL agent's behavior against the desired behavior described in the prompt. The RL agent then uses this reward to update its behavior. The approach is evaluated in the Ultimatum Game, matrix games, and the DEALORNODEAL negotiation task. \n6. Conclusion: The authors demonstrate that the proposed approach simplifies reward design and enables users to communicate their preferences in a more intuitive and cost-effective manner. The use of large language models as proxy reward functions allows for easier generalization to new objectives and outperforms RL agents trained with reward functions learned via supervised learning." } ]
ChenQiaoling00
commented
1 year ago input:
instru="Mark the keywords of this paper and give the definitions of each keyword"
inp2="The remarkable progress in Natural Language Processing(NLP) brought about by deep learning, particularly with the recent ad-vent of large pre-trained neural language models, is brought into scrutinyas several studies began to discuss and report potential biases in NLPapplications."
output = evaluate(instru,inp2)output:
ictive for a, for a, for a,, and, and, and, and, andialist, is a. 1,, 1 orally, a orally with a orifies. 1 orifies for a. 1-conifically.
for a for a for a for a is a is a is a is a is a, input:
instru="Summarize the following text as two words:"
inp="The remarkable progress in Natural Language Processing(NLP) brought about by deep learning, particularly with the recent ad-vent of large pre-trained neural language models, is brought into scrutinyas several studies began to discuss and report potential biases in NLPapplications. Bias in NLP is found to originate from latent historicalbiases encoded by humans into textual data which gets perpetuated oreven ampli\ufb01ed by NLP algorithm. We present a survey to comprehendbias in large pre-trained language models, analyze the stages at whichthey occur in these models, and various ways in which these biases couldbe quanti\ufb01ed and mitigated. Considering wide applicability of textuala\ufb00ective computing based downstream tasks in real-world systems suchas business, healthcare, education, etc., we give a special emphasis oninvestigating bias in the context of a\ufb00ect (emotion) i.e., A\ufb00ective Bias,in large pre-trained language models. We present a summary of variousbias evaluation corpora that help to aid future research and discuss chal-lenges in the research on bias in pre-trained language models. We believethat our attempt to draw a comprehensive view of bias in pre-trainedlanguage models, and especially the exploration of a\ufb00ective bias will behighly bene\ufb01cial to researchers interested in this evolving \ufb01eld.\ufffd The examples provided in this paper may be o\ufb00ensive in nature andmay hurt your moral beliefs.Keywords: NLP Bias \u00b7 Fairness \u00b7 Large Pre-trained Language Models\u00b7 A\ufb00ective Bias \u00b7 A\ufb00ective Computing.1IntroductionNatural Language Processing (NLP) has recently achieved rapid progress withthe aid of deep learning, especially Pre-trained Language Models (PLM)"
output = evaluate(instru,inp2)
output:
NLP Bias, Fairness, Large Pre-trained Language Models, Affective Bias, Affective Computing.input:
ins='Summarize the given introduction to generate the research background of the following.'
inp="The remarkable progress in Natural Language Processing(NLP) brought about by deep learning, particularly with the recent ad-vent of large pre-trained neural language models, is brought into scrutinyas several studies began to discuss and report potential biases in NLPapplications. Bias in NLP is found to originate from latent historicalbiases encoded by humans into textual data which gets perpetuated oreven ampli\ufb01ed by NLP algorithm. We present a survey to comprehendbias in large pre-trained language models, analyze the stages at whichthey occur in these models, and various ways in which these biases couldbe quanti\ufb01ed and mitigated. Considering wide applicability of textuala\ufb00ective computing based downstream tasks in real-world systems suchas business, healthcare, education, etc., we give a special emphasis oninvestigating bias in the context of a\ufb00ect (emotion) i.e., A\ufb00ective Bias,in large pre-trained language models. We present a summary of variousbias evaluation corpora that help to aid future research and discuss chal-lenges in the research on bias in pre-trained language models. We believethat our attempt to draw a comprehensive view of bias in pre-trainedlanguage models, and especially the exploration of a\ufb00ective bias will behighly bene\ufb01cial to researchers interested in this evolving \ufb01eld.\ufffd The examples provided in this paper may be o\ufb00ensive in nature andmay hurt your moral beliefs.Keywords: NLP Bias \u00b7 Fairness \u00b7 Large Pre-trained Language Models\u00b7 A\ufb00ective Bias \u00b7 A\ufb00ective Computing.1IntroductionNatural Language Processing (NLP) has recently achieved rapid progress withthe aid of deep learning, especially Pre-trained Language Models (PLM)"
output = evaluate(instru,inp2)output:
as bullet point: output:
1. BERT is robust against injected noise, but could be vulnerable to noise from weak supervision. In fact, the latter, even at a low level, can be more challenging than high injected noise.
2. Existing noise-handling methods do not improve the peak performance of BERT under any noise settings we investigated; as is shown with further analysis, noise-handling methods rarely render the correct labels more distinguishable from the incorrect ones.input:
method_ins='List all the research methodologies proposed by the following and summarize their details.'
method_inp='"LaMini-LM: A Diverse Herd of Distilled Modelsfrom Large-Scale InstructionsMinghao Wu1,2\u2217Abdul Waheed1 Chiyu Zhang1,3 Muhammad Abdul-Mageed1,3 Alham Fikri Aji11Mohamed bin Zayed University of Arti\ufb01cial Intelligence2Monash University3The University of British Columbia{minghao.wu,abdul.waheed,chiyu.zhang,muhammad.mageed,alham.fikri},@mbzuai.ac.aeAbstractLarge language models (LLMs) with instruc-tion \ufb01netuning demonstrate superior genera-tive capabilities. However, these models areresource intensive. To alleviate this issue, weexplore distilling knowledge from instruction-tuned LLMs to much smaller ones. To this end,we carefully develop a large set of 2.58M in-structions based on both existing and newly-generated instructions.In addition to beingsizeable, we design our instructions to covera broad set of topics to ensure. A thorough in-vestigation of our instruction data demonstratetheir diversity, and we generate responses forthese instructions using gpt-3.5-turbo. Wethen exploit the instructions to tune a hostof models, dubbed LaMini-LM, of varyingsizes, both from the encoder-decoder as wellas the decoder-only families.We evaluateour models both automatically (on 15 differ-ent NLP benchmarks) and manually. Resultsshow that our proposed LaMini-LM are on parwith competitive baselines while being nearly\u00d710 smaller in size.11IntroductionLarge language models (LLMs) with instructiontuning are capable of generating remarkable out-puts for a wide range of use cases (Ouyang et al.,2022; Wei et al., 2022; Sanh et al., 2022; Chunget al., 2022; OpenAI, 2023). However, these mod-els usually have billions of parameters, which re-quire massive computational resources for bothtraining and inference (Brown et al., 2020; Thop-pilan et al., 2022; Hoffmann et al., 2022; Chowd-hery et al., 2022). Kaplan et al. (2020) suggestthat the performance of LLMs scales proportion-ally with model and dataset size. Consequently,scaling the models raises many issues such as thoserelated to the energy footprint (Strubell et al., 2019).\u2217 work done while visiting MBZUAI1Our code, model checkpoints, and dataset are available athttps:\/\/github.com\/mbzuai-nlp\/LaMini-LM+GenerateExistingInstructionsGPT2+GenerateFine-tuneT5NeoLaMini-LMC.ModelsSeedInstructionsSyntheticInstructionsSynthetic ResponsesLaMini-Instruction datasetFigure 1: Overview of LaMini-LMMoreover, the accessibility of large models is a realconcern for many NLP practitioners due to lim-ited access to computing resources (Nityasya et al.,2020).In this work, we present LaMini-LM, a collec-tion of language models that are notably smallerin size than most existing instruction-tuned mod-els. We develop LaMini-LM models by employingsequence distillation (also known as of\ufb02ine distilla-tion) (Kim and Rush, 2016) from LLMs. Althoughsimilar attempts have been made in recent work(e.g., (Taori et al., 2023; Chiang et al., 2023; Anandet al., 2023)), there are several gaps in this literaturethat we aim to address. Speci\ufb01cally, these worksoften (i) provide a small-scale distilled dataset (ii)that is not necessarily diverse, and a (iii) limitednumber of models (typically only one), (iv) with-out comprehensive evaluation nor analysis of themodels\u2019 performance. Furthermore, many of thedistilled models resulting from prior work tend tostill be relatively computationally intensive. Thatis, parameters of these recent models usually rangefrom 7B to 13B, making them dif\ufb01cult to deploy inresource-constrained settings especially for under-resourced institutions.To alleviate these issues, we \ufb01rstly generate alarge-scale of\ufb02ine distillation dataset comprisingarXiv:2304.14402v1 [cs.CL] 27 Apr 2023'
output = evaluate(method_ins,method_inp)output:
LaMini-LM is a collection of language models that are notably smaller in size than most existing instruction-tuned models. We develop LaMini-LM models by employing sequence distillation (also known as offline distillation) (Kim and Rush, 2016) from LLMs. Although similar attempts have been made in recent work (e.g., (Taori et al., 2023; Chiang et al., 2023; Anand et al., 2023), there are several gaps in this literature that we aim to address. Specifically, these works often (i) provide a small-scale distilled dataset (ii) that is not necessarily diverse, and a (iii) limited number of models (typically only one), (iv) without comprehensive evaluation nor analysis of the models’ performance. Furthermore, many of the distilled models resulting from prior work tend to still be relatively computationally intensive. That is, parameters of these recent models usually range from 7B to 13B, making them difficult to deploy in resource-constrained settings especially for under-resourced institutions. To alleviate these issues, we firstly generate a large-scale offline distillation dataset comprising arXiv:2304.14402v1 [cs.CL] 27 Apr 2023 method_ins='List all the research methodologies proposed by the following as bulletpoint'
method_inp='"LaMini-LM: A Diverse Herd of Distilled Modelsfrom Large-Scale InstructionsMinghao Wu1,2\u2217Abdul Waheed1 Chiyu Zhang1,3 Muhammad Abdul-Mageed1,3 Alham Fikri Aji11Mohamed bin Zayed University of Arti\ufb01cial Intelligence2Monash University3The University of British Columbia{minghao.wu,abdul.waheed,chiyu.zhang,muhammad.mageed,alham.fikri},@mbzuai.ac.aeAbstractLarge language models (LLMs) with instruc-tion \ufb01netuning demonstrate superior genera-tive capabilities. However, these models areresource intensive. To alleviate this issue, weexplore distilling knowledge from instruction-tuned LLMs to much smaller ones. To this end,we carefully develop a large set of 2.58M in-structions based on both existing and newly-generated instructions.In addition to beingsizeable, we design our instructions to covera broad set of topics to ensure. A thorough in-vestigation of our instruction data demonstratetheir diversity, and we generate responses forthese instructions using gpt-3.5-turbo. Wethen exploit the instructions to tune a hostof models, dubbed LaMini-LM, of varyingsizes, both from the encoder-decoder as wellas the decoder-only families.We evaluateour models both automatically (on 15 differ-ent NLP benchmarks) and manually. Resultsshow that our proposed LaMini-LM are on parwith competitive baselines while being nearly\u00d710 smaller in size.11IntroductionLarge language models (LLMs) with instructiontuning are capable of generating remarkable out-puts for a wide range of use cases (Ouyang et al.,2022; Wei et al., 2022; Sanh et al., 2022; Chunget al., 2022; OpenAI, 2023). However, these mod-els usually have billions of parameters, which re-quire massive computational resources for bothtraining and inference (Brown et al., 2020; Thop-pilan et al., 2022; Hoffmann et al., 2022; Chowd-hery et al., 2022). Kaplan et al. (2020) suggestthat the performance of LLMs scales proportion-ally with model and dataset size. Consequently,scaling the models raises many issues such as thoserelated to the energy footprint (Strubell et al., 2019).\u2217 work done while visiting MBZUAI1Our code, model checkpoints, and dataset are available athttps:\/\/github.com\/mbzuai-nlp\/LaMini-LM+GenerateExistingInstructionsGPT2+GenerateFine-tuneT5NeoLaMini-LMC.ModelsSeedInstructionsSyntheticInstructionsSynthetic ResponsesLaMini-Instruction datasetFigure 1: Overview of LaMini-LMMoreover, the accessibility of large models is a realconcern for many NLP practitioners due to lim-ited access to computing resources (Nityasya et al.,2020).In this work, we present LaMini-LM, a collec-tion of language models that are notably smallerin size than most existing instruction-tuned mod-els. We develop LaMini-LM models by employingsequence distillation (also known as of\ufb02ine distilla-tion) (Kim and Rush, 2016) from LLMs. Althoughsimilar attempts have been made in recent work(e.g., (Taori et al., 2023; Chiang et al., 2023; Anandet al., 2023)), there are several gaps in this literaturethat we aim to address. Speci\ufb01cally, these worksoften (i) provide a small-scale distilled dataset (ii)that is not necessarily diverse, and a (iii) limitednumber of models (typically only one), (iv) with-out comprehensive evaluation nor analysis of themodels\u2019 performance. Furthermore, many of thedistilled models resulting from prior work tend tostill be relatively computationally intensive. Thatis, parameters of these recent models usually rangefrom 7B to 13B, making them dif\ufb01cult to deploy inresource-constrained settings especially for under-resourced institutions.To alleviate these issues, we \ufb01rstly generate alarge-scale of\ufb02ine distillation dataset comprisingarXiv:2304.14402v1 [cs.CL] 27 Apr 2023'
output = evaluate(method_ins,method_inp)output:
LaMini-LM is a collection of language models that are notably smaller in size than most existing instruction-tuned models. We develop LaMini-LM models by employing sequence distillation (also known as offline distillation) (Kim and Rush, 2016) from LLMs. Although similar attempts have been made in recent work (e.g., (Taori et al., 2023; Chiang et al., 2023; Anand et al., 2023), there are several gaps in this literature that we aim to address. Specifically, these works often (i) provide a small-scale distilled dataset (ii) that is not necessarily diverse, and a (iii) limited number of models (typically only one), (iv) without comprehensive evaluation nor analysis of the models’ performance. Furthermore, many of the distilled models resulting from prior work tend to still be relatively computationally intensive. That is, parameters of these recent models usually range from 7B to 13B, making them difficult to deploy in resource-constrained settings especially for under-resourced institutions. To alleviate these issues, we firstly generate a large-scale offline distillation dataset comprising arXiv:2304.14402v1 [cs.CL] 27 Apr 2023效果不好 出现乱码，重复，乱写等情况
input：
conclu_ins='Give a conclusion about the following major achievements and breakthroughs.'
conclu_inp='Cross-Lingual Summarization (CLS) is a task that extracts impor-\ntant information from a source document and summarizes it into\na summary in another language. It is a challenging task that re-\nquires a system to understand, summarize, and translate at the\nsame time, making it highly related to Monolingual Summariza-\ntion (MS) and Machine Translation (MT). In practice, the training\nresources for Machine Translation are far more than that for cross-\nlingual and monolingual summarization. Thus incorporating the\nMachine Translation corpus into CLS would be beneficial for its\nperformance. However, the present work only leverages a simple\nmulti-task framework to bring Machine Translation in, lacking\ndeeper exploration.\nIn this paper, we propose a novel task, Cross-lingual Summariza-\ntion with Compression rate (CSC), to benefit Cross-Lingual Sum-\nmarization by large-scale Machine Translation corpus. Through\nintroducing compression rate, the information ratio between the\nsource and the target text, we regard the MT task as a special CLS\ntask with a compression rate of 100%. Hence they can be trained as\na unified task, sharing knowledge more effectively. However, a huge\ngap exists between the MT task and the CLS task, where samples\nwith compression rates between 30% and 90% are extremely rare.\nHence, to bridge these two tasks smoothly, we propose an effective\ndata augmentation method to produce document-summary pairs\nwith different compression rates. The proposed method not only\nimproves the performance of the CLS task, but also provides con-\ntrollability to generate summaries in desired lengths. Experiments\ndemonstrate that our method outperforms various strong baselines\nin three cross-lingual summarization datasets. We released our code\nand data at https:\/\/github.com\/ybai-nlp\/CLS_CR.\n\u2020Corresponding author.\nPermission to make digital or hard copies of all or part of this work for personal or\nclassroom use is granted without fee provided that copies are not made or distributed\nfor profit or commercial advantage and that copies bear this notice and the full citation\non the first page. Copyrights for components of this work owned by others than ACM\nmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,\nto post on servers or to redistribute to lists, requires prior specific permission and\/or a\nfee. Request permissions from permissions@acm.org.\nSIGIR \u201922, July 11\u201315, 2022, Madrid, Spain\n\u00a9 2022 Association for Computing Machinery.\nACM ISBN 978-1-4503-8732-3\/22\/07...$15.00\nhttps:\/\/doi.org\/10.1145\/3477495.3532071\nCCS CONCEPTS\n\u2022 Information systems \u2192 Summarization; Multilingual and\ncross-lingual retrieval.\nKEYWORDS\nCross-lingual Summarization; Machine Translation; Compression\nRate\n1\nINTRODUCTION\nNowadays, acquiring information in a foreign language has become\nan important requirement for people of any nation. Millions of new\ndocuments are produced every day on the Internet. These docu-\nments are written in various languages, bringing more difficulties\nfor people who are not familiar with other languages. Hence, how to\nextract and understand the important information from documents\nin foreign languages is a meaningful and challenging research prob-\nlem. To achieve such a goal, Cross-lingual Summarization has been\ncontinuously explored by plenty of researchers [3, 18, 30, 31].\nCross-lingual Summarization (CLS) aims at converting a docu-\nment from one language to a summary in another language. It com-\nbines Machine Translation (MT) and Monolingual Summarization\n(MS), requiring the ability to both extract key ideas and translate\nthem into the target language, illustrated in Figure 1. Exploring\nthe relationship of these relevant tasks is important to improve\nthe performance of CLS. Zhu et al. [30] apply the classic multitask\nframework of sequence-to-sequence model [13], using a unified\nencoder to share the knowledge between CLS and MT. Takase and\nOkazaki [22] suggest using one single Transformer to learn the\nMT task, the MS task, and the CLS task, only distinguished by a\nspecial token. As mentioned, CLS has a close relationship with the\nMT task. However, these previous works simply treat MT as an\nindependent and auxiliary task for CLS, lacking deeper exploration\nof their relationship. Hence, how to better leverage the huge MT\ncorpus up to the hilt still remains a challenge.\nTo achieve the mentioned goal, we thoroughly probe the rela-\ntionship between MT and CLS. We observe that the MT task can be\nviewed as a special case of the CLS task via a concept in the summa-\nrization domain: compression rate, which refers to the information\nratio between the target summary and the original document in the\nsummarization task [1, 15, 16, 23]. If the target summary contains\nthe exact same amount of information as the source document, the\ncompression rate becomes 100%. In a cross-lingual scenario, we\narXiv:2110.07936v2 [cs.CL] 24 Apr 2022\n'
output = evaluate(conclu_ins,conclu_inp)output:
乱码input:
conclu_ins='Conclude the following '
conclu_inp='Cross-Lingual Summarization (CLS) is a task that extracts impor-\ntant information from a source document and summarizes it into\na summary in another language. It is a challenging task that re-\nquires a system to understand, summarize, and translate at the\nsame time, making it highly related to Monolingual Summariza-\ntion (MS) and Machine Translation (MT). In practice, the training\nresources for Machine Translation are far more than that for cross-\nlingual and monolingual summarization. Thus incorporating the\nMachine Translation corpus into CLS would be beneficial for its\nperformance. However, the present work only leverages a simple\nmulti-task framework to bring Machine Translation in, lacking\ndeeper exploration.\nIn this paper, we propose a novel task, Cross-lingual Summariza-\ntion with Compression rate (CSC), to benefit Cross-Lingual Sum-\nmarization by large-scale Machine Translation corpus. Through\nintroducing compression rate, the information ratio between the\nsource and the target text, we regard the MT task as a special CLS\ntask with a compression rate of 100%. Hence they can be trained as\na unified task, sharing knowledge more effectively. However, a huge\ngap exists between the MT task and the CLS task, where samples\nwith compression rates between 30% and 90% are extremely rare.\nHence, to bridge these two tasks smoothly, we propose an effective\ndata augmentation method to produce document-summary pairs\nwith different compression rates. The proposed method not only\nimproves the performance of the CLS task, but also provides con-\ntrollability to generate summaries in desired lengths. Experiments\ndemonstrate that our method outperforms various strong baselines\nin three cross-lingual summarization datasets. We released our code\nand data at https:\/\/github.com\/ybai-nlp\/CLS_CR.\n\u2020Corresponding author.\nPermission to make digital or hard copies of all or part of this work for personal or\nclassroom use is granted without fee provided that copies are not made or distributed\nfor profit or commercial advantage and that copies bear this notice and the full citation\non the first page. Copyrights for components of this work owned by others than ACM\nmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,\nto post on servers or to redistribute to lists, requires prior specific permission and\/or a\nfee. Request permissions from permissions@acm.org.\nSIGIR \u201922, July 11\u201315, 2022, Madrid, Spain\n\u00a9 2022 Association for Computing Machinery.\nACM ISBN 978-1-4503-8732-3\/22\/07...$15.00\nhttps:\/\/doi.org\/10.1145\/3477495.3532071\nCCS CONCEPTS\n\u2022 Information systems \u2192 Summarization; Multilingual and\ncross-lingual retrieval.\nKEYWORDS\nCross-lingual Summarization; Machine Translation; Compression\nRate\n1\nINTRODUCTION\nNowadays, acquiring information in a foreign language has become\nan important requirement for people of any nation. Millions of new\ndocuments are produced every day on the Internet. These docu-\nments are written in various languages, bringing more difficulties\nfor people who are not familiar with other languages. Hence, how to\nextract and understand the important information from documents\nin foreign languages is a meaningful and challenging research prob-\nlem. To achieve such a goal, Cross-lingual Summarization has been\ncontinuously explored by plenty of researchers [3, 18, 30, 31].\nCross-lingual Summarization (CLS) aims at converting a docu-\nment from one language to a summary in another language. It com-\nbines Machine Translation (MT) and Monolingual Summarization\n(MS), requiring the ability to both extract key ideas and translate\nthem into the target language, illustrated in Figure 1. Exploring\nthe relationship of these relevant tasks is important to improve\nthe performance of CLS. Zhu et al. [30] apply the classic multitask\nframework of sequence-to-sequence model [13], using a unified\nencoder to share the knowledge between CLS and MT. Takase and\nOkazaki [22] suggest using one single Transformer to learn the\nMT task, the MS task, and the CLS task, only distinguished by a\nspecial token. As mentioned, CLS has a close relationship with the\nMT task. However, these previous works simply treat MT as an\nindependent and auxiliary task for CLS, lacking deeper exploration\nof their relationship. Hence, how to better leverage the huge MT\ncorpus up to the hilt still remains a challenge.\nTo achieve the mentioned goal, we thoroughly probe the rela-\ntionship between MT and CLS. We observe that the MT task can be\nviewed as a special case of the CLS task via a concept in the summa-\nrization domain: compression rate, which refers to the information\nratio between the target summary and the original document in the\nsummarization task [1, 15, 16, 23]. If the target summary contains\nthe exact same amount of information as the source document, the\ncompression rate becomes 100%. In a cross-lingual scenario, we\narXiv:2110.07936v2 [cs.CL] 24 Apr 2022\n'
output = evaluate(conclu_ins,conclu_inp)output:
乱码
1 that, a with a with a with a with a and is a and of the of a for a for a for a for a for a for a.
for a. 1 that. ictive. ictive for a, for a, for a,, and, and, and, and, andialist, is a. 1,, 1 orally, a orally with a orifies. 1 orifies for a. 1-conifically.
for a for a for a for a is a is a is a is a is a, 1, 1, 1 in a and with a with a and with a with a �ays for a for a for a in a in a. 1 for a in a for a for a is for a
for a is a,, 1, and, and, and, and, 1 or with a or with a is with a is for a. 1 in a, 1 in a in a, 1, 1 in 1, 1, 1 in a, 1, 1, 1, and, 1, 1 in the 1 in 1,, 1, 1,, and,,, 2 orice- 1 in a. 1. 1 in a. 1 or in a or or or or with a or or is with a or is or is for the is for the or and or and. or and for a for a is in a is in a is is 1 is for a for a for a for a for a for a for a for a for a and is is. is or and. . . in it. in it. in. in. in in in. in. in a in a in a in a in or to in in in, in,,, and, and....... and.. and. and. in. in. of, of, of for a for a of, of, of, of for for for for for for for for for for for for for for for for for for for the ,,,,, ,,, , in for for for for the for the ,, ,, ,, for the, and for the for the for the for the and in and and and in the and in the and and,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, and, and, and, and,,,,,,,,,,,, with, and and and and in in in in in the in the in a in a in a in a in the in a in a in a in a in and in
YanJiaHuan
commented
1 year ago 原来：
alpaca_data.json "Write a review of a recent movie you watched.", "Generate a list of marketing strategies to promote a new mobile app." "Give a description of the following customer segment." ...
改：
Original: 'Mark the title of the given paper.' Rewritten: 'Identify the title of the following document.'
Original: "List all the authors' names." Rewritten: "Enumerate the names of all contributors to the following document."
Original: 'Mark the keywords of this paper and give their definitions.' Rewritten: 'Highlight the key terms in the following document and provide their meanings.'
Original: 'Summarize the given introduction to generate the research background of this paper.' Rewritten: 'Create a brief summary of the following document to understand its main context or background.'
Original: 'List all the research methodologies proposed by this paper and summarize their details.' Rewritten: 'Enumerate the methods or approaches proposed in the following document and provide a summary of each.'
Original: "Give a conclusion about this paper's major achievements and breakthroughs." Rewritten: "Provide a summary of the following document's key findings or significant advancements."