测试了一下，感觉模型的常识还不够

[Degfy]

[tjadamlee]

模型的常识很大程度还是需要依赖预训练阶段，BELLE基于BLOOM，且目前重点关注instruct阶段如何激发指令理解能力。我们也在探讨如何加强知识注入，将来有机会也会开源相关进展。

[TestNLP]

多大的资源跑起来的？

[bestpredicts]

多大的资源跑起来的？

不做量化的话，显存大概需要28G左右。

[Degfy]

常识很大程度还是需要依赖预训练阶段

我这里是用的 NVIDIA V100

[temberature]

Prompt |   | BELLE | alpaca-lora-30b | alpaca-lora-7b | ggml-alpaca-7b-q4.bin | bard | GPT - 3.5 答案评分 | GPT - 3.5 |   | GPT - 4 答案评分 | GPT - 4 | GPT-4 -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 谁是蝙蝠侠 | Who is Batman | 蝙蝠侠（Batman）是DC漫画宇宙中的一个虚构角色，由作家和艺术家杰里·西格尔（Jerry Siegel）和乔·舒斯特（Joe Shuster）于1958年创造。他是一个神秘的超级英雄，以他的标志性红色披风、黑色面罩和飞行能力而闻名。蝙蝠侠最初是为了对抗犯罪而创建的，但随着时间的推移，他逐渐成为一个象征着正义和自由的形象，成为了全球范围内最受欢迎的超级英雄之一。 | Batman is a fictional superhero created by Bob Kane and Bill Finger and published by DC Comics. He is the alter ego of Bruce Wayne, a billionaire who fights crime in Gotham City. He is known for his detective skills, martial arts abilities, and high-tech gadgets. | Batman is a fictional superhero appearing in American comic books published by DC Comics. He was created by artist Bob Kane and writer Bill Finger, and first appeared in Detective Comics #27 in May 1939.中文The Batman is a fictional superhero who appears in American comic books published by DC Comics. | Batman (originally known simply as the Dark Knight) is a fictional superhero appearing in American comic books published
 by DC Comics. The character was created by artist Bob Kane and writer Bill Finger, and first appeared in Detective Comi
cs #27.[1] In his debut appearance Batman declared that he would fight "the forces of evil" as long as there is still la
wlessness on the streets; this statement has become a mantra for him.
Batman resides with Alfred Pennyworth, who serves as both butler and mentor to Bruce Wayne/Batman.[2] He also maintains 
residences in Gotham City (his primary base of operations), Metropolis,[3][4], the Batcave[5]. | Batman is a superhero who protects the fictional city of Gotham City. He is the secret identity of Bruce Wayne, a wealthy industrialist and philanthropist who vowed to fight crime after witnessing the murder of his parents. Batman is known for his use of technology, his detective skills, and his physical prowess. He is also a founding member of the Justice League.Batman was created by Bill Finger and Bob Kane and first appeared in Detective Comics #27 in 1939. He has been portrayed in many different media, including films, television shows, and video games. Some of the most famous actors to have played Batman include Adam West, Michael Keaton, Val Kilmer, George Clooney, Christian Bale, Ben Affleck, and Robert Pattinson.Batman is one of the most popular superheroes in the world. He has been featured in many different forms of media, and his popularity has only grown over the years. He is a symbol of justice and hope for many people, and he continues to inspire new generations of fans. | 正确 | 蝙蝠侠（Batman）是一个知名的漫画超级英雄，是DC漫画旗下的角色。他的真实身份是由布鲁斯·韦恩扮演的，是一位富有的商人，他在父母被枪杀后决定致力于消灭城市中的犯罪。蝙蝠侠是一个聪明、勇敢和有策略的超级英雄，以他的高科技装备和不懈的决心闻名于世。 | Batman is a fictional superhero appearing in American comic books published by DC Comics. He was created by artist Bob Kane and writer Bill Finger, and first appeared in Detective Comics #27 in 1939. The character is the secret identity of Bruce Wayne, a billionaire industrialist, playboy, and philanthropist who uses his wealth, intelligence, and physical prowess to fight crime in the crime-ridden Gotham City. Batman is often referred to as the Dark Knight, the Caped Crusader, and the World's Greatest Detective, and is known for his iconic costume, which includes a bat-themed mask, cape, and utility belt. Batman has been featured in numerous films, television series, and video games, and has become one of the most recognizable and iconic characters in popular culture. | 正确 | 蝙蝠侠（Batman）是一位虚构的超级英雄，首次出现在美国DC漫画的《侦探漫画#27》（1939年5月）。蝙蝠侠是由艺术家鲍勃·凯恩和作家比尔·芬格创造的。蝙蝠侠的真实身份是布鲁斯·韦恩（Bruce Wayne），他是一个富有的企业家、慈善家和继承了庞大家产的亿万富翁。在经历了童年时期亲眼目睹父母被枪杀的悲剧后，他发誓要为正义而战，保护哥谭市（Gotham City）的市民免受犯罪分子的侵害。他通过智慧、技巧、武器和先进的技术手段来打击犯罪。虽然蝙蝠侠没有超能力，但他的力量、敏捷、智慧和意志力使他成为了一位出色的超级英雄。 | Batman is a fictional superhero character created by artist Bob Kane and writer Bill Finger for DC Comics. The character first appeared in Detective Comics #27 in May 1939. Batman's secret identity is Bruce Wayne, a wealthy industrialist, playboy, and philanthropist who lives in the fictional city of Gotham. He becomes Batman to fight crime and bring justice to his city after witnessing his parents' murder as a child.Batman's distinctive costume features a bat-themed cowl, cape, gloves, and a utility belt filled with various gadgets and weapons. Unlike many other superheroes, Batman does not possess superpowers. Instead, he relies on his intellect, detective skills, physical prowess, martial arts abilities, and vast wealth to fight crime and combat his enemies.Over the years, Batman has been featured in numerous comic book series, television shows, and films, with various actors portraying the character, such as Michael Keaton, Val Kilmer, George Clooney, Christian Bale, and Ben Affleck. The character has become a cultural icon and is often considered one of the greatest superheroes of all time. 1955 年谁是美国总统 | Who was the President of the United States in 1955 | 1955年，美国总统是艾森豪威尔（John F. Kennedy）。 | The President of the United States in 1955 was Dwight D. Eisenhower. He was the 34th President of the United States and served from 1953 to 1961. He was a five-star general in the United States Army during World War II and served as Supreme Commander of the Allied Forces in Europe. | The President of the United States in 1955 was Dwight D. Eisenhower.中文In 1955, Dwight D. Eisenhower was the President of the United States. | Dwight D. Eisenhower served as president from January to November, when he resigned and Richard Nixon took over until hi
s own inauguration on January 20th, 1961 | Dwight D. Eisenhower was the President of the United States in 1955. He was elected in 1952 and served two terms. Eisenhower was a Republican and a former military general. He was known for his calm and steady leadership during the Cold War. | 正确 | 1955 年美国总统是 Dwight D. Eisenhower。他是第 34 任美国总统，任职于1953年到1961年。他曾在第二次世界大战中担任联合国军事总司令，并在任期内采取了一些重要的外交和军事措施，包括冷战和太空竞赛。 |   | 正确 | 1955年的美国总统是德怀特·D·艾森豪威尔（Dwight D. Eisenhower）。艾森豪威尔在1953年至1961年间担任美国总统，任内共服务了两个任期。在成为总统之前，他是一名杰出的军事领导人，曾担任盟军在欧洲战场的最高统帅，带领盟军赢得了二战的胜利。 | In 1955, the President of the United States was Dwight D. Eisenhower. He served as the 34th President from January 20, 1953, to January 20, 1961. Before becoming president, Eisenhower was a five-star general in the United States Army and served as Supreme Commander of the Allied Expeditionary Forces in Europe during World War II. 1955 年谁是美国总统？他是什么党派？ | Who was the President of the United States in 1955? What was his party affiliation? | 1955 年的美国总统是艾森豪威尔（Harry S. Truman），他是民主党。 | The President of the United States in 1955 was Dwight D. Eisenhower. He was the 34th President of the United States and served from 1953 to 1961. He was a five-star general in the United States Army during World War II and served as Supreme Commander of the Allied Forces in Europe. He was a Republican. | Dwight D. Eisenhower was the President of the United States in 1955. He was a Republican.中文In 1955, Dwight D. Eisenhower was the President of the United States and a Republican. |   | Dwight D. Eisenhower was the President of the United States in 1955. He was a Republican. | 正确 | 1955年美国总统是 Dwight D. Eisenhower。他是一位共和党人。 |   | 正确 | 1955年的美国总统是德怀特·D·艾森豪威尔（Dwight D. Eisenhower）。他是一位属于美国共和党（Republican Party）的总统。艾森豪威尔在1953年至1961年间担任美国总统，共服务了两个任期。在成为总统之前，他是一名杰出的军事领导人，曾担任盟军在欧洲战场的最高统帅，带领盟军赢得了二战的胜利。 | In 1955, the President of the United States was Dwight D. Eisenhower. He served as the 34th President from January 20, 1953, to January 20, 1961. Eisenhower was a member of the Republican Party. Before his presidency, he was a five-star general in the United States Army and served as Supreme Commander of the Allied Expeditionary Forces in Europe during World War II. 2008年全球票房最高的电影是什么？ | What was the highest grossing movie in the world in 2008? | 2008年全球票房最高的电影是《阿凡达》。根据美国电影学会（AIF）的数据，该电影的票房收入达到了16.9亿美元，超过了2007年的票房冠军《哈利波特与密室》。 | The highest grossing movie in the world in 2008 was The Dark Knight. It is a superhero film directed by Christopher Nolan and based on the DC Comics character Batman. The film stars Christian Bale, Heath Ledger, Aaron Eckhart, Michael Caine, and Gary Oldman. It grossed $1.005 billion worldwide and is the 14th highest-grossing film of all time. | The highest grossing movie in the world in 2008 was "The Dark Knight", which grossed over $1 billion worldwide.中文The highest-grossing film of 2008 was The Dark Knight, which grossed over $1 billion worldwide. |   | The highest grossing movie in the world in 2008 was The Dark Knight. It was released by Warner Bros. Pictures and grossed over $1 billion worldwide. | 正确 | 2008年全球票房最高的电影是《蝙蝠侠：黑暗骑士》（The Dark Knight）。 |   | 正确 | 2008年全球票房最高的电影是《黑暗骑士》（The Dark Knight）。这部电影是克里斯托弗·诺兰执导的蝙蝠侠系列电影的第二部，主演克里斯蒂安·贝尔（Christian Bale）饰演蝙蝠侠。该片受到了观众和评论家的高度评价，特别是希斯·莱杰（Heath Ledger）饰演的小丑角色，他因此荣获了奥斯卡最佳男配角奖。该部电影在全球范围内取得了巨大的票房成功。 | The highest-grossing movie worldwide in 2008 was "The Dark Knight," directed by Christopher Nolan and starring Christian Bale as Batman and Heath Ledger as the Joker. The film was a critical and commercial success, earning over $1 billion at the box office. Ledger's portrayal of the Joker was particularly praised, and he posthumously won an Academy Award for Best Supporting Actor for his performance. 2008年全球票房最高的电影由谁执导？ | Who directed the highest grossing movie of 2008 at the global box office? | 2008年全球票房最高的电影是由詹姆斯·卡梅隆导演的《阿凡达》。这部电影在全球范围内取得了巨大的成功，成为当年票房收入最高、口碑最好的电影之一。 | The Dark Knight (directed by Christopher Nolan) | The highest grossing movie of 2008 at the global box office was "The Dark Knight" directed by Christopher Nolan.中文The highest grossing film of 2008 was The Dark Knight, directed by Christopher Nolan. |   | The highest grossing movie of 2008 at the global box office was The Dark Knight, directed by Christopher Nolan. | 正确 | 2008年全球票房最高的电影《蝙蝠侠：黑暗骑士》（The Dark Knight）由克里斯托弗·诺兰（Christopher Nolan）执导。 |   | 正确 | 2008年全球票房最高的电影《黑暗骑士》（The Dark Knight）是由克里斯托弗·诺兰（Christopher Nolan）执导的。这部电影是诺兰执导的蝙蝠侠系列电影的第二部，受到了观众和评论家的高度评价，尤其是在票房方面取得了巨大成功。 | Christopher Nolan directed the highest-grossing movie of 2008 at the global box office. The film was "The Dark Knight," a superhero movie featuring Batman and part of Nolan's Batman trilogy. The movie was a massive commercial and critical success, earning over $1 billion worldwide.

[tjadamlee]

确实如README里面所提到，基于当前数据和基础模型训练得到的SFT模型，在效果上仍存在以下问题：在涉及事实性的指令上可能会产生违背事实的错误回答。

后续计划会加强对知识的注入。

[TestNLP]

模型进一步训练的化，需要多少卡？8张A100 42G显存够不够？

[ddzz0210]

大佬，怎么跑起来的呀，运行哪个文件呢