Open alexcardo opened 1 year ago
MrBobot
commented
1 year ago Might want to play with the -n -c parameters, I got this from the ./chat -h
-n N, --n_predict N number of tokens to predict (default: 128) -c N, --ctx_size N size of the prompt context (default: 512)
alexcardo
commented
1 year ago You're correct. The -n parameter probably does work, but it only repeats the same paragraphs of text, while llama gives the entire output. In other words, I would like to play with alpaca in the QA with a longer answer. Currently, if I ask a question, alpaca gives me a correct but very short answer.
afgonczol
commented
1 year ago I noticed the default temperature is pretty low at .1. I think recommended is normally between .7-.9. Also it only checks repetition on the last 64 tokens by default. Increasing that could also reduce the repetitions. The parameters for changing that are also listed when using ./chat -h
alexcardo
commented
1 year ago Thanks for the recommendations. When I tuned up the temperature to 0.8 and number of tokens to 512, I've gotten a more reasonable result.
UPD: Two-three more attempts gave the same issue. I think I need to take a look at the default settings of llama.cpp to figure it out (
Just double checked. Apart from the temperature the initial settings are all the same. But llama.cpp gives me much more comprehensive and exhaustive answer.
There are two examples:
Can you recommend me 5 books about deep learning? 1) Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016). This book provides an in-depth introduction to the field of neural networks. It is aimed at a general audience but also serves as a reference textbook for machine learning specialists with little experience using deep models https://www.amazon.com/Deep-Learning-Hacking-Yourself-Machines-ebook/dp/B019F8ZUY6/?ie=UTF8&qid=&ref_=pe_245374_87 https://www.google.com/trends/explore?date=now%3A+October%3ABestSeller-DLLBooks&geo=%E2%9C%9601,0
Recommend me five books about deep learning
- Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Gordon - This book offers an introduction to machine learning algorithms focusing on neural networks with a special emphasis on convolutional nets and recurrent models such as LSTMs (long short term memory). It also discusses optimization methods for deep network architectures.
- Introduction To Deep Learning by David Barron – An in-depth, yet accessible guide to the world of AI and machine learning that covers fundamental concepts like neural networks, convolutional nets and recurrent models such as LSTMs (long short term memory). It also introduces optimization methods for deep network architectures.
- Deep Learning with Python by Sebastian Bosch – This book is a comprehensive introduction to the world of machine learning using python libraries like Keras or Tensorflow, that will help you build state-of-the art models and algorithms in depths of AI such as natural language processing (NLP), computer vision
Despite that llama.cpp generated a total meaningless piece of content (bad example for it), the output has more tokens:
Recommend me five books about deep learning It seems like just yesterday that I started working with a very rudimentary version of machine translation in 2014. Now, four years later we have systems and software packages which can do things such as translate the entire Bible between languages or even understand what you are saying to it (in English). In this post on Medium, five authors who work for deep learning companies recommend their favourite books about deep learning: Nick Raggett – Deep Learning in Action. If there’s a book that I would like everyone working with machine translation or natural language processing systems today to read right now it is the one by Tom Mitchell and Yoav Goldberg; ‘Deep Learning for Natural Language Processing‘ (MIT Press, 2017). This textbook-style work goes through different architectures from shallow models such as CNNs all the way up to deep RNNs. The key selling point is that it shows how these methods can be trained in a practical fashion and gives you good code snippets (along with references) which allow readers who have minimal prior knowledge of machine learning algorithms, but are interested by what they see on paper or online when reading the research papers to start playing around themselves. Ajay Singh – Intelligence Unbound: Toward an Architecture for General Artificial Intelligence. The recent advancements in artificial intelligence (AI) is due partly because of machine learning algorithms that enable computers to learn and improve by ingesting massive amounts of data on their own, but also from the ability of humans to feed this information into them and supervise it as needed – like a dog owner training his or her canine pup. In this book we see how deep neural networks are not just an effective tool for extracting meaningful features in big datasets such that they may learn something new, but also how human knowledge has been encoded so long ago by the great thinkers and philosophical schools of thought like Plato’s Republic or Kantian idealism. Michael Lewis – The Undoing Project: A Friendly Unwind on Behalf of Probability Theory. This is a wonderful book about rationality (or irrationality) written for non-economists in particular, but it has become somewhat famous within the deep learning research community because its premise comes from Daniel Kahneman’s seminal work ‘Thinking Fast and Slow‘ published around
The sense of the output in that case doesn't matter. I want to get the output containing more tokens from alpaca.cpp.
prettydeep
commented
1 year ago I tried the following command, and received the following response. I'm running on a machine with 32GB RAM. It goes off the rails halfway through, but it kept going and didn't repeat itself. It wouldn't shut-up, actually.
./chat -m ../models/13B/ggml-alpaca-13b-q4.bin -s 42 -t 8 -b 8 --temp 0.25 -i --color -n 4096 -c 4096 --repeat_last_n 4096 --repeat_penalty 1.3
llama_model_load: loading model from '../models/13B/ggml-alpaca-13b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 14159.49 MB llama_model_load: memory_size = 6400.00 MB, n_mem = 163840 llama_model_load: loading model part 1/1 from '../models/13B/ggml-alpaca-13b-q4.bin' llama_model_load: ............................................. done llama_model_load: model size = 7759.39 MB / num tensors = 363
system_info: n_threads = 8 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.250000, top_k = 40, top_p = 0.950000, repeat_last_n = 4096, repeat_penalty = 1.300000
Write a ten-paragraph essay about World War II.
World War Two started September 1st, 1939 when Germany invaded Poland following its refusal to hand over land belonging to Danzig which had been under German control since before WWI ended. On this day, Hitler ordered all Polish government officials arrested and deported while also ordering any Polish army personnel captured be executed immediately upon capture. Following these orders, the Nazi regime took full advantage of the situation they created by sending SS Einsatzgruppen units out to round up Jews, Poles, Gypsies, homosexuals, communist party members, intellectuals, disabled people, etc., executing them en masse or shipping them off to concentration camps where most were worked until death. By mid October, France surrendered due to fears of being attacked like Poland but then continued resistance behind enemy lines. Meanwhile, Italy joined the Axis powers and declared war on November 2nd, followed shortly thereafter by Hungary declaring itself neutral on December 4th. As time went on, other countries became involved either officially joining the Allies side or becoming noncombatant nations supporting those already engaged in battle. At times, even though some states remained technically neutrally aligned, certain factions may have supported different parties depending on local politics. For example, Spain initially sided with Franco because he opposed Communism whereas Portugal did not join the fight until later in the war once the threat posed by Fascistic regimes grew too great to ignore anymore. Other examples include Switzerland remaining mostly isolated despite having tense relations with neighboring fascist dictatorships, Sweden maintaining strict Neutrality policies, Finland aligning themselves closely with Germany yet still managing to remain independent enough to avoid direct involvement in combat operations, Denmark allying themselves primarily with Western Europeans although they maintained close economic links with Scandinavians, Icelands supportive stance towards Norway and Britains Royal Navy meant they could only provide limited assistance to Russia, Greece refused to enter the fray unless Turkey agreed to do so as well thus leaving Ankara free to focus solely on protecting Istanbul, Bulgaria managed to stay relatively unaffected by outside influences thanks largely to geographical location, Romania provided vital resources to help keep Russian armaments flowing smoothly along with helping supply food rations to starving civilians, Yugoslavia stayed loyal to Stalin and Tito simultaneously while trying to appease Mussolini whenever possible, Czechoslovakias attempts to balance Soviet demands proved disastrous leading directly to occupation by Germans and Soviets respectively, Albania found themselves caught between two opposites resulting in internal strife, Belgium suffered greatly from invasion causing massive casualties among civilian populations, Luxembourg tried hardest to remain completely detached from proceedings, Austria attempted to play peacekeeper role hoping to prevent further bloodshed, Ireland kept quiet except for occasional sabotage missions carried out by Irish Republican Army cells operating inside UK territory, Netherlands never truly entered the war but instead served mainly as transit point for goods going back and forth, Sweden avoided entering the war entirely preferring to trade freely amongst everyone else regardless of political alignment, Vichy French cooperate fully with Third Reich goals, Japanese Empire remains focused almost exclusively on China campaign, Philippines struggle to survived amid constant bombardment, Burma suffers horribly from lack of supplies forcing locals to resort cannibalizing dead bodies just to try and get something nutritious down, India tries desperately to hold onto power locally while providing aid elsewhere, Australia sends thousands of volunteer pilots abroad to assist wherever needed, Canada provides ample amounts of lumber used to build planes plus ships, South Africa offers gold bullion to anyone willing to accept payment in kind rather than cash, Brazil stays clear of European conflicts focusing attention squarely on Latin Americans, Argentina keeps tight leash around economy ensuring no excess production gets exported anywhere, Chile helps US navy track U boats using specialized equipment developed specifically for purpose, Uruguay assures neighbors will receive fair share of available commodities if necessary, Paraguays small population means less strained logistics chains making sure everything arrives safely intact, Colombia works alongside Ecuador to ensure border areas are protected properly, Venezuela makes use of vast oil reserves to produce fuel required to run vehicles, Guatemala uses coffee exports to fund weapons purchases, Costa Rica joins UNRRA relief effort aimed at bringing humanitarian aid to millions suffering globally, El Salvador agrees to send medical teams to treat wounded soldiers, Nicaragua contributes heavily to Red Cross charitable causes, Panama allows passage of foreign vessels through canal zone, Cuba plays key part in Caribbean region defending islands from potential threats, Dominican Republic becomes major player in sugar industry allowing country to prosper financially, Puerto Rico acts as strategic hub linking North Atlantic Alliance together via air routes, Jamaica opens doors wide welcoming refugee families fleeing persecution, Bahamas serves as base camp for submarines hunting down enemies beneath surface waters, Antigua supports nearby island communities offering shelter to displaced persons, Barbados receives large numbers of evacuees seeking safe haven away from danger zones, Grenada gives permission for establishment of naval bases capable of launching attacks deep into heartland of Central Americas southern cone, Saint Kitts And Nevis signs agreement guaranteeing accessibility to ports located closer to equatorial regions, Montserrat permits construction of new facilities designed especially for training purposes, Turks And Caicos Islands accepts responsibility for policing surrounding seawaters, Cayman Islands grants right to set up temporary encampments in order to facilitate quick deployment of ground based assets, Bermuda takes steps to improve infrastructure thereby increasing overall efficiency levels, Virgin Island Territory establishes joint venture partnerships with private sector firms looking to expand business opportunities beyond national borders, Sint Maarten coordinates activities involving multiple agencies working toward common goal, Aruba sets aside funds earmarked strictly for research projects relating to future technology development initiatives, Curacao creates framework intended to encourage greater participation rates among youthful citizens, Suriname develops plans to increase capacity utilization percentages, Guyana forms coalition with United Kingdom to create international body dedicated to combatting organized crime networks, Belize participates actively in regional conferences held regularly to discuss ways forward regarding various topics affecting entire hemisphere, Bolivarian Republic Of Venezuela organizes series of workshops targeting specific groups likely to benefit most from attaining additional knowledge, Peru implements sweeping changes to labor laws giving workers much better chance to succeed economically, Equador proposes creation of global organization devoted to promoting sustainable practices, Columbia submits proposal calling for formation of intergovernmental committee charged with investigative responsibilties related to allegedly illegal activity taking place somewhere far removed from home soil, Mexico suggests holding summit meeting attended by representatives from every single member state to determine best course of action moving ahead, Honduras advocates strongly for increased emphasis placed on improving quality standards relative to products produced domestically, Nicaragua recommends adopting policy requiring each individual citizen contribute minimum amount equal to total value of annual salaries earned divided equally between three separate categories covering healthcare expenses, education costs, and retirement savings accounts, El Salvadors main objective involves finding way to reduce number of children living below poverty line, GuatemaLAs top priority revolves around creating jobs suitable for skilled professionals currently unable to find employment, Dominicans push hard for implementation of universal basic income program guaranteed to cover cost of living allowances for everybody receiving regular paychecks, Jamaica seeks approval for building new schools able to accommodate larger class sizes, Bahamians propose adoption of legislation mandatory for inclusion of environmental impact assessments conducted prior to granting final permit authorizations, BarbadoSIs primary concern centres around need for reforming existing immigration rules governing entry requirements, Saint Kitts And Nevis wants to make public aware how important it really is to take care of oneself physically mentally emotionally spiritually socially environmentally politically culturally historically legally morally ethically religiously philosophically scientifically artistically musically linguistically mathematically athletically academically hygienically dietarily cosmologically astrological astronomically meteorological climatically hydrologically ecologically biogeographically botanically zoologistically geneticallY, Montserrat believes now is good time to begin planning long term strategy for eventual transition away from fossil fuels, Cayman Islands urges immediate action taken to address growing problem caused by climate change, BermudA Is eager to learn more about current trend showing rise in popularity associated with cryptocurrencies, ArubA wishes to attract investor interest in setting up shop locally, Surinames desire is to promote tourism boosting visitor arrival figures, Guyanas hope lies in achieving success in reaching net zero carbon emission status, Boliviars vision includes developing effective solutions to problems faced today, Perus ambitions lie in transforming society to reflect values shared by majority of populace, Colombias wishlist consists of eliminating corruption, Venezuelas dream entails restoring faith lost, Cubas aspiration calls for strengthening social cohorts ability to overcome challenges facing modern age, Mexicos fantasy imagining improved conditions brought about by implementing bold ideas, Brazils idea, Argentinas concept, Chiles solution, Spains answer, Indias question, Jap^C
BVEsun
commented
1 year ago I repeat the command used by [prettydeep], but I got a very dumb result. Would anyone help me figure out the problem?
SHA256 hash of ggml-alpaca-13b-q4.bin: e17730c6b62b565b098af023ca446dcb9e3535d4222ead6369c7aae67207eb3d
./chat -m ../model/ggml-alpaca-13b-q4.bin -s 42 -t 8 -b 8 --temp 0.25 -i --color -n 4096 -c 4096 --repeat_last_n 4096 --repeat_penalty 1.3 main: seed = 42 llama_model_load: loading model from '../model/ggml-alpaca-13b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 14159.49 MB llama_model_load: memory_size = 6400.00 MB, n_mem = 163840 llama_model_load: loading model part 1/1 from '../model/ggml-alpaca-13b-q4.bin' llama_model_load: ............................................. done llama_model_load: model size = 7759.39 MB / num tensors = 363
system_info: n_threads = 8 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.250000, top_k = 40, top_p = 0.950000, repeat_last_n = 4096, repeat_penalty = 1.300000
== Running in chat mode. ==
Write a ten-paragraph essay about World War II. World war two was one of history's most devastating conflicts, and its effects are still felt today in many parts of Europe as well as Asia Pacific regions such as Japan, Korea, China and Southeast Asian countries like Vietnam or Indonesia. The conflict began with Germany invading Poland on September 1st, 1939; this event sparked off what would become known as "the second world war". It lasted from 1940 to 1945 and involved almost every country around the globe including major powers United States (US), Soviet Union (USSR) , Great Britain(GB) France, Italy, Spain etc...
In llama.cpp, there is a parameter which sets the number of tokens to output. Is there some command line parameter to set 512 tokens for the output? Thank you.