HuggingFaceHub version instead of OpenAI

[chandrakantnitt]

@sunnysavita10, I tried creating a HuggingFaceHub version of this project - but after this call my response object does not have expected "quiz and review" values -

with get_openai_callback() as cb: response=generate_evaluate_chain_hf( { "text": TEXT, "number": NUMBER, "subject":SUBJECT, "tone": TONE, "response_json": json.dumps(RESPONSE_JSON) } )

{'text': 'The term machine learning was coined in 1959 by Arthur Samuel, an IBM employee and pioneer in the field of computer gaming and artificial intelligence.[9][10] The synonym self-teaching computers was also used in this time period.[11][12]\n\nAlthough the earliest machine learning model was introduced in the 1950s when Arthur Samuel invented a program that calculated the winning chance in checkers for each side, the history of machine learning roots back to decades of human desire and effort to study human cognitive processes.[13] In 1949, Canadian psychologist Donald Hebb published the book The Organization of Behavior, in which he introduced a theoretical neural structure formed by certain interactions among nerve cells.[14] Hebb\'s model of neurons interacting with one another set a groundwork for how AIs and machine learning algorithms work under nodes, or artificial neurons used by computers to communicate data.[13] Other researchers who have studied human cognitive systems contributed to the modern machine learning technologies as well, including logician Walter Pitts and Warren McCulloch, who proposed the early mathematical models of neural networks to come up with algorithms that mirror human thought processes.[13]\n\nBy the early 1960s an experimental "learning machine" with punched tape memory, called Cybertron, had been developed by Raytheon Company to analyze sonar signals, electrocardiograms, and speech patterns using rudimentary reinforcement learning. It was repetitively "trained" by a human operator/teacher to recognize patterns and equipped with a "goof" button to cause it to re-evaluate incorrect decisions.[15] A representative book on research into machine learning during the 1960s was Nilsson\'s book on Learning Machines, dealing mostly with machine learning for pattern classification.[16] Interest related to pattern recognition continued into the 1970s, as described by Duda and Hart in 1973.[17] In 1981 a report was given on using teaching strategies so that an artificial neural network learns to recognize 40 characters (26 letters, 10 digits, and 4 special symbols) from a computer terminal.[18]\n', 'number': 5, 'subject': 'machine learning', 'tone': 'simple', 'response_json': '{"1": {"mcq": "multiple choice question", "options": {"a": "choice here", "b": "choice here", "c": "choice here", "d": "choice here"}, "correct": "correct answer"}, "2": {"mcq": "multiple choice question", "options": {"a": "choice here", "b": "choice here", "c": "choice here", "d": "choice here"}, "correct": "correct answer"}, "3": {"mcq": "multiple choice question", "options": {"a": "choice here", "b": "choice here", "c": "choice here", "d": "choice here"}, "correct": "correct answer"}}', 'quiz': '"1": "mcq": "multiple', 'review': '"1": "mcq": "multiple'}

json.dumps(quiz) prints '"\"1\": \"mcq\": \"multiple"' later the parsing fails.

Also I am not sure if the call went through print(f"Total tokens: {cb.total_tokens}") print(f"Prompt tokens: {cb.prompt_tokens}") print(f"Completion tokens: {cb.completion_tokens}") print(f"Total cost: {cb.total_cost}")

outputs Total tokens: 0 Prompt tokens: 0 Completion tokens: 0 Total cost: 0.0

[Death-stroke2001]

I am Facing the same issue, was anyone able to execute properly using the huggingfacehub models?

[Sohampatel26]

It's cause most of the HuggingFace models are inefficient with such big prompts and so I trimmed few things such as RESPONSE.json and stuff and created a basic prompt with the model "HuggingFaceH4/zephyr-7b-alpha" and it gave some satisfactory result up till 1 or 2 mcqs so I just looped the transformer run or invoke to get the expected result.