A lot of errors

[jbdatascience]

When I excute this code:

`# !cat /content/tree-of-thoughts/huggingfaceExample.py from tree_of_thoughts.treeofthoughts import OpenAILanguageModel, GuidanceOpenAILanguageModel, TreeofThoughts, OptimizedOpenAILanguageModel, OptimizedTreeofThoughts, HuggingLanguageModel

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name="gpt2"

model_tokenizer="gpt2tokenizer"

model = HuggingLanguageModel(model_name, model_tokenizer)

class HuggingLanguageModel:

def init(self, model_name):

self.model = AutoModelForCausalLM.from_pretrained(model_name)

self.tokenizer = AutoTokenizer.from_pretrained(model_name)

def generate_text(self, prompt, max_length=50):

inputs = self.tokenizer(prompt, return_tensors="pt")

outputs = self.model.generate(inputs["input_ids"], max_length=max_length)

generated_text = self.tokenizer.decode(outputs[0], skip_special_tokens=True)

return generated_text

Initialize the HuggingLanguageModel with the GPT-2 model

model_name = "gpt2" model = HuggingLanguageModel(model_name, model_tokenizer="gpt2", verbose=True)

choose search algorithm('BFS' or 'DFS')

search_algorithm = "DFS" # "BFS"

cot or propose

strategy="cot"

value or vote

evaluation_strategy = "value"

gpt2_model = HuggingLanguageModel(model_name)

tree_of_thoughts= OptimizedTreeofThoughts(model, search_algorithm)

input_problem = "use 4 numbers and basic arithmetic operations (+-*/) to obtain 24" k = 5 T = 3 b = 5 vth = 0.5 timeout = 10 confidence = 0.8 #cmodel is confident on performance max_iterations = 40 #tree branh nodes convergence_threshold = 0.01 convergence_count = 5

solution = tree_of_thoughts.solve(input_problem, k, T, b, vth, timeout, confidence_threshold=confidence, max_iterations=max_iterations, convergence_threshold=convergence_threshold, convergence_count=convergence_count)

use the solution in your production environment

print(f"solution: {solution}")`

I get a lot of errors:

" Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Setting pad_token_id to eos_token_id:50256 for open-end generation. Input length of input_ids is 189, but max_length is set to 100. This can lead to unexpected behavior. You should consider increasing max_new_tokens. Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None

...

Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error generating thoughts for state: u s e 4 n u m b e r s a n d b a s i c a r i t h m e t i c o p e r a t i o n s ( + - / ) t o o b t a i n 2 4 Error: num_return_sequences has to be 1 when doing greedy search, but is 5. result: None Saving the current tree and metrics. solution: None

"

What could be the problem and how can it be solved?

[kyegomez]

It ran I believe but the model is just very dumb

[jbdatascience]

It ran I believe but the model is just very dumb

Couldn't we use a better Language Model ? There are a lot of them around these days, their number increasing nearly every day! Yesterday I learned from a model named Falcon: “Falcon 40B,” is now open source for research and commercial use.

Falcon, a fundamental large language model (LLM) trained on one trillion tokens and 40 billion parameters.

Sounds like an Open Source LLM of high quality. Perhaps you could use it in your Tree of Thoughts implementation?

See: https://www.opensourceforu.com/2023/05/open-source-falcon-40b-large-language-model-gets-released/

Details: Falcon-40B is a 40B parameters causal decoder-only model built by TII and trained on 1,000B tokens of RefinedWeb enhanced with curated corpora. It is made available under the TII Falcon LLM License - Hugging Face !!!!! !!!!!

https://huggingface.co/tiiuae/falcon-40b

[kyegomez]

Absolutely, you can plug in and play! Just input your own model and tokenizer! I'm here for you if you seek me

[jbdatascience]

Absolutely, you can plug in and play! Just input your own model and tokenizer! I'm here for you if you seek me

I will do that ! But at the moment I am trying to get the large 40B Falcon working. His little brother Falcon 7B already works, so I could give it a try on this Train of Thoughts implementation !

[kyegomez]

Absolutely! Input the model name and the tokenizer for that model and it should work! Let me know if you need anything