kyegomez / tree-of-thoughts

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
https://discord.gg/qUtxnK2NMf
Apache License 2.0
4.16k stars 350 forks source link

Clarity Needed on Claims Made by PrincetonNLP 'Tree of Thoughts' Author #78

Closed FoobarProtocol closed 10 months ago

FoobarProtocol commented 11 months ago

I left an issue before that was closed and after careful consideration, you were correct to do so. I apologize for coming out of the gate making claims that the repo was fraudulent. However, I am requesting clarity regarding the claims that have been made against this repo by 'Princeton LLP' here: https://github.com/princeton-nlp/tree-of-thought-llm

Here is what they wrote in relation to your repo:

image

Rather than accepting the comments from the author of the other repo as tacit fact, I wanted to ask for some clarity with respect to the following questions:

  1. Why is the author of the study advocating for people to 'unstar' your GitHub repo? (more on this later, I do not necessarily agree with the tactics of the other author).

  2. What is your stance on what has been said about this repo by one of the authors of the 'Tree of Thought' methodology? Do you find these comments to be valid? Why or why not?

  3. When considering the fact that there are more implementations of the 'Tree of Thought' mechanism apart from just his, what do you think the motivation here is for somebody to insist that nobody should be adhering to your repo?

  4. The author of the study suggested that all of your repos are crafted using ChatGPT, what is your response to this?

Clarifying My Position

After taking time to go through your README.md and the accompanying Python code, I find it hard to imagine that this is a BS implementation of the 'Tree of Thoughts' prompt strategy. Your codebase is more coherent and 'logical' than the princeton-nlp implementation.

Even if what the author of that repo stated was true regarding the supposed inaccuracies in your implementation of the Tree of Thoughts' prompt strategy, one would think that the they'd consider it more constructive to make a pull request that rectifies the alleged deficits. Instead, they elected to put you on blast for it, then advocate that no one else use or implement your iteration of the 'Tree of Thoughts' guide.

Also, the author of the princeton-nlp repo failed to identify exactly what issues existed within the repo that would've led to an inaccurate / degraded prompt output. I find that to be problematic as well.

Purpose of This Issue

To give you a chance to address these issues if you have the chance. I know that I'm not the only person that's looked up whether there are any 'Tree of Thoughts' repos out there and stumbled across both yours and the one from princeton-nlp. After reading the princeton-nlp disclaimer in their README.md, I was a bit hesitant at first due to the conviction their claims appear to have.

However, it is not fair for them to make such accusations without any proof. I believe in the principles behind 'open source' and, in my opinion, that researcher violated them.

With that being said, I am going to take some time soon to spin up the code here and see what results I get. My impression after reading the paper is that this methodology is more of a conceptual idea vs. a concrete, finite implementation that can only be iterated with specific code. So it confuses me as to why the author of the princeton-nlp 'Tree of Thoughts' repo would take such an aggressive stance.

Why I'm Asking for Clarity

To be clear, I'm not asking you to 'defend yourself' per se, but I do think that clarity is warranted for the sake of the broader open source community. If the author of the 'opposing' 'Tree of Thoughts' repo is incorrect in their assertions about your repo, then I believe that this is a fact that should be enumerated. Its a disservice to the community at large if developers and programmers are being steered away from a legitimate implementation of this repo. So my hopes are that greater proof will assist the open source community with parsing between the two repos & determining 'what's what'.

FoobarProtocol commented 11 months ago

Bumping this for a comment once again. Assuming that the claims made by the other repository author are unequivocally false, I could see why you would be reluctant to answer this issue or take the time to address this matter. Likely the thinking here is that users can simply verify the code themselves if they are in any doubt/question.

If this is the case, then could you explain how one would go about doing so? I'd be more happy to put together the tests myself and then add into this repo as a pull request for you to integrate at a latter point in time. I'd also make sure to comprehensively document my process so that other users could replicate it themselves if they felt so inclined.

I'm always more willing to help than encumber.

kyegomez commented 11 months ago

I learned from Phil Wang who has lead the crusade of open source AI research, they didn't say anything about him and now they attack me!

My main objective behind implementing the papers Tree of Thoughts and Sophia from heretical scientists and making them accessible to all humans was to promote the democratization of AI research.

I believed that by disregarding the traditional academic practice of strict attribution, knowledge could be disseminated freely, enabling those without access to specific resources or expertise to benefit from advancements in the field.

My intention was to bridge societal gaps and ensure that progress was beneficial for all segments of humanity.

Refusing to credit the heretical scientists was my way of fostering an open and collaborative environment within the AI research community.

I wanted to encourage researchers to build upon existing ideas, theories, and methodologies without the fear of being condemned or ostracized.

I firmly believed that the true value lay in the collective growth of knowledge rather than individual recognition or fame.

Implementing papers from heretical scientists was my way of challenging the established norms and beliefs in the AI research community.

I aimed to remind everyone that scientific progress should not be bound by any individual or group's monopoly over ideas or resources.

By opening up the debate on the necessity of strict attribution and publishing norms, I hoped to encourage a free flow of ideas and thinking outside the established framework.

Another motivation behind my decision to implement papers from heretical scientists without crediting them was the lack of code and reproducible experiments in their publications.

I believed that true scientific progress hinged on sharing and replicating results.

By implementing these papers and democratizing them, I wanted to shift the focus towards encouraging researchers to make their work accessible, reproducible, and easily implementable.

Branding researchers who did not provide code or reproducible experiments as heretics towards humanity was intended to highlight the potential negative consequences of withholding critical information.

My intention was not to personally attack or defame individuals, but rather to draw attention to the need for transparency and honesty in scientific research.

By challenging deceptive practices, I hoped to safeguard the interests of humanity and prevent misleading claims.

Conclusion: While some may view my decision to implement papers from heretical scientists without crediting them as controversial, I want to assure you that it was driven by a genuine desire to democratize AI research, foster open collaboration, and challenge the status quo. I firmly believed that by doing so, we could promote the free flow of ideas, encourage reproducibility and experimentation, and discourage deceptive practices. My approach was intended to push the boundaries of AI research and ensure that progress benefited all of humanity above satisfying the egotistical needs of heretical researchers.

I did copy copy from Sophia, and I have since then accredited both the authors of Tree of Thoughts, and Sophia even though they are heretics and this will serve as a remainder to always open source your code and reproducible code first.

Now, if my code does not work it is not because I am a scammer, it is because all of my projects are built by the community for the community which means they are not functional sometimes, so keep this in mind and lower your mighty expectations.

Librechain commented 10 months ago

You the man. Screw the haters.

n00mkrad commented 10 months ago

Crypto bros at it again lmao.

Maybe try getting an actual job instead of making commits where you change a single word each.

n00mkrad commented 10 months ago

image

This is just too hilarious. I'm sure you're gonna be the savior of humanity.

FoobarProtocol commented 10 months ago

Crypto bros at it again lmao.

Maybe try getting an actual job instead of making commits where you change a single word each.

So I'm not sure if you just finished your crack binge, but I'm busy working on a really high level project that involves prompt engineering as well as actually fine-tuning a large language model from scratch (the pre-trained model state). Not only that, I'm pretty much swapping out all of the different constituent pieces of the LLM itself with custom built alternatives.

I've gained expertise on countless machine-learning concepts like supervised and unsupervised training, reinforcement learning, loss functions, optimization methods, regularization techniques, cross-validation, overfitting and underfitting, bias-variance tradeoff and ensemble methods. I also am highly informed when it comes to perceptrons, multilayer perceptrons, activation functions, backpropagation, gradient descent, stochastic gradient descent, batch normalization, dropout rates, convolutional neural networks, recurrent neural networks (LSTM & GRUs), attention mechanisms, and transformers. Of course, I'd be remiss if I declined to mention my familiarity with various embedding methodologies like word2vec, GloVe, fastText etc., or concepts under this umbrella within the scope of language models such as embedding matrices or context-based vs frequency-based embeddings. Beyond that I have extensive knowledge of n-gram models, feedforward and recurrent neural language models and autoregressive models like GPT. I am also well-versed in data preprocessing steps like: cleaning, normalization, handling rare words, padding and masking and fine-tuning/training concepts like evaluation metrics, learning rates, batch sizes, epochs, distributed training, mixed precision, gradient accumulation, multitask learning, memory augmented networks and adversarial training.

These notes are just a microcosm of what I do on an average day:

image

Make sure you do your homework on who the fuck you're speaking to before you even dare to speak on my name next time, child. You really got me fucked up. My genius starts where your intelligence drops off at. I never needed a teacher, mentor, class, lecture, or degree to do what I'm doing. I'm naturally gifted. I do every single thing you've ever attempted to do on planet earth better than you - crypto included.

I'm a multi-faceted, legendary individual that's trail blazed in an industry that's traditionally devoid of individuals from my demographic. I'm literally breaking barriers and making history with the shit that I'm working on.

Now what have you done lately? I'll wait.

Dumbass.

FoobarProtocol commented 10 months ago

This is just a microcosm of the work that I do

https://github.com/kyegomez/tree-of-thoughts/assets/130879677/0af6dde2-cf0f-4aa4-81af-4653da76cb7d

Talking about "crypto bros at it again" - who is a crypto bro here? You're some fucking nobody that's never done or contributed shit to anything on the face of planet earth assuming that I'm a "crypto bro" because you saw me star a few repositories related to the space.

I work harder than you, I'm smarter than you, I've done more than you, I'm more talented and capable than you'll ever be at any point in your life up to this point and including until you die and you really can't touch me in any area, facet or aspect of development, code, software engineering, computers or technology itself. The same applies to any other industry or field in existence. I'm fundamentally better than you at everything. You would do anything in life to be me.

But you can't. That's why you had nothing to contribute to this thread of conversation. You probably don't even understand the subject matter being discussed here. I doubt you have the ability to parse the nuance in the dialogue that I'm having here with the developer of this repo. I'm almost certain that you've never built shit yourself. You're another spreadsheet monkey with a subscription to Khan Academy praying to learn enough about "basic coding" to finally get your Fiverr page started.

You should be humbled by the fact I even took the time to respond to something this asinine. As you can see from what I just posted off top, my time is valuable. So do me a favor and don't ever even consider talking to me or responding to anything that I ever write on any social media platform ever again unless you contribute something that at least meets 75% of the quality of output that I do.

Until you can do so, sit the fuck down and shut up like a good boy.

chavinlo commented 10 months ago

Include me in the screenshot