search

spcl / graph-of-thoughts

Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
https://arxiv.org/pdf/2308.09687.pdf
Other
2.1k stars 152 forks source link

Segmentation fault #16

Closed Antiman-cmyk closed 11 months ago

Antiman-cmyk commented 11 months ago

I tried to modify the GoT framework using data multi-threading, but a segmentation fault occurred. Is there any solution? image

Antiman-cmyk commented 11 months ago

The program can start normally, but an error occurs suddenly during operation.

nblach commented 11 months ago

Hi @Antiman-cmyk ,

firstly I want to say that I think it is very nice that you are working on introducing parallel execution for operations. We would be happy to see your code be introduced into the project once it is running.

Regarding your question, without having access to your modified code it is unfortunately impossible to debug. It seems that the segfault occurs during a retry in the chat method of chatgpt.py and it seems to be the call to the OpenAI's ChatCompletion API. I would investigate whether the API is thread-safe and go from there, but it seems that the issue is not with any non thread-safe data structures within GoT. For that reason I will close the issue, but please reopen it or open a new one if you notice that the segfault is due to a bug in GoT.

tonyzhao6 commented 10 months ago

@Antiman-cmyk

Since GoT executes a DAG, it is relatively straightforward to have concurrent (i.e., multi-threaded) execution of operations within the DAG since (by the nature of a DAG) all operations that can execute can be executed concurrently.

Thus, if you are trying to execute multiple operations concurrently (e.g., using a ThreadPool executor), then you can make the following changes:

  1. Construct the initial execution_queue as a set that includes all operations that can execute concurrently based off their predecessor statuses.
  2. Execute all operations in the execution_queue in a thread pool and await their completion (either all at once or as completed).
  3. Construct the next execution_queue as a set.

Those 3 changes will enable concurrent execution of operations within the DAG.