Closed devdimit93 closed 6 months ago
Any updates on this issue, It too face the same problem.
It looks like this might be an issue with windows paths having backslashes in them, which causes invalid escape characters to be inadvertantly used when the file paths are read (and they contain backslashes). A quick workaround would be to turn off USE_FILENAMES in the config because that will stop the file paths from appearing in the prompt, meaning they won't be read by the json formatter. I'll try to get on the actual fix for this later today when I have more time and am not at work. Thanks for the bug report!
I think the latest push will have solved this (should solve all future JSON decode issues in fact). Please let me know if it works now after pulling @suridol @devdimit93
I had same issue but looks like it cuz cant connected to api (i used tabbyapi OAI as backend, http://127.0.0.1:5000/v1/, it work on Sillytavern OAI mode)
File "/home/omni/miniconda3/envs/tb/lib/python3.11/site-packages/openai/_base_client.py", line 1548, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
0%| | 0/13 [00:07<?, ?it/s]
Traceback (most recent call last):
File "/home/omni/exllamav2/augmentool/processing.py", line 440, in <module>
asyncio.run(main())
File "/home/omni/miniconda3/envs/tb/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/home/omni/miniconda3/envs/tb/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/omni/miniconda3/envs/tb/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/omni/exllamav2/augmentool/processing.py", line 189, in main
await control_flow_functions.filter_all_questions(
File "/home/omni/exllamav2/augmentool/augmentoolkit/control_flow_functions/control_flow_functions.py", line 1462, in filter_all_questions
await future
File "/home/omni/miniconda3/envs/tb/lib/python3.11/asyncio/tasks.py", line 615, in _wait_for_one
return f.result() # May raise f.exception().
^^^^^^^^^^
File "/home/omni/exllamav2/augmentool/processing.py", line 105, in run_task_with_limit
return await task
^^^^^^^^^^
File "/home/omni/exllamav2/augmentool/augmentoolkit/control_flow_functions/control_flow_functions.py", line 1358, in determine_worthy
judgement = await judge.generate(arguments={"text": p[0], "textname": p[1]})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/omni/exllamav2/augmentool/augmentoolkit/generation_functions/generation_step_class.py", line 152, in generate
raise Exception("Generation step failed -- too many retries!")
Exception: Generation step failed -- too many retries!
here is config.yaml
PATH:
INPUT: "./raw_txt_input"
OUTPUT: "./output"
DEFAULT_PROMPTS: "./prompts" # the baseline prompt folder that Augmentoolkit falls back to if it can't find a step in the PROMPTS path
PROMPTS: "./prompts" # Where Augmentoolkit first looks for prompts
API:
API_KEY: "b4c8725b9cfa36e1b76666dcf70f553d" # Add the API key for your favorite provider here
BASE_URL: "http://127.0.0.1:5000/v1/" # add the base url for a provider, or local server, here. Some possible values: http://127.0.0.1:5000/v1/ # <- local models. # https://api.together.xyz # <- together.ai, which is real cheap, real flexible, and real high-quality, if a tad unreliable. # https://api.openai.com/v1/ # <- OpenAI. Will bankrupt you very fast. # anything else that accepts OAI-style requests, so basically any API out there (openrouter, fireworks, etc etc etc...)
LOGICAL_MODEL: "command-r-p-4.5bpw" # model used for everything except conversation generation at the very end
LARGE_LOGICAL_MODEL: "command-r-p-4.5bpw" # model used for conversation generation at the very end. A pretty tough task, if ASSISTANT_MODE isn't on.
SYSTEM:
USE_FILENAMES: False # give the AI context from the filenames provided to it. Useful if the filenames are meaningful, otherwise turn them off.
ASSISTANT_MODE: False # If False, the conversations generated are between a user and an AI assistant. If True, the generated convs are between fictional characters in historical or fictional settings, with randomized personalities (some are nsfw by default, because a lot of model creators make models for that purpose. Change this (or amplify it) in ./augmentoolkit/generation_functions/special_instructions.py, it only requires changes to some strings.)
DOUBLE_CHECK_COUNTER: 3 # How many times to check a question and answer pair during each validation step. Majority vote decides if it passes that step. There are three steps. So most questions are by default checked around 9 times (fewer if the first two checks for a step pass, obviously).
USE_SUBSET: True # Whether to take only the first 13 chunks from a text during the run. Useful for experimenting and iterating and seeing all the steps without costing too much money or time.
REARRANGEMENTS_TO_TAKE: 3 # How many times to rearrange the questions and answers for generating different conversations from the same group of questions and answers.
CONCURRENCY_LIMIT: 50 # Hard limit of how many calls can be run at the same time, useful for API mode (aphrodite automatically manages this and queues things, as far as I know)
COMPLETION_MODE: False # Change to false if you want to use chat (instruct) mode; this requires .json files in your chosen prompts directory, in the OpenAI API format. Not all APIs support completion mode.
MODE: "api" # can be one of "api"|"aphrodite"
GRAPH: False # Whether to show a pretty graph after filtering out stuff not worthy for questions, useful for seeing whether or not your text is suitable for making data from using Augmentoolkit by default. Will pause the pipeline's execution until you close the window, which is why this is false by default.
STOP: True # True = Use stop tokens, False = do not use stop tokens. OpenAI's API restricts you to four stop tokens and all steps have way more than four stop tokens, so you'll need to turn this to False if you're using OAI's API. Also NOTE that if you turn this OFF while using COMPLETION MODE, EVERYTHING WILL BREAK and it will cost you money in the process. Don't do that.
hi @e-p-armstrong , nice repo, keep it up!
I am getting a very similar error, I assume the model is not being found? I am using ollama. The trace:
2024-08-22 12:25:21,396 - ERROR - Above prompt resulted in error, probably the model's fault: 404 page not found
Traceback (most recent call last):
File "/media/jvel/data/PycharmProjects/augmentoolkit/augmentoolkit/generation_functions/generation_step_class.py", line 142, in generate
response, timeout = await self.engine_wrapper.submit_chat(
File "/media/jvel/data/PycharmProjects/augmentoolkit/augmentoolkit/generation_functions/engine_wrapper_class.py", line 127, in submit_chat
stream = await self.client.chat.completions.create(
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/site-packages/openai/resources/chat/completions.py", line 1339, in create
return await self._post(
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/site-packages/openai/_base_client.py", line 1816, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/site-packages/openai/_base_client.py", line 1510, in request
return await self._request(
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/site-packages/openai/_base_client.py", line 1611, in _request
raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: 404 page not found
Traceback (most recent call last):
File "/media/jvel/data/PycharmProjects/augmentoolkit/processing.py", line 508, in <module>
asyncio.run(main())
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/media/jvel/data/PycharmProjects/augmentoolkit/processing.py", line 209, in main
await control_flow_functions.filter_all_questions(
File "/media/jvel/data/PycharmProjects/augmentoolkit/augmentoolkit/control_flow_functions/control_flow_functions.py", line 1094, in filter_all_questions
await future
File "/home/jvel/anaconda3/envs/augmentkit/lib/python3.9/asyncio/tasks.py", line 609, in _wait_for_one
return f.result() # May raise f.exception().
File "/media/jvel/data/PycharmProjects/augmentoolkit/processing.py", line 140, in run_task_with_limit
return await task
File "/media/jvel/data/PycharmProjects/augmentoolkit/augmentoolkit/control_flow_functions/control_flow_functions.py", line 985, in determine_worthy
judgement, judgement_output = await judge.generate(arguments={"text": p[0], "textname": p[1]})
File "/media/jvel/data/PycharmProjects/augmentoolkit/augmentoolkit/generation_functions/generation_step_class.py", line 174, in generate
raise Exception("Generation step failed -- too many retries!")
Exception: Generation step failed -- too many retries!
Here is my config:
API:
API_KEY: 123123412
BASE_URL: http://127.0.0.1:11434/
LARGE_LOGICAL_MODEL: llama3
LOGICAL_MODEL: llama3
# QUANTIZATION_LARGE: gptq
# QUANTIZATION_SMALL: gptq
HUGGINGFACE:
HUB_PATH: Heralax/test-atk-dataset-do-not-use-3
PRIVATE: false
PUSH_TO_HUB: false
PATH:
DEFAULT_PROMPTS: ./prompts
INPUT: ./raw_text_input
OUTPUT: ./output
PROMPTS: ./prompts
PHASE:
PHASE_INDEX: 3
WORK_IN_PHASES: false
SKIP:
ANSWER_RELEVANCY_CHECK: false
FILTER_CHUNKS: false
QUESTION_CHECK: false
SYSTEM:
CHUNK_SIZE: 1900
COMPLETION_MODE: false
CONCURRENCY_LIMIT: 50
CONVERSATION_INSTRUCTIONS: For this conversation, you are generating a chat between
a generalist, generic AI assistant, and a human.
DOUBLE_CHECK_COUNTER: 1
FINAL_ASSISTANT_PROMPT_NO_RAG: 'You are a helpful AI assistant.
'
FINAL_ASSISTANT_PROMPT_RAG: 'You are a helpful AI assistant.
Context information is below:
----------------------
{data}
'
MODE: api
STOP: true
SUBSET_SIZE: 10
USE_FILENAMES: false
USE_SUBSET: false
I installed augmentoolkit for windows 10 and anaconda, but have not correct results in output folder I installed it with next commands.
When I tested it with any documents, ever with sample document from raw_txt_input folder, it return next error result
It returns result texts in console, but has not correct results in outputs folder. Also sometimes program return error beetween text blocks. Sample of error
Here is my config.yaml
I tried ollama with llama2 and mixtral models, reduced CONCURRENCY_LIMIT, DOUBLE_CHECK_COUNTER, tried reduce "max_tokens" and completion_mode and retries in generation_step_class.py. I got same error. Also tried previous commit - also same error result.