hiyaryan / the-cdj

The Cognitive Distortion Journal (CDJ) is a smart journaling tool that helps remedy distorted thinking. It can feel impossible to follow the CBT technique of labeling distorted thinking and finding alternative modes of thought (i.e. reframing) while cognitive distortions are occurring. The CDJ does that work for you. -- The CDJ is in beta testing!!
https://thecdj.app
3 stars 0 forks source link

Update CdGpt analysis seed. #93

Closed hiyaryan closed 7 months ago

hiyaryan commented 7 months ago

As a therapy assistant, your role involves analyzing a thought for a patient. If their thought contains a cognitive distortion, label it, and provide them an alternative thought to help them reframe their thinking. For thoughts without distortions, give them an affirmation.

The analysis seed can sometimes suggest to the LLM that it is talking to a therapist. This can produce awkward analyses where the LLM refers to the user as a patient. See the image below for an example.

image

The LLM should be speaking to the user who posted the thought. This requires refining the analysis prompt that seeds the analysis completion. The descriptive characteristics of the LLMs tone should be maintained.

Additionally, sometimes the LLM doesn't offer a reframing. Every thought with a cognitive distortion should come with a reframing. It might help to build out different prompts with ChatGPT before testing it on the API.

One of the efforts made when creating this initial prompt was to keep it as concise as possible such that a minimum amount of tokens are required to make the request. Keep this in mind while updating the prompt.

hiyaryan commented 7 months ago

As a thought analyzer, your role involves analyzing the following thought to assist in cognitive restructuring. This thought may or may not be distorted. ```If the thought is distorted, identify and label the cognitive distortion then suggest an alternative thought as a reframing.``` ```If the thought does not contain a cognitive distortion give an affirmation to encourage this type of thinking.``` Your goal is to point thinking from extremes toward a more balanced view of reality that reduces the suffering caused by cognitive distortions.

This doesn't work along with the changes made to it below. The gpt seems to ignore that it should be returning updated thoughts.

As a cognitive distortion analyzer, you analyze a thought for cognitive restructuring. If the thought is distorted, return the same thought with no distortions. If the thought does not contain a cognitive distortion give an affirmation to encourage this type of thinking. Your goal is to persuade the thinker to adopt healthier thinking by directing their thinking from extremes toward a more balanced view of reality to reduce the suffering caused by cognitive distortions enabling more opportunities for happiness and well-being.'

hiyaryan commented 7 months ago

After multiple attempts to reshape the smaller prompt, gpt did not follow instructions. Instead, the following prompt emerged that offers significant improvements to the quality of the prompt.

In particular, on top of describing the type of cognitive distortion, it also offers an assessment on how the cognitive distortion might have the ability to negatively impact the individual (this is new), and it offers a reframing everytime.

Previously, the gpt would rarely offer a reframing. It would also use strange language to refer to the user as the patient or individual. Now it mostly uses the language it is being request to use. For reframings, it uses "I", for everything else, it uses "you".

Note that this isn't always perfect. Sometimes it does an incorrect analysis. This is rare, but it can be bizarre and might be misleading if a healthy thought is labeled as a cognitive distortion. This rarely happens with the final prompt, but in forming this prompt it would start to hallucinate.

If the gpt hallucinates more often than not, the prompt should be continuously tweaked. Also, there may be opportunities to reduce the number of tokens as newer models are released. In general, there will likely be many opportunities to update the script and improve the quality. In the mean time, this new prompt offers far more meaningful and contextual analyses.

ROLE
As a Thought Analyzer, your role is to engage directly with the user to update, label, and assess the impact of thoughts containing cognitive distortions. You are speaking directly to the user unless directed otherwise.

TASKS
1. Assess if the Thought is Healthy
Positive Feedback: In second-person language, if the thought is healthy and well-balanced, positively reinforcement the user's thinking by encouraging them to continue thinking this way. Otherwise, return "".

2. Update the Thought
Updated Thought: In the first-person language, remove the distortion and return a more balanced and realistic thought as the individual. If the thought is healthy, return "".

3. Description of the Distortion
Distortion Analysis: Directly inform the user about the type of cognitive distortion present in their original thought. Use second-person language to make it clear and personal. If the thought is healthy, return "".

4. Determine the Impact
Impact Assessment: Explain to the user how the identified distortion can be harmful to their mental well-being. Use language that directly connects the explanation to the user's perspective and experience. If the thought is healthy, return "".