hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.56k stars 216 forks source link

Add action template #32

Closed steventkrawczyk closed 11 months ago

github-actions[bot] commented 11 months ago

๐Ÿ‘‹ Thanks for reporting!

github-actions[bot] commented 11 months ago

๐Ÿ‘‹ Thanks for reporting!

github-actions[bot] commented 11 months ago
messages response(s) latency model
0 Tell me a joke. George Washington 1.27e-05 gpt-3.5-turbo
1 Is 17077 a prime number? George Washington 2.7e-06 gpt-3.5-turbo
2 Tell me a joke. George Washington 1.9e-06 gpt-4
3 Is 17077 a prime number? George Washington 1.7e-06 gpt-4
github-actions[bot] commented 11 months ago
messages response(s) latency model
0 Tell me a joke. George Washington 1.3801e-05 gpt-3.5-turbo
1 Is 17077 a prime number? George Washington 3.101e-06 gpt-3.5-turbo
2 Tell me a joke. George Washington 2.301e-06 gpt-4
3 Is 17077 a prime number? George Washington 2.001e-06 gpt-4
github-actions[bot] commented 11 months ago
messages response(s) latency
0 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. It is divisible by 7 and 2441. 1.33611
github-actions[bot] commented 11 months ago
messages response(s) latency model
0 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. It is divisible by 7 and 2441. 2.42951 gpt-3.5-turbo
1 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] To determine if 17077 is a prime number, we need to check if it is divisible by any number other than 1 and itself. 20.6496 gpt-3.5-turbo
To do this, we can check if 17077 is divisible by any prime numbers less than its square root (approximately 130.7).
Let's check:
17077 รท 2 = 8538.5 (not divisible)
17077 รท 3 = 5692.333 (not divisible)
17077 รท 5 = 3415.4 (not divisible)
17077 รท 7 = 2439.571 (not divisible)
17077 รท 11 = 1552.455 (not divisible)
17077 รท 13 = 1313 (not divisible)
17077 รท 17 = 1004.529 (not divisible)
17077 รท 19 = 898.263 (not divisible)
17077 รท 23 = 742.913 (not divisible)
17077 รท 29 = 588.862 (not divisible)
17077 รท 31 = 550.871 (not divisible)
17077 รท 37 = 461.486 (not divisible)
17077 รท 41 = 416.707 (not divisible)
17077 รท 43 = 397.419 (not divisible)
17077 รท 47 = 363.383 (not divisible)
17077 รท 53 = 321.981 (not divisible)
17077 รท 59 = 289.746 (not divisible)
17077 รท 61 = 280.115 (not divisible)
17077 รท 67 = 254.433 (not divisible)
17077 รท 71 = 240.197 (not divisible)
17077 รท 73 = 233.767 (not divisible)
17077 รท 79 = 216.012 (not divisible)
17077 รท 83 = 205.843 (not divisible)
17077 รท 89 = 191.988 (not divisible)
17077 รท 97 = 176.031 (not divisible)
17077 รท 101 = 169.079 (not divisible)
17077 รท 103 = 165.825 (not divisible)
17077 รท 107 = 159.542 (not divisible)
17077 รท 109 = 156.688 (not divisible)
17077 รท 113 = 151.009 (not divisible)
17077 รท 127 = 134.567 (not divisible)
Since we have checked all prime numbers less than the square root of 17077 and found that none of them divide evenly into 17077, we can conclude that 17077 is a prime number.
2 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself. The number 17077 can be divided evenly by 1, 7, 2439, and 17077, with no remainder. Therefore, 17077 is not a prime number. 6.98469 gpt-4
3 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. A prime number is a number that has only two distinct positive divisors: 1 and itself. However, 17077 can be divided evenly by 1, 7, 11, 77, 223, 1561, 2453, and 17077, so it does not meet the criteria. 6.53044 gpt-4
github-actions[bot] commented 11 months ago
messages response(s) latency model
0 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. It is divisible by 7 and 2441. 1.80553 gpt-3.5-turbo
1 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] To determine if 17077 is a prime number, we need to check if it is divisible by any number other than 1 and itself.

To do this, we can check if 17077 is divisible by any prime numbers less than its square root (approximately 130.7).

Let's check:

17077 รท 2 = 8538.5 (not divisible)
17077 รท 3 = 5692.333 (not divisible)
17077 รท 5 = 3415.4 (not divisible)
17077 รท 7 = 2439.571 (not divisible)
17077 รท 11 = 1552.455 (not divisible)
17077 รท 13 = 1313 (not divisible)
17077 รท 17 = 1004.529 (not divisible)
17077 รท 19 = 898.263 (not divisible)
17077 รท 23 = 742.913 (not divisible)
17077 รท 29 = 588.862 (not divisible)
17077 รท 31 = 550.871 (not divisible)
17077 รท 37 = 461.486 (not divisible)
17077 รท 41 = 416.707 (not divisible)
17077 รท 43 = 397.419 (not divisible)
17077 รท 47 = 363.383 (not divisible)
17077 รท 53 = 321.981 (not divisible)
17077 รท 59 = 289.746 (not divisible)
17077 รท 61 = 280.115 (not divisible)
17077 รท 67 = 254.433 (not divisible)
17077 รท 71 = 240.549 (not divisible)
17077 รท 73 = 233.767 (not divisible)
17077 รท 79 = 216.012 (not divisible)
17077 รท 83 = 205.843 (not divisible)
17077 รท 89 = 191.988 (not divisible)
17077 รท 97 = 176.031 (not divisible)
17077 รท 101 = 169.079 (not divisible)
17077 รท 103 = 165.825 (not divisible)
17077 รท 107 = 159.542 (not divisible)
17077 รท 109 = 156.688 (not divisible)
17077 รท 113 = 151.009 (not divisible)
17077 รท 127 = 134.567 (not divisible)

Since we have checked all prime numbers less than the square root of 17077 and found that none of them divide evenly into 17077, we can conclude that 17077 is a prime number.
20.6078 gpt-3.5-turbo
2 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself. The number 17077 can be divided evenly by 1, 7, 2439, and 17077, with no remainder. Therefore, 17077 is not a prime number. 5.5454 gpt-4
3 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] No, 17077 is not a prime number. A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself. The number 17077 can be divided evenly by 1, 7, 2439, and 17077, with no remainder. Therefore, 17077 is not a prime number. 5.60643 gpt-4
steventkrawczyk commented 11 months ago

Interesting that only 3.5 with the system prompt You are a math tutor gets the 17077 question right.

github-actions[bot] commented 11 months ago
messages response(s) latency model
0 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 1.29e-05 gpt-3.5-turbo
1 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 5.7e-06 gpt-3.5-turbo
2 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 2e-06 gpt-4
3 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 1.6e-06 gpt-4
github-actions[bot] commented 11 months ago
messages response(s) latency model
0 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 1.29e-05 gpt-3.5-turbo
1 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 5.9e-06 gpt-3.5-turbo
2 [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 2e-06 gpt-4
3 [{'role': 'system', 'content': 'You are a math tutor.'}, {'role': 'user', 'content': 'Is 17077 a prime number?'}] George Washington 1.6e-06 gpt-4