openai / evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Other
15k stars 2.61k forks source link

Vector maping, origins, 2-3-4D definitions. #1569

Open Really-69 opened 2 days ago

Really-69 commented 2 days ago

Describe the bug

CHATGPT4 has unconstrained vector definitions within 3d and 2d graphic generating. Half_Round_Solid_Spatial_Logic_Feedback.pdf AI_Spatial_Corrections_Feedback.pdf

To Reproduce

Ask for the creating of an image including a human face. Ask for one of the eyes to be configured with a halfroudnd solid pupil, oriented vertically with the flat oriented, facing right.

Ask for a ring placed on the facemask of a hockey helmet. As for that ring to support circular motion of a small actuator that hold a hand and will contact the facemask at random intervals around the ring.

Code snippets

No response

OS

Android 14

Python version

N/A

Library version

Openai-evals ver 11

Really-69 commented 2 days ago

Please, re-define where this goes if needed. The "half round" was the first time this issue was noticed by me. Chatgpt and i worked on it for about 30 minutes ending in chatgpt actually admitting an issue. Follow through with reporting it, did not happen even though i specifically asked chat to submit the conversation string to someone. Second issue(which i did not expect due to the previous being fixed,? Hopefully) The ring, adjacent to the origin and being a rotational vector ended is chat giving me a visual image of an egg shape on a 3d grid environment, x, y, z, vectors defined, with the ring oriented 90 degrees off when the chosen plane was implied by the eggs orientation. Also, ring placement was halfway into the body of the egg shape. The object, was oriented outside of the rotational ring, also 90 Degrees off but coincidental with the ring. These two misconceptions are consistent with lack of system vector definitions.