Implement AI Judging System

[CraftsMan-Labs]

Objective

Develop a robust AI Judging System to evaluate code submissions quickly and objectively in the AI Hacking League.

Tasks

1. API Integration:

   • Integrate OpenAI's GPT-4 model through their API.
   • Ensure secure and efficient API communication.

2. Custom Prompts Development:

   • Design prompts that focus on functionality, innovation, efficiency, and code quality.
   • Test and refine prompts to achieve accurate evaluations.

3. Scoring Algorithm Design:

   • Construct a weighted scoring system based on evaluation criteria.
   • Validate the scoring methodology with test datasets.

4. Feedback Generation:

   • Develop a mechanism to provide detailed feedback on submissions.
   • Ensure feedback includes strengths, weaknesses, improvement suggestions, and comparative analysis.

Technical Requirements

• Proficiency in Python and API handling.
• Experience with AI models, especially OpenAI's APIs.

Labels

• technical, AI, backend

[wubbyweb]

@ohdearquant is working on this....