Silicon Sages - Connect with Them Expedition

[AaronSosaRamos]

Silicon Sages - Connect with Them Expedition - PR

Summary

This PR introduces several enhancements and new features to the Connect with Them Expedition. The document loaders included in this PR support the following file types:

• PDF
• CSV
• TXT
• MD
• URL (e.g., Wikipedia or other info sources)
• PPTX
• DOCX
• XLS / XLSX
• XML
• GOOGLE DOCS
• GOOGLE SHEETS
• GOOGLE SLIDES
• GOOGLE PDF
• IMAGES (PNG / JPG / JPEG)

Changes

• Reused the document loaders from Epic 7.2 #61.
• Reused the utility file allowed_file_extensions.py to manage file types robustly.
• Created a new tool.py to integrate the designed workflow for generating Connect with Them assignments.
• Created a new core.py to integrate the Connect with Them generator workflow from tool.py appropriately.
• Decoupled the input arguments as a best practice to allow for flexible maintainability.
• Created new error classes to improve the error mapping process.
• Enabled a multimodal approach for generating responses from images, representing an innovative proposal for the project.

Testing

Each document loader was tested using pytest, with scenarios and edge cases properly managed.

Results

All document loaders are functioning as expected, yielding optimal results.

Notes

For Google Docs, Slides, Sheets, and PDF files:

• These files need to be shared as public in Google Drive.
• Files created in Google Drive need to be downloaded and then uploaded to be detected. This was determined to be the most appropriate approach, as the GoogleDriver loader from LangChain requires OAuth2, which is not suitable for production deployment.

How to Test

1. Clone the repository to your local environment.

2. Create and activate a virtual environment.

3. Run pip install -r requirements.txt to install the required libraries.

4. Create a .env file with the following fields:

   • ENV_TYPE (e.g., dev)
   • GCP_PROJECT_ID (your project ID from Cloud Console)
   • GOOGLE_API_KEY (your API key from AI Studio)

5. Start the application by running ./local-start.sh.

6. Add sample requests and responses for each file type mentioned above (with screenshots), and test them.

Request interface

Schema:

class ConnectWithThemArgs(BaseModel):
    grade_level: str = Field(..., description="The grade level the teacher is instructing.")
    task_description: str = Field(..., description="A brief description of the subject or topic the teacher is instructing.")
    students_description: str = Field(..., description="A description of the students including age group, interests, location, and any relevant cultural or social factors.")
    file_url: str = Field(..., description="URL of any relevant file associated with the teaching material.")
    file_type: str = Field(..., description="The type of the file")
    lang: str = Field(..., description="The language in which the subject is being taught.")

Example:

{
   "user":{
      "id":"string",
      "fullName":"string",
      "email":"string"
   },
   "type":"tool",
   "tool_data":{
      "tool_id":10,
      "inputs":[
         {
            "name":"connect_with_them_args",
            "value":{
               "grade_level":"College",
               "task_description":"Develop a Neural Network Architecture for emulating Data Science Workflow",
               "students_description":"Students aged 19-22, interested in hands-on activities, located in an urban area, with diverse cultural backgrounds.",
               "file_url":"http://ijsmsjournal.org/2021/volume-4%20issue-4/ijsms-v4i4p137.pdf",
               "file_type":"pdf",
               "lang":"en"
            }
         }
      ]
   }
}

Response interface

Schema:

class Recommendation(BaseModel):
    project_overview: str = Field(..., description="A detailed description of the project or activity recommendation.")
    rationale: str = Field(..., description="An explanation of why this recommendation is relevant to the students' interests or background.")
    difficulty_level: str = Field(..., description="The difficulty level of the project (e.g., easy, moderate, challenging).")
    required_tools: List[str] = Field(..., description="A list of tools, software, or resources required to complete the project.")
    estimated_time: str = Field(..., description="The estimated time to complete the project or activity.")

class RecommendationsOutput(BaseModel):
    recommendations: List[Recommendation] = Field(..., description="A list of personalized recommendations based on the input.")