Project Title: Adversarial data collection with Gradio
Description:
This project will focus on building an infrastructure to allow exposing models submitted to EvalAI as demos in order to collect adversarial data for the model. As part of the project, we will integrate Gradio with our code upload challenge pipeline to allow deploying the models as web services. Additionally, this web service will record all interactions to curate a "in-the-wild" dataset for each submission.
Deliverables:
[ ] Build system design for EvalAI and Gradio integration
[ ] Set up Gradio integration for a single VQA demo as a proof of concept
[ ] Write a gradio interface wrapper that is modular for model inputs
[ ] Integrate the gradio interface with dockerized model for inference
[ ] Set up auto launching of a worker service for deploying a Gradio app using a celery task from EvalAI django backend
[ ] Add frontend controls to deploy a Gradio web service for a single submission made by participant to a code upload challenge.
[ ] Add support to log interactions made by a user on the gradio web service and push the interactions to a database table specific to a demo.
[ ] Add API support to export the logged interactions for a demo
[ ] Add frontend changes to allow users to download the logged interactions in a standard dataset format
Mentors: - Ram Ramrakhya, Rishabh Jain
Skills Required: - Python, Django, AngularJS, AWS
Project size - 175 hours
Difficulty - Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Project Title: Adversarial data collection with Gradio
Description:
This project will focus on building an infrastructure to allow exposing models submitted to EvalAI as demos in order to collect adversarial data for the model. As part of the project, we will integrate Gradio with our code upload challenge pipeline to allow deploying the models as web services. Additionally, this web service will record all interactions to curate a "in-the-wild" dataset for each submission.
Deliverables:
Mentors: - Ram Ramrakhya, Rishabh Jain
Skills Required: - Python, Django, AngularJS, AWS
Project size - 175 hours
Difficulty - Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Important Links: