Data Analysis and Generation

[JensLe97]

Ideas:

• ChatGPT style search (definitions, source code,)
• Input could be text (questions), images, video
• Output could be text and images
• Generate (VAE, GAN) Multimodal (Text, Images, Video, Audio, Sensor):
  • Images
  • Images that help to explain/understand the tasks/information
  • Insert additional content into an image that has been missing before
  • Modify images so that certain properties change:
    • Style
    • Sharpen
    • Text
    • Design of a certain object
  • Text:
  • Tasks for a specific topic
  • Structured code with these tasks to insert into flutter project
  • Synthetic data Using Transformers
• Upload image/video -> Data Analysis:
  • object detection
  • image classification
  • Semantic segmentation
  • Panoptic segmentation
  • Motion object Tracking
  • Detect text and read it
  • Tagging (labeling)
  • Sim to real / Real to sim

1. Train/finetune text-to-text model: ask any knowledge question (science, homework, etc.)

   • Use this as a personal AI assistant (chat)
   • Private chat with other users (help of AI)
   • Group chat wither other users (help of AI)

2. Train/finetune text-to-image model: ask model to generate postings for other users (comics, picture book, pictures for tasks, etc.)

   • Use this to add it to your own material collection
   • Post the generated content to others in form of reel/short
   • Comment/Like/Share content

3. Train/finetune image-to-text model: ask to solve the tasks on a worksheet, explain the content

   • Use this as a help for your homework/knowledge base (upload and answer page)
   • Find a caption/description for the image

4. Image-to-image model: ask to solve the tasks on a worksheet and edit the solution into the image

   • Use this as a help for your homework

5. Image+Text-to-image: Ask a questions that refers to the image (what is the content? directions on what to do with the image)

6. Image+Text-to-text:

7. Image-to-image+text:

8. Text-to-image+text: Ask a question that is answered with explaining pictures

9. Image+Text-to-image+text:

[JensLe97]

Google Colab: T4 GPU 3 hours and 30 minutes

[JensLe97]

https://lightning.ai/ 22 Free GPU hours per month