search

HeardLibrary / vandycite

0 stars 0 forks source link

Computer vision ML #6

Open baskaufs opened 2 years ago

baskaufs commented 2 years ago

Prior notes:

Saint George on a Bike project

EuropeanaTech x AI series: "Saint George on a Bike" project https://pro.europeana.eu/event/europeanatech-x-ai-introduction-to-the-series-and-the-saint-george-on-a-bike-project

St. George on a bike Colab notebook demo: https://colab.research.google.com/drive/1ZJ3JAWkLkyku7OhSTDf6blbkHE7C6DeT (made a copy of it in my own Drive Colab folder)

Artificial Intelligence for Libraries, Archives & Museums https://sites.google.com/view/ai4lam

O'Reilly books

O'Reilly books web access

Hands-On Machine Learning with Scikit-Learn & TensorFlow - Chapter 13 ?

Introduction to Machine Learning with Python - Chapter 3 uses the faces dataset

Face recognition

Face analysis walk-through

Face detection with OpenCV

Optical character recognition in image?

VanessaPei010213 commented 2 years ago

The main challenges:

  1. the old objects not use anymore e.g.inkwell
  2. Objects with different shapes in the past e.g. plow
  3. unusual actions for everyday life e.g. man killing a horse//only for object/facial recognition on paintings
VanessaPei010213 commented 2 years ago

Global architecture:

  1. NLP for description generation
  2. generate candidate visual relations between BBs
  3. generate candidate labels for each bounding box
VanessaPei010213 commented 2 years ago

Further thoughts: object or sculpture recognition

Similar challenges, longer process

Process:

  1. Images -> Extract region proposal
  2. R-CNN -> CNN feature compute
  3. Objects' labels -> Classify regions
  4. Objects' bounding boxes -> Draw bounding boxes
baskaufs commented 2 years ago

I've created a starter notebook using Keras OCR. It seems to work OK if you don't care about word order.

baskaufs commented 2 years ago

The face recognition script needs to be run on all of the test images that were put into the shared Box folder under various categories (photos, prints, statues, paintings). The script saves the images with the bounding boxes. Then an assessment needs to be done as to how effective it is on those various categories.

baskaufs commented 2 years ago

Stacy C.J. identified: He is a PhD candidate in Anthropology James Zimmer-Dauphinee james.r.zimmer-dauphinee@vanderbilt.edu

as a potential resource who does ML on images.

baskaufs commented 2 years ago

Aiden Layer shared his paper with me in a 2021-12-08 email. He mentioned Clarafai https://www.clarifai.com/ Google Vision AI https://cloud.google.com/vision He also mentioned Harvard Art Museum's AI Explorer https://ai.harvardartmuseums.org/ Paper saved in gallery folder in OneDrive

baskaufs commented 2 years ago

2021-12-13 work session. To use Clarifai, open an application and use the Add inputs button to upload an image. The View in Explorer button (or eyeball icon on the left menu) takes you to thumbnails for uploaded images. When you click on an image, a right sidebar opens up and you can drop down the various menus to see what it automatically did on the general-detection tab. I can't remember, but you may have to have selected general-detection when you set up the app as the model you want.

We also and clicked the "Try for free" button (which changes to "Go to console" after you've tried one). I think we did "Image Classification" and it did OCR as well as object detection.

dannihuang830 commented 2 years ago

Another computer-vision service is https://imagga.com/