search

Bigjob-team-12 / Project

4 stars 0 forks source link

readme

빅데이터 청년인재 고려대 과정 12조

프로젝트 명 : 찾아주 Cat! :cat: Dog! :dog:

1. 개발 배경 :question:

종합유기견보호센터 포인핸드
유기동물 종합 커뮤니티 사설 모바일 서비스
실종 신고 등록 활성화 입양 사업에 초점

2. 서비스 개요 :memo:

3. 팀 소개 및 역할 :two_men_holding_hands:

손우진

  • 개발 환경 구축, 전반적인 프로젝트 관리, 발표

권태양

  • 공고 데이터 수집, 적재 및 모델 학습 자동화, 품종 분류기 구현

이상헌

  • 데이터 전처리, 분류 모델 평가, re id 구현, 발표 보조

홍승혜

  • 모델 학습 데이터 수집, WEB 구현, 이메일 push 기능

4. 시스템 사용 Tool

5. 프로젝트 요약

주요 기능

6. System

$ main_project
│
├─ _db
│   └─ data
│       ├─ crawling_data
│       │       └─ [개]  #  Directory for crawled images. When the image processing is complete, go [preprocessed_data] directory
│       ├─ model_data
│       │       ├─ gallery
│       │       │     └─ gallery_list # Temp directory for gallery images(option) 
│       │       ├─ query
│       │       │     └─ query_list  # Temp directory for query image
│       │       └─ working
│       │             ├─ dog_breeds.txt  # Image statistics used in training 
│       │             ├─ result.csv   # List of image softmax by breeds
│       │             ├─ to_reid.csv   # List of images that have passed Pearson correlation coefficient, region, and date filters
│       │             ├─ to_web.csv   # List of images sent to the web
│       │             └─ tmp.h5   # weights for classification model 
│       └─ preprocessed_data
│               └─ dog_class ─ image   # Preprocessed image using YOLO-v4
└─ _src
     ├─ batch
     │     ├─ send_email.py     # Check updated DB and send e-mail
     │     └─ update_data.bat   # Batch file for crawling and Preprocessing 'post' images
     ├─ data_analysis
     │     ├─ dog_image_similarity
     │     │          ├─ copy_image.py # Copy the file from the input path to the output path
     │     │          ├─ crawling_to_preprocessed.py # After preprocessing, train the model and move the file
     │     │          ├─ dog_breed_similarity_comparison.py # Image similarity comparison by pearson correlation
     │     │          ├─ extract_similar_image_path.py # Filtering images through breed classifier
     │     │          └─ predict_dog_data.py # Prediction by image
     │     └─ re_id
     │           └─ code
     │                ├─ reid_query.py   # Extract query image feature (512 vectors)
     │                ├─ reid_gallery.py  # Extract gallery image feature (512 vectors)
     │                ├─ reid_sort.py   # Calculate image similarity using cosine distance and sorting index   
     │                ├─ train.py   # To train model 
     │                └─ model.py   # Model structure for train.py
     │                
     ├─ data_analysis
     │     └─ data_collection_zooseyo.py # 'http://www.zooseyo.or.kr/zooseyo_or_kr.html' site data crawling
     ├─ data_processing
     │     └─ image_data_download.py # Code for downloading image 
     │     └─ yolo_v4
     │            └─  detect.py   # Detect dogs and cats. Crop target image and save 
     └─ web
         ├─ static
         │     └─ assets
         │           └─ img # Image and Icon for Web Design
         │     ├─ bootstrap # Bootstrap CSS, JS Files
         │     ├─ css # CSS files for Web Design
         │     ├─ js # JavaScript files for Web Design
         │     └─ images
         │           ├─ input_image
         │           └─ uploads # User Query Images
         ├─ templates
         │      ├─ find_my_dog_a.html  # Result Page
         │      └─ index.html   # Main page
         └─ app.py # Web Application (Calls Models)

7. requirement

python==3.8
pytorch==1.6.0 (window, conda, cuda==10.1) conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
absl-py==0.10.0
easydict==1.9
Flask==1.1.2
Flask-Dropzone==1.5.4
Flask-SQLAlchemy==2.4.4
matplotlib==3.2.2
numba==0.50.1
numpy==1.18.5
opencv-python==4.4.0.42
pandas==1.0.5
pillow==7.2.0
pretrainedmodels==0.7.4
pymysql==0.9.2
scipy==1.4.1
tensorflow==2.3.0 
torchvision==0.7.0 
tqdm==4.47.0
urllib3==1.25.9
werkzeug==1.0.1
yaml==0.2.5

8. 프로젝트 진행 과정