CCExtractor / Rekognition

Free and Open Source alternative to Amazon's Rekognition service. CCExtractor Development | Poor Man's Rekognition
GNU General Public License v3.0
98 stars 52 forks source link

Image captioning #222

Closed augsaksham closed 2 years ago

augsaksham commented 2 years ago

Added Image Captioning the the code base as per issue : #181 Two models are added to the code base the facilitate the captioning process . The caption feature can be used via a request at localhost/8000/caption/

The captioning module can predict using two algorithms ### { Greedy or Beam Search } ,this can be changed using the method parameter.

The method call can be given a single argument ('file' - > the image ) or two arguments ('file' - > the image , 'method' -> 'greedy'/'beam' ).

By default greedy method is used if not specified in arguments.

augsaksham commented 2 years ago

Note: For downloading the latest model and test files use the below lines:

mkdir tests/testdata cd tests/testdata wget https://www.dropbox.com/s/1bnxg32zvgjv0pl/compareImage.jpeg wget https://www.dropbox.com/s/1bnxg32zvgjv0pl/compareImage.jpeg wget https://www.dropbox.com/s/x3qpga9gc4ifamn/t1.png wget https://www.dropbox.com/s/l5t09lp8u4ok593/t2.jpeg wget https://www.dropbox.com/s/hzlpo74tk0xwzzh/t3.jpeg wget https://www.dropbox.com/s/lni50cgunua5mij/test1.mp4 wget https://www.dropbox.com/s/wm3llx0ydbnq8mn/test2.mp4 wget https://www.dropbox.com/s/ato4fie6k3lmctu/test3.mp4 wget https://www.dropbox.com/s/ifd7254x29oxjze/test4.mp4 wget https://www.dropbox.com/s/iwtgwz24eipd629/obj1.mp4 wget https://www.dropbox.com/s/ull2tqlou1p8l16/obj2.mp4 wget https://www.dropbox.com/s/3w5ghr5jj6opr58/scene1.mp4 wget https://www.dropbox.com/s/ij5hj4hznczvfcw/text.mp4 wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1B-goSqkAqyq2dssvvpNy8vRhfxaZEMf5' -O caption1.jpg wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1B-goSqkAqyq2dssvvpNy8vRhfxaZEMf5' -O caption2.jpg cd ../.. cd media mkdir object mkdir nsfw cd .. cd corelib/model mkdir facenet cd facenet wget https://www.dropbox.com/s/jm8grrifh5yk7is/2017.zip?dl=1 -O 2017.zip unzip 2017.zip rm 2017.zip cd .. mkdir tfs cd tfs wget --load-cookies /tmp/cookies.txt "https://drive.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.google.com/uc?export=download&id=1Yix5evLAqHOoZ_dLdFtayduxZwsf_BFh' -O- | sed -rn 's/.confirm=([0-9A-Za-z_]+)./\1\n/p')&id=1Yix5evLAqHOoZ_dLdFtayduxZwsf_BFh" -O module.zip && rm -rf /tmp/cookies.txt unzip module.zip rm module.zip cd ../../.. cd data mkdir text_reco cd text_reco wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1zmxO5NoEqTdYRGHXalaK7bn2m27egQMN' -O char_dict_en.json wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1XGO2ycQcBYtwxT_KanapuoR6LeGR4s_E' -O ord_map_en.json cd ../..

PulkitMishra commented 2 years ago

@joeyouss incorporate these change in frontend and contributing guide