imSanko / Image_Caption_Generator_With_Transformers

This repository contains code for generating captions for images using a Transformer-based model. The model used is the `VisionEncoderDecoderModel` from the Hugging Face Transformers library, specifically the `nlpconnect/vit-gpt2-image-captioning` model.
https://imsanko.github.io/Image_Caption_Generator_With_Transformers/
MIT License
9 stars 1 forks source link

ERROR: Oh no. Error running app #8

Open imSanko opened 4 months ago

imSanko commented 4 months ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Go to '.DEPLOYMNET'
  2. Click on https://sap-photo-cap.streamlit.app
  3. wait and upload any img
  4. See error

[07:17:36] 🐍 Python dependencies were installed from /mount/src/image_caption_generator_with_transformers/requirements.txt using uv. Check if streamlit is installed Streamlit is already installed [07:17:38] πŸ“¦ Processed dependencies! /home/adminuser/venv/lib/python3.11/site-packages/transformers/models/vit/feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead. warnings.warn( /app/scripts/run-streamlit.sh: line 9: 197 Killed sudo -E -u appuser /home/adminuser/venv/bin/streamlit "$@" [07:21:00] ❗️ Streamlit server consistently failed status checks [07:21:00] ❗️ Please fix the errors, push an update to the git repo, or reboot the app.



Refer to #2 issue after deployment issue !! 

**Expected behavior**
Doesn't work after deployment.

**Screenshots**
<img width="1326" alt="Screenshot 2024-04-28 at 12 54 04" src="https://github.com/imSanko/Image_Caption_Generator_With_Transformers/assets/69459134/2f081683-026f-438d-8bd6-4821a33ddfda">

**Desktop (please complete the following information):**
 - OS: [ MacOS Sonoma
 - Browser safari Latest

**Additional context**
Help me to fix this !
imSanko commented 4 months ago

Hopefully it fixes

from transformers import VisionEncoderDecoderModel, ViTImageProcessor, AutoTokenizer
import torch
from PIL import Image

model = VisionEncoderDecoderModel.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
image_processor = ViTImageProcessor.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
tokenizer = AutoTokenizer.from_pretrained("nlpconnect/vit-gpt2-image-captioning")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

max_length = 16
num_beams = 4
gen_kwargs = {"max_length": max_length, "num_beams": num_beams}

def predict_step(image_paths):
    images = []
    for image_path in image_paths:
        i_image = Image.open(image_path)
        if i_image.mode != "RGB":
            i_image = i_image.convert(mode="RGB")

        images.append(i_image)

    pixel_values = image_processor(images=images, return_tensors="pt").pixel_values
    pixel_values = pixel_values.to(device)

    output_ids = model.generate(pixel_values, **gen_kwargs)

    preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
    preds = [pred.strip() for pred in preds]
    return preds

The main changes are:

  1. Instead of importing ViTFeatureExtractor, you now import ViTImageProcessor.
  2. You instantiate ViTImageProcessor instead of ViTFeatureExtractor.
  3. When encoding the images, you use image_processor(images=images, return_tensors="pt").pixel_values instead of feature_extractor(images=images, return_tensors="pt").pixel_values.

The rest of the code remains the same.

By making these changes, you should no longer encounter the deprecation warning, and your code should work with the latest version of the Transformers library.

Regarding the error message you provided, it seems related to running the Streamlit app with sudo and the appuser user. This error is likely caused by a permission issue or a misconfiguration in your deployment environment. Without more context about your deployment setup, it's difficult to provide a specific solution for this error.

imSanko commented 4 months ago

Working > https://sap-photo-cap.streamlit.app

check @chandan1995 with the new code and try to RUN > DEPLOY AND UPLOAD > THEN CHECK ITS WORKING OR NOT !!

imSanko commented 4 months ago

Still not working check with the new code @chandan1995 and let's see me and create a PR and rectify it, so that we can connect the app from anywhere using the, Link > https://sap-photo-cap.streamlit.app/

Screenshot_20240428-132709__01.jpg

chandan1995 commented 4 months ago

@imSanko please check the new pull request, have modified code to make it working uses ViTImageProcessor https://branchtransformersfix.streamlit.app/

image

imSanko commented 4 months ago

Issue hasn't been solved yet @chandan1995

![Uploading Screenshot 2024-04-29 at 19.43.56.png…]()

[14:07:57] 🐍 Python dependencies were installed from /mount/src/image_caption_generator_with_transformers/requirements.txt using uv. Check if streamlit is installed Streamlit is already installed [14:07:59] πŸ“¦ Processed dependencies! /home/adminuser/venv/lib/python3.11/site-packages/transformers/models/vit/feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead. warnings.warn( We strongly recommend passing in anattention_masksince your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. You may ignore this warning if yourpad_token_id(50256) is identical to thebos_token_id(50256),eos_token_id(50256), or thesep_token_id(None), and your input is not padded. /app/scripts/run-streamlit.sh: line 9: 205 Killed sudo -E -u appuser /home/adminuser/venv/bin/streamlit "$@"

With a new linkhttps://san-img-cap.streamlit.app

chandan1995 commented 4 months ago

Hello @imSanko , after deploying - one time app works. later throws error saying process killed. Streamlit community cloud offers only 1G RAM which we may be exceeding and could possibly killing process. There is no way to monitor resource on cloud, I tried same code on local and didnt face any issue

imSanko commented 4 months ago

@chandan1995 Yes, Definitely now we can to a conclusion here that is the project should be done locally. How about making it for MongoDB or Vercel APP ??

chandan1995 commented 4 months ago

I'm not an expert on Mongodb (also worried about resource consumption on deployment) , my suggestion is we could keep it as is, so beginner with Python knowledge could explore it.

imSanko commented 4 months ago

@chandan1995 Nvm, thanks for the contributing here <3