Installation Issues with Demo Notebook

[raviy0807]

While attempting to set up and run the demo notebook from the repository, I encountered multiple issues related to environment setup, package dependencies, and code configurations that significantly hindered progress. This issue aims to document these problems and propose solutions to aid in troubleshooting and improving the setup process for all users.

Environment Python Version: 3.9 Operating System: ubuntu 20

Issues and Proposed Resolutions

• Unclear Python Version Compatibility

• Dependency Conflicts in YAML Problem: Conflicting dependencies are present in the provided YAML file. Proposed Solution: Update the YAML file to ensure all dependencies are compatible.

• Unavailable Package in Requirements Problem: The package petrel_oss_sdk==v2.2.1_2_g1505ef3_master listed in the requirements file cannot be found.

Demo Notebook Issues

• for relative import add following code in the code ( add ur folder path) import sys sys.path.append('path_of_your_repo_multi_modality_folder')
• Import Errors in Demo Notebook Problem: Issues with importing .easydict due to ambiguity whether it's a package or a module import. Proposed Solution: Change the import statement in the notebook to from utils.easydict import EasyDict.
• Directory Naming Consistency Problem: The directory named utils is sometimes referred to as util, causing import errors. Proposed Solution: Standardize the directory name to utils across all references and documentation.
• Hardware-Specific Configuration for Flash Attention Problem: Flash Attention error resolution only applicable for NVIDIA A100 GPUs and above, unclear implementation. Proposed Solution: Implement a conditional import based on the use_fused_rmsnorm flag to handle different hardware configurations: python

  if use_fused_rmsnorm:
  from flash_attn.ops.rms_norm import DropoutAddRMSNorm

• Missing peft Package Problem: The peft package is not included in the initial setup but is required. Proposed Solution: Add pip install peft to the installation instructions.
• BERT Tokenizer Path Issue Problem: BERT tokenizer is set to use a local path that is undefined. Proposed Solution: Specify the correct path to the tokenizer files or update the setting to not require use_local.
• Configuration File Path Error Problem: Path error for config_bert_large.json leading to 'No such file or directory'. Proposed Solution: Ensure that config['TextEncoders']['bert_large']['config'] uses an absolute path to the configuration file.

[shepnerd]

Thank you for your feedback. We will address your mentioned issues soon.

[chinmayad]

Getting this error while installing requirements for Internvideo2 multimodality. ERROR: Could not find a version that satisfies the requirement petrel_oss_sdk==v2.2.1_2_g1505ef3_master (from versions: none) ERROR: No matching distribution found for petrel_oss_sdk==v2.2.1_2_g1505ef3_master

[belamarib]

I'm having the same issues with the relative imports regarding the following lines:

from config import (Config,
                    eval_dict_leaf)

from utils import (retrieve_text,
                  _frame_from_video,
                  setup_internvideo2)

I've tried the sys.path.append code, adding an init.py file and other strategies and nothing worked

[bdager]

Hi there, I'm still not able to solve the BERT tokenizer path error. I'm trying to run the demo.py and currently I'm getting this error:

raise EnvironmentError(
OSError: Can't load tokenizer for 'bert-large-uncased'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'bert-large-uncased' is the correct path to a directory containing all relevant files for a BertTokenizer tokenizer.

I'm providing a local path to the model: InternVideo2-Stage2_1B-224p-f4.pt (), in the script: multi_modality/demo/internvideo2_stage2_config.py

So, I am not sure if I am providing the correct model or it is another problem what is causing the above error.

Any help or idea would be very valuable and appreciate it :) Thanks you so much in advance