Dataset and Code for our ACL 2024 paper: "Multimodal Table Understanding". We propose the first large-scale Multimodal IFT and Pre-Train Dataset for table understanding and develop a generalist tabular MLLM named Table-LLaVA.
按照 readme的方式准备的环境,进行inference 报错:
Traceback (most recent call last):
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 175, in convert_to_tensors
tensor = as_tensor(value)
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 149, in as_tensor
return torch.tensor(value)
RuntimeError: Could not infer dtype of numpy.float32
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/mnt/localdisk/tanm/nltk_data/backup_llm/Table-LLaVA/llava/eval/model_vqa.py", line 112, in
eval_model(args)
File "/mnt/localdisk/tanm/nltk_data/backup_llm/Table-LLaVA/llava/eval/model_vqa.py", line 59, in eval_model
image_tensor = image_processor.preprocess(image, return_tensors='pt',padding=True)['pixel_values'][0]
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/models/clip/image_processing_clip.py", line 337, in preprocess
return BatchFeature(data=data, tensor_type=return_tensors)
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 78, in init
self.convert_to_tensors(tensor_type=tensor_type)
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 181, in convert_to_tensors
raise ValueError(
ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.
按照 readme的方式准备的环境,进行inference 报错: Traceback (most recent call last): File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 175, in convert_to_tensors tensor = as_tensor(value) File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 149, in as_tensor return torch.tensor(value) RuntimeError: Could not infer dtype of numpy.float32 During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/mnt/localdisk/tanm/nltk_data/backup_llm/Table-LLaVA/llava/eval/model_vqa.py", line 112, in
eval_model(args)
File "/mnt/localdisk/tanm/nltk_data/backup_llm/Table-LLaVA/llava/eval/model_vqa.py", line 59, in eval_model
image_tensor = image_processor.preprocess(image, return_tensors='pt',padding=True)['pixel_values'][0]
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/models/clip/image_processing_clip.py", line 337, in preprocess
return BatchFeature(data=data, tensor_type=return_tensors)
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 78, in init
self.convert_to_tensors(tensor_type=tensor_type)
File "/mnt/localdisk/tanm/miniconda/envs/table_llava/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 181, in convert_to_tensors
raise ValueError(
ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.