RuntimeError : Sizes of tensors must match except in dimension . Excepted size 244 but got size 243 for tensor number 1 in the list.

[BOUTAYEBAnasse]

Hi, first of all I'd like to thank you for this very interesting project. I've tried to run your code under google colab by adding the whole project and adding the following elements to the checkpoints folder (HRNET_LoveDA_best.pth, last_swint_upernet_finetune.pth, Res34_AID_best.pth, yolov5_best.pt) when I tried to run the following command : !python /drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py --openai_key my_open_ai_key --image_dir /drive/MyDrive/Remote-Sensing-ChatGPT-main/test_image.png I got the following error :

/usr/local/lib/python3.10/dist-packages/langchain/chat_models/init.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.chat_models import ChatOpenAI.

To install langchain-community run pip install -U langchain-community. warnings.warn( Initializing RSChatGPT, load_dict={'ImageCaptioning': 'cuda:0', 'SceneClassification': 'cuda:0', 'ObjectDetection': 'cuda:0', 'LandUseSegmentation': 'cuda:0', 'InstanceSegmentation': 'cuda:0', 'ObjectCounting': 'cuda:0', 'EdgeDetection': 'cpu'} Initializing ImageCaptioning to cuda:0 2024-02-05 21:51:21.669993: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-02-05 21:51:21.670040: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-02-05 21:51:21.671336: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-02-05 21:51:22.692478: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Initializing SceneClassification Initializing SceneClassification /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None. warnings.warn(msg) Initializing LandUseSegmentation Initializing InstanceSegmentation Initializing InstanceSegmentation /usr/local/lib/python3.10/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] Initializing Edge Detection Function.... Initializing Image2Canny All the Available Functions: {'ImageCaptioning': <main.ImageCaptioning object at 0x7af43ee69b10>, 'SceneClassification': <main.SceneClassification object at 0x7af43ee699f0>, 'ObjectDetection': <main.ObjectDetection object at 0x7af430e7b340>, 'LandUseSegmentation': <main.LandUseSegmentation object at 0x7af430e7b250>, 'InstanceSegmentation': <main.InstanceSegmentation object at 0x7af5421f32b0>, 'ObjectCounting': <main.ObjectCounting object at 0x7af3eedcc0d0>, 'EdgeDetection': <main.EdgeDetection object at 0x7af3eed94070>} /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class langchain_community.chat_models.openai.ChatOpenAI was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run pip install -U langchain-openai and import as from langchain_openai import ChatOpenAI. warn_deprecated( /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function initialize_agent was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use Use new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc. instead. warn_deprecated( RSChatGPT initialization done, you can now chat with RSChatGPT~ /usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1273: UserWarning: Using the model-agnostic default max_length (=20) to control the generation length. We recommend setting max_new_tokens to control the maximum length of the generation. warnings.warn(

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed run_image, Input image: image/d3bc52dc.png Current state: [('image/d3bc52dc.png', 'Received.')] Current Memory: Human: Provide a remote sensing image named image/d3bc52dc.png. The description is: A satellite image of a large airport with many planes parked on the tar. This information helps you to understand this image, but you should use tools to finish following tasks, rather than directly imagine from my description. If you understand, say "Received". AI: Received. /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function __call__ was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead. warn_deprecated(

Entering new AgentExecutor chain... HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Count object Action Input: image/d3bc52dc.png, plane Processed Object Counting, Input Image: image/d3bc52dc.png, Output text: plane is not a supported category for the model.

Observation: plane is not a supported category for the model. Thought:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Get Photo Description Action Input: image/d3bc52dc.png/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1273: UserWarning: Using the model-agnostic default max_length (=20) to control the generation length. We recommend setting max_new_tokens to control the maximum length of the generation. warnings.warn(

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Observation: A satellite image of a large airport with many planes parked on the tar Thought:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Detect the given object Action Input: image/d3bc52dc.png, planeTraceback (most recent call last): File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 239, in state=bot.run_image(args.image_dir, [], txt) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 218, in run_image state=self.run_text(f'{txt} {image_filename} ', state) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 184, in run_text res = self.agent({"input": text.strip()}) File "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py", line 145, in warning_emitting_wrapper return wrapped(*args, kwargs) File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 363, in call return self.invoke( File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 162, in invoke raise e File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 156, in invoke self._call(inputs, run_manager=run_manager) File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1391, in _call next_step_output = self._take_next_step( File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1097, in _take_next_step [ File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1097, in [ File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1182, in _iter_next_step yield self._perform_agent_action( File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1204, in _perform_agent_action observation = tool.run( File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 373, in run raise e File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 345, in run self._run(*tool_args, run_manager=run_manager, *tool_kwargs) File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 523, in _run else self.func(args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 124, in inference log_text=self.func.inference(image_path, det_prompt,updated_imagepath) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/YOLOv5.py", line 25, in inference out, = self.model(image.to(self.device), augment=False,val=True) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/common.py", line 307, in forward y = self.model(im, augment=augment, visualize=visualize)[0] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/yolo.py", line 135, in forward return self._forward_once(x, profile, visualize) # single-scale inference, train File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/yolo.py", line 158, in _forward_once x = m(x) # run File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/common.py", line 276, in forward return torch.cat(x, self.d) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 244 but got size 243 for tensor number 1 in the list.

[ucasligang]

Hi, first of all I'd like to thank you for this very interesting project. I've tried to run your code under google colab by adding the whole project and adding the following elements to the checkpoints folder (HRNET_LoveDA_best.pth, last_swint_upernet_finetune.pth, Res34_AID_best.pth, yolov5_best.pt) when I tried to run the following command : !python /drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py --openai_key my_open_ai_key --image_dir /drive/MyDrive/Remote-Sensing-ChatGPT-main/test_image.png I got the following error :

/usr/local/lib/python3.10/dist-packages/langchain/chat_models/init.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.chat_models import ChatOpenAI.

To install langchain-community run pip install -U langchain-community. warnings.warn( Initializing RSChatGPT, load_dict={'ImageCaptioning': 'cuda:0', 'SceneClassification': 'cuda:0', 'ObjectDetection': 'cuda:0', 'LandUseSegmentation': 'cuda:0', 'InstanceSegmentation': 'cuda:0', 'ObjectCounting': 'cuda:0', 'EdgeDetection': 'cpu'} Initializing ImageCaptioning to cuda:0 2024-02-05 21:51:21.669993: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-02-05 21:51:21.670040: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-02-05 21:51:21.671336: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-02-05 21:51:22.692478: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Initializing SceneClassification Initializing SceneClassification /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None. warnings.warn(msg) Initializing LandUseSegmentation Initializing InstanceSegmentation Initializing InstanceSegmentation /usr/local/lib/python3.10/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.) return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined] Initializing Edge Detection Function.... Initializing Image2Canny All the Available Functions: {'ImageCaptioning': <main.ImageCaptioning object at 0x7af43ee69b10>, 'SceneClassification': <main.SceneClassification object at 0x7af43ee699f0>, 'ObjectDetection': <main.ObjectDetection object at 0x7af430e7b340>, 'LandUseSegmentation': <main.LandUseSegmentation object at 0x7af430e7b250>, 'InstanceSegmentation': <main.InstanceSegmentation object at 0x7af5421f32b0>, 'ObjectCounting': <main.ObjectCounting object at 0x7af3eedcc0d0>, 'EdgeDetection': <main**.EdgeDetection object at 0x7af3eed94070>} /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class langchain_community.chat_models.openai.ChatOpenAI was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run pip install -U langchain-openai and import as from langchain_openai import ChatOpenAI. warn_deprecated( /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function initialize_agent was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use Use new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc. instead. warn_deprecated( RSChatGPT initialization done, you can now chat with RSChatGPT~ /usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1273: UserWarning: Using the model-agnostic default max_length (=20) to control the generation length. We recommend setting max_new_tokens to control the maximum length of the generation. warnings.warn(

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed run_image, Input image: image/d3bc52dc.png Current state: [('image/d3bc52dc.png', 'Received.')] Current Memory: Human: Provide a remote sensing image named image/d3bc52dc.png. The description is: A satellite image of a large airport with many planes parked on the tar. This information helps you to understand this image, but you should use tools to finish following tasks, rather than directly imagine from my description. If you understand, say "Received". AI: Received. /usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function __call__ was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead. warn_deprecated(

Entering new AgentExecutor chain... HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Count object Action Input: image/d3bc52dc.png, plane Processed Object Counting, Input Image: image/d3bc52dc.png, Output text: plane is not a supported category for the model.

Observation: plane is not a supported category for the model. Thought:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Get Photo Description Action Input: image/d3bc52dc.png/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1273: UserWarning: Using the model-agnostic default max_length (=20) to control the generation length. We recommend setting max_new_tokens to control the maximum length of the generation. warnings.warn(

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Processed ImageCaptioning, Input Image: image/d3bc52dc.png, Output Text: A satellite image of a large airport with many planes parked on the tar

Observation: A satellite image of a large airport with many planes parked on the tar Thought:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" Thought: Do I need to use a tool? Yes Action: Detect the given object Action Input: image/d3bc52dc.png, planeTraceback (most recent call last): File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 239, in state=bot.run_image(args.image_dir, [], txt) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 218, in run_image state=self.run_text(f'{txt} {image_filename} ', state) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 184, in run_text res = self.agent({"input": text.strip()}) File "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py", line 145, in warning_emitting_wrapper return wrapped(*args, kwargs) File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 363, in call return self.invoke( File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 162, in invoke raise e File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 156, in invoke self._call(inputs, run_manager=run_manager) File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1391, in _call next_step_output = self._take_next_step( File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1097, in _take_next_step [ File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1097, in [ File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1182, in _iter_next_step yield self._perform_agent_action( File "/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py", line 1204, in _perform_agent_action observation = tool.run( File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 373, in run raise e File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 345, in run self._run(*tool_args, run_manager=run_manager, *tool_kwargs) File "/usr/local/lib/python3.10/dist-packages/langchain_core/tools.py", line 523, in _run else self.func(args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RSChatGPT-shell.py", line 124, in inference log_text=self.func.inference(image_path, det_prompt,updated_imagepath) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/YOLOv5.py", line 25, in inference out, = self.model(image.to(self.device), augment=False,val=True) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/common.py", line 307, in forward y = self.model(im, augment=augment, visualize=visualize)[0] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/yolo.py", line 135, in forward return self._forward_once(x, profile, visualize) # single-scale inference, train File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/yolo.py", line 158, in _forward_once x = m(x) # run File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/drive/MyDrive/Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/common.py", line 276, in forward return torch.cat(x, self.d) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 244 but got size 243 for tensor number 1 in the list.

I have the same question. “Remote-Sensing-ChatGPT-main/RStask/ObjectDetection/models/common.py", line 279, in forward return torch.cat(x, self.d) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 76 but got size 75 for tensor number 1 in the list.” Have you solved it so far？