No module named 'magma.transformers'

[Tuxius]

Hi,

just download the "magma-master" file and followed the instructions (I think), but trying to run test.py I get errors. It seems there are some parts missing?

First I get: (magma) c:\Python\magma-master>python test.py Traceback (most recent call last): File "c:\Python\magma-master\test.py", line 4, in <module> from magma.language_model import get_language_model ImportError: cannot import name 'get_language_model' from 'magma.language_model' (c:\Python\magma-master\magma\language_model.py)>

looking at the code it seems like get_language_model is not used anyhow, so commented line 4 out. But after that there is a similar miss:

(magma) c:\Python\magma-master>python test.py Traceback (most recent call last): File "c:\Python\magma-master\test.py", line 25, in <module> from magma.transformers import GPTJForCausalLM ModuleNotFoundError: No module named 'magma.transformers'

And here GPTJForCausalLM is used right in the next line. Looking at transformers.py there is just nothing like GPTJForCausalLM in there at all. Seems like something is missing here completly?

Best Tuxius

[CoEich]

Hi,

test.py was an overlooked leftover which I just now deleted. Follow the setup instructions from the readme in a clean environment and try to run example_inference.py, that should work.

Best, Constantin

[Tuxius]

Hi,

thank you for the fix. Trying to run example_inference.py on my PC with a 8 GB RTX 2070 GPU I get a RuntimeError: CUDA out of memory. Having 64 GB on my AMD Ryzen 9 5900x I tried to switch from GPU to CPU. Therefore in the example_inference.py I changed line 7 from to device = 'cuda:0' to device = 'cpu'. However, this had no effect. Looking at magma.py the reason is line 40:

"cuda" if torch.cuda.is_available() else "cpu"

so it does not use the setting from example_inference.py. Changing that line to "cpu" is a dirty quick fix that works for me, I am getting magma successfully loaded

... until I get the next error:

Traceback (most recent call last):
  File "C:\Python\magma-master\example_inference.py", line 18, in <module>
    embeddings = model.preprocess_inputs(inputs)
  File "C:\Python\magma-master\magma\magma.py", line 192, in preprocess_inputs
    return self.embed(input_list)
  File "C:\Python\magma-master\magma\magma.py", line 209, in embed
    image_embeddings = self.image_prefix(x)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Python\magma-master\magma\image_prefix.py", line 83, in forward
    logits = self.enc(x)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\clip\model.py", line 143, in forward
    x = stem(x)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\clip\model.py", line 138, in stem
    x = self.relu(bn(conv(x)))
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\torch\nn\modules\conv.py", line 447, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\frank\anaconda3\envs\magma\lib\site-packages\torch\nn\modules\conv.py", line 443, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'

Seems like CPU is not yet implemented?

Best

[FalconMadhab]

Yeah same issue as Tuxius. Whenever I run example_inference.py it shows the below error. Specs of Machine: NVIDIA RTX 2080 6GB , cuda 11.1, nvidia-driver 470

Loading GPTJ language model...
[1]    145615 killed     python example_inference.py

[Mayukhdeb]

Hi @FalconMadhab, when you load magma as shown on example_inference.py, it takes up approximately 14GB of GPU memory. But as far as I see you have 6GB of GPU memory, which is not enough.

If you're loading the model on device = 'cpu', then the logs you've shown most likely means that you're running out of RAM.

In case of @Tuxius, I think the issue is with the fact that some torch operations which work on the GPU on half precision don't necessarily work on the cpu.

[Tuxius]

I did address this with the Pytorch team: https://github.com/pytorch/pytorch/issues/74625, but their response is: "No, we won't be implementing this with CPU. CPU does not support efficient computations with half datatype, and therefore half datatype support on cpu is limited and shouldn't be relied upon." :-(

I think currently only NVIDA RTX 3090 has 24 GB, but prices are prohitiv: https://pangoly.com/en/hardware/vga/gpu-memory-size/24gb - can anyone confirm this running on 24 GB GPUs?

Is there a chance to have magma not using half datatype as an option? Otherwise only people with RTX 3090 GPUs or TITANs will be able to run it, which is propably <1% of all AI enthusiasts ...

[Mayukhdeb]

@Tuxius I can personally confirm that this works on an RTX-3090 as shown on example_inference.py :slightly_smiling_face: