something error happens when I update the version of triton

[laliwang]

Wonderful works! While I have installed the flash-attn wheel, which version is 2.3.5 with cuda11.7, torch2.0.1, to successfully import flash_attn as you mentioned in issue2, I update the version of triton to 2.1.0. However, something goes wrong when I run the app_gradio.py in your folder, the details are shown as follows:

(tapmap) wjh@ps:/data/wjh/LVLM_pkgs/tokenize-anything$ CUDA_VISIBLE_DEVICES=6 python3 scripts/app_gradio.py --model-type tap_vit_b --checkpoint ./weights/tap_vit_b_b45cbf.pkl --concept ./weights/merged_2560.pkl
Running on local URL:  http://127.0.0.1:2030

To create a public link, set `share=True` in `launch()`.
ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/lib64/libstdc++.so' from /etc/ld.so.preload cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/lib64/libstdc++.so' from /etc/ld.so.preload cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so: error loading plugin: /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so: wrong ELF class: ELFCLASS64
collect2: error: ld returned 1 exit status
Process Process-2:
Traceback (most recent call last):
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/engine/test_engine.py", line 81, in run
    self.send_results(predictor, indices, examples)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/engine/test_engine.py", line 47, in send_results
    results = predictor.get_results(examples)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/data/wjh/LVLM_pkgs/tokenize-anything/scripts/app_gradio.py", line 97, in get_results
    captions = self.model.generate_text(sem_tokens).reshape(batch_shape)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/modeling/image_tokenizer.py", line 188, in generate_text
    logits = self.text_decoder.transformer(prompts, x, prev_pos)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/modeling/text_decoder.py", line 160, in forward
    x = blk(x)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/modeling/text_decoder.py", line 134, in forward
    x = self.dropout(self.attn(self.norm1(x))).add_(x)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/modeling/text_decoder.py", line 104, in forward
    q, k = self.cache.forward_rotary(q, k, inplace=True)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/tokenize_anything/modeling/text_decoder.py", line 73, in forward_rotary
    q = apply_rotary_emb(q, cos, sin, interleaved=True, inplace=inplace)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/flash_attn/layers/rotary.py", line 122, in apply_rotary_emb
    return ApplyRotaryEmb.apply(
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/flash_attn/layers/rotary.py", line 48, in forward
    out = apply_rotary(
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/flash_attn/ops/triton/rotary.py", line 213, in apply_rotary
    rotary_kernel[grid](
  File "<string>", line 63, in rotary_kernel
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/triton/compiler/compiler.py", line 425, in compile
    so_path = make_stub(name, signature, constants)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/triton/compiler/make_launcher.py", line 39, in make_stub
    so = _build(name, src_path, tmpdir)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/triton/common/build.py", line 90, in _build
    ret = subprocess.check_call(cc_cmd)
  File "/home/wjh/anaconda3/envs/tapmap/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmpmsxq3wqb/main.c', '-O3', '-I/home/wjh/anaconda3/envs/tapmap/lib/python3.10/site-packages/triton/common/../third_party/cuda/include', '-I/home/wjh/anaconda3/envs/tapmap/include/python3.10', '-I/tmp/tmpmsxq3wqb', '-shared', '-fPIC', '-lcuda', '-o', '/tmp/tmpmsxq3wqb/rotary_kernel.cpython-310-x86_64-linux-gnu.so', '-L/usr/lib/x86_64-linux-gnu', '-L/usr/lib/x86_64-linux-gnu']' returned non-zero exit status 1.

[PhyscalX]

Hi, @laliwang

It seems that your GCC is broken. Check the environment variable LD_PRELOAD, the default value should be empty.

[laliwang]

Thanks for your quick reply! When I check the value of LD_PRELOAD on my desktop, it outputs: /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so just as the log indicates. So does this mean I need to reset or disable it?

[PhyscalX]

Try to unset this variable.