Closed BrownsugarZeer closed 2 years ago
Hi, thank you for telling this! I'll try to build from the main branch later, to see if it works.
Note: this fix was merged into mainstream after tag v1.9.0
was released, so the version built will be 1.10.0a0
.
Update:
There might be something wrong with Google's breakpad
:
[2146/4295] Building CXX object third_party/breakpad/CMakeFil.../breakpad.dir/src/client/linux/handler/exception_handler.cc.o
FAILED: third_party/breakpad/CMakeFiles/breakpad.dir/src/client/linux/handler/exception_handler.cc.o
/opt/rh/devtoolset-8/root/usr/bin/c++ -DHAVE_A_OUT_H -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -I/opt/pytorch/torch/build/caffe2/contrib/aten -I/opt/pytorch/torch/third_party/onnx -I/opt/pytorch/torch/build/third_party/onnx -I/opt/pytorch/torch/third_party/foxi -I/opt/pytorch/torch/build/third_party/foxi -I/opt/pytorch/torch/third_party/breakpad/src -I/opt/pytorch/torch/third_party/breakpad/src/third_party/linux/include -isystem /opt/pytorch/torch/build/third_party/gloo -isystem /opt/pytorch/torch/cmake/../third_party/gloo -isystem /opt/pytorch/torch/third_party/protobuf/src -isystem /opt/pytorch/torch/third_party/gemmlowp -isystem /opt/pytorch/torch/third_party/neon2sse -isystem /opt/pytorch/torch/third_party/XNNPACK/include -isystem /opt/pytorch/torch/cmake/../third_party/eigen -isystem /opt/python/cp39-cp39/include/python3.9 -isystem /opt/python/cp39-cp39/lib/python3.9/site-packages/numpy/core/include -isystem /opt/pytorch/torch/cmake/../third_party/pybind11/include -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -O3 -DNDEBUG -DNDEBUG -fPIC -DCAFFE2_USE_GLOO -D__NEON__ -DUSE_GCC_GET_CPUID -DTH_HAVE_THREAD -std=gnu++14 -MD -MT third_party/breakpad/CMakeFiles/breakpad.dir/src/client/linux/handler/exception_handler.cc.o -MF third_party/breakpad/CMakeFiles/breakpad.dir/src/client/linux/handler/exception_handler.cc.o.d -o third_party/breakpad/CMakeFiles/breakpad.dir/src/client/linux/handler/exception_handler.cc.o -c /opt/pytorch/torch/third_party/breakpad/src/client/linux/handler/exception_handler.cc
In file included from /opt/pytorch/torch/third_party/breakpad/src/client/linux/minidump_writer/linux_dumper.h:54,
from /opt/pytorch/torch/third_party/breakpad/src/client/linux/minidump_writer/minidump_writer.h:42,
from /opt/pytorch/torch/third_party/breakpad/src/client/linux/handler/exception_handler.h:42,
from /opt/pytorch/torch/third_party/breakpad/src/client/linux/handler/exception_handler.cc:66:
/opt/pytorch/torch/third_party/breakpad/src/client/linux/dump_writer_common/thread_info.h:69:27: error: field ‘regs’ has incomplete type ‘google_breakpad::user_regs_struct’
struct user_regs_struct regs;
^~~~
/opt/pytorch/torch/third_party/breakpad/src/client/linux/dump_writer_common/thread_info.h:69:10: note: forward declaration of ‘struct google_breakpad::user_regs_struct’
struct user_regs_struct regs;
^~~~~~~~~~~~~~~~
/opt/pytorch/torch/third_party/breakpad/src/client/linux/dump_writer_common/thread_info.h:70:29: error: field ‘fpregs’ has incomplete type ‘google_breakpad::user_fpsimd_struct’
struct user_fpsimd_struct fpregs;
^~~~~~
/opt/pytorch/torch/third_party/breakpad/src/client/linux/dump_writer_common/thread_info.h:70:10: note: forward declaration of ‘struct google_breakpad::user_fpsimd_struct’
struct user_fpsimd_struct fpregs;
^~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
I have suspended building and wait for upstream patches...
Thank you for the rapid response and testing!!!
Sadly to hear the bad news, but I will keep track of this. If there are any further information, I will let you know :D
Hi.
Still unable to solve the error, but I just realized that since breakpad
is an error reporting tool and is newly add in PyTorch, simply disabling it (export USE_BREAKPAD=0
) should be okay.
By this way I've managed to build the wheel of v1.10.0a0
:
torch-1.10.0a0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
torch-1.10.0a0-cp39-cp39-linux_aarch64.whl
Python 3.9.6 (default, Aug 26 2021, 06:38:01)
[GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import torch
>>> print(os.uname().machine)
aarch64
>>> print(torch.__version__)
1.10.0a0
>>> print(torch.fft.fft(torch.randn(3,3)))
tensor([[-0.4325+0.0000j, -0.0342-0.3125j, -0.0342+0.3125j],
[ 0.1138+0.0000j, -2.5345-0.4279j, -2.5345+0.4279j],
[ 3.2972+0.0000j, 0.4888-1.5191j, 0.4888+1.5191j]])
Hi, I had already installed and it worked!!!
But unfortunately, there was another problem happened. I'm not sure that torchaudio
should be also built as latest version or what. Since the latest ver. of pytorch
and torchaudio
are still under developing(perhaps there are more uncertain errors).
I guess this error means pytorch
does not contain the function that torchaudio
needs (version mismatch between torchaudio
and pytorch
).
Python 3.9.5 (default, May 11 2021, 08:20:37)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.fft.fft(torch.randn(3, 3))
tensor([[ 1.6230+0.0000j, -0.5391+0.9954j, -0.5391-0.9954j],
[-1.5692+0.0000j, 2.3500+0.4336j, 2.3500-0.4336j],
[ 0.4668+0.0000j, -1.4437+1.2189j, -1.4437-1.2189j]])
>>>
>>> import torchaudio
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/sugar/test/test/lib/python3.9/site-packages/torchaudio/__init__.py", line 1, in <module>
from . import extension # noqa: F401
File "/home/sugar/test/test/lib/python3.9/site-packages/torchaudio/extension/__init__.py", line 5, in <module>
_init_extension()
File "/home/sugar/test/test/lib/python3.9/site-packages/torchaudio/extension/extension.py", line 11, in _init_extension
_init_script_module(ext)
File "/home/sugar/test/test/lib/python3.9/site-packages/torchaudio/extension/extension.py", line 19, in _init_script_module
torch.classes.load_library(path)
File "/home/sugar/test/test/lib/python3.9/site-packages/torch/_classes.py", line 48, in load_library
torch.ops.load_library(path)
File "/home/sugar/test/test/lib/python3.9/site-packages/torch/_ops.py", line 110, in load_library
ctypes.CDLL(path)
File "/usr/lib/python3.9/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/sugar/test/test/lib/python3.9/site-packages/torchaudio/_torchaudio.so: undefined symbol: _ZNK2at6Tensor9index_putERKN3c104ListINS1_8optionalIS0_EEEERKS0_b
>>>
It's my personal project and I don't want to bother you too much. But if you have any ideas, please let me know.
Thanks again, sincerely.
Hi, just as you expected, all secondary packages (torchaudio, vision, text, etc.) require a specific PyTorch version.
In your situation torchaudio
is built on top of PyTorch v1.9.0, so it won't work on v1.10.0 unless recompiled.
Hi, I have a great news!!!
After installed some dependencies and ninja, I ran the script in pytorch-aarch64/build/manylinux/audio.sh
with a following command. The the wheel was built successfully.
bash audio.sh 0.10.0a0 master
(Although faced some problems, and finally realized that I needed a RPi4 with 8G RAM. maybe you can mention this on the README)
In short, here is the result and works well !!
code:
import os
import torch
import torchaudio
from speechbrain.pretrained import EncoderClassifier
classifier = EncoderClassifier.from_hparams(
source="/home/sugar/Desktop/vscode_workspace/ASR/repos/GSC10/pretrained_model/best_model",
hparams_file='xvector_interface.yaml',
savedir="/home/sugar/Desktop/vscode_workspace/ASR/repos/GSC10/pretrained_model/best_model"
)
print(os.uname().machine)
print(torch.__version__)
print(torchaudio.__version__)
# Perform classification
audio_file = '/home/sugar/Desktop/vscode_workspace/ASR/repos/GSC10/pretrained_model/022cd682_nohash_0.wav'
signal, fs = torchaudio.load(audio_file) # id: 022cd682, lb: yes
output_probs, score, index, text_lab = classifier.classify_batch(signal)
print('Target: 022cd682_yes, Predicted: ' + text_lab[0])
result
aarch64
1.10.0a0
0.10.0a0
Target: 022cd682_yes, Predicted: yes
Hi, great to hear that!
(Although faced some problems, and finally realized that I needed a RPi4 with 8G RAM. maybe you can mention this on the README)
You may try using zram
to extend your RAM.
# sudo
modprobe zram num_devices=4
sleep 5
zramctl --find --size 2G
mkswap /dev/zram0
swapon -p 5 /dev/zram0
sleep 1
zramctl --find --size 2G
mkswap /dev/zram1
swapon -p 5 /dev/zram1
sleep 1
zramctl --find --size 2G
mkswap /dev/zram2
swapon -p 5 /dev/zram2
sleep 1
zramctl --find --size 2G
mkswap /dev/zram3
swapon -p 5 /dev/zram3
Hello, I've met the question as you too when use stft in torch 1.7.1 。But compile failed using torch-1.10.0a0, My gcc version is 7.3.0 。
I try to compile libtorch many times in my own machine but failed at last ! Can you provide a solved version of torch1.7.1?
Thanks a lot
Hi @hk0901 ,
Can you provide a solved version of torch1.7.1?
I'm afraid I can't, because MKL support for aarch64 will be added since 1.10, patching older versions is likely not feasible.
But compile failed using torch-1.10.0a0, My gcc version is 7.3.0
Although GCC 5 should be able to support C++14 which is needed by PyTorch, you might need gcc 8 or newer based on my experience. It might be okay to use a newer gcc version.
你好, torch-1.10.0a0-cp39-cp39-linux_aarch64.whl 的运行结果好像不正确, 我在x64平台的master版本,arm的1.9.0的版本,得到的结果都是相同,但是你的这个whl模型输出结果和stft的结果都和他们不一样, 请问是哪里有问题吗
另外,我自己编译总是报错,所以希望能用您提供的arm版本最新的pytorch
但是你的这个whl模型输出结果和stft的结果都和他们不一样
我没有用过 stft
算法,不甚清楚具体原因,或许可以提供下实例代码和运行结果
自己编译总是报错
armv8.2-a+dotprod
要求 GCC 版本在 8 以上
First
Thank you for building these lifesavers :D\ I found that someone posted a same issue#6, and also noticed that fixed in #60976 with third-party then merged into lastest pytorch version (here) few days ago.
For now, I still faced the same problem about MKL(the following for more details). The first thing I tried to do was building from source, but it was my first time to build and everything was so strange to me :(\ Could you help me to check whether works on RPi4?\ Thanks a lot!!!
Bug
fft: ATen not compiled with MKL support
\ RuntimeError raised when trying to compute STFT on RPi4.To Reproduce
My code
Error trace:
Simple sample:
Error trace:
Expected behavior
Everything run successfully.
Environment
PyTorch version: 1.9.0\ OS: Ubuntu 21.04 64bit\ Python version: 3.9.5