Closed dnhkng closed 2 years ago
@dnhkng Sorry for delay. Did you solve this problem?
I gave up, as I found that the model was trained on TPU (brainfloat16), which is not supported on the RTX Titan.
Inferencing would just lead to garbage output, due to precision differences.
That sounds bad. I am sorry for not being able to help.
getting same error Bus error (core dumped)
/usr/local/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 16 leaked semaphores to clean up at shutdown len(cache))
Hello @tahercoolguy Did you able to figure out the issue? I am also getting same exception while running it in docker.
@hyunwoongko any pointer to fix this issue?
The model: google/ul2
The Hardware: 2x RTX Titan AMD Ryzen 9 5900X 12-Core Processor 64Gb RAM
The Environment: Python 3.9.13 Pytorch 1.12.0+cu102 NVIDIA-SMI 495.29.05 Driver Version: 495.29.05 CUDA Version: 11.5
Code used:
Error Message:
Is this something I can fix? I would love to use this large model, as it's near SOTA on everything :)