qinyang39
commented
3 years ago Hello, I am trying to apply FAD but facing the Fatal Python error: Segmentation fault here. Currently I am using tf 2.5.0 and apache-beam 2.24.0 with python 3.7.
I am stuck on the step > Compute embeddings and eastimate multivariate Gaussians
python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_statsI found the problem Fatal Python error happened in
def create_pipeline()ofcreate_embeddings_beam.py:if files_input_list: examples = ( pipeline | 'Read File List' >> ReadFromText(files_input_list) | 'Read Files' >> beam.ParDo(ReadWavFiles()))
Cause I am really not an expert using apache so couldn't figure it out what's going on.
Also by googling it, it seems something wrong with stack memory which may cause the Segmentation fault problem, so I have tried to increase the stack that my operating system allocates for the python process ended by
ulimit -s 262140. But it's still not working.I will be appreciate if someone has any ideas. Thank you!!
I attached my log here:
(fad-env) kp1@nipa2021-1226:~/google-research$ python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_stats 2021-07-26 15:34:57.534827: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-07-26 15:34:57.534877: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. I0726 15:34:58.903872 140039776012032 pipeline.py:190] Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. Fatal Python error: Segmentation fault Current thread 0x00007f5d8d1a8700 (most recent call first): File "/usr/local/lib/python3.7/bz2.py", line 346 in decompress File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 274 in loads File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 787 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1266 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/iobase.py", line 1544 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 228 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 198 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 664 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pvalue.py", line 141 in or File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/iobase.py", line 892 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 228 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 198 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 664 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 561 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/textio.py", line 564 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 228 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 198 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 664 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 621 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 561 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 997 in ror File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_beam.py", line 359 in create_pipeline File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 69 in main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 258 in _run_main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 312 in run File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 75 in
File "/usr/local/lib/python3.7/runpy.py", line 85 in _run_code File "/usr/local/lib/python3.7/runpy.py", line 193 in _run_module_as_main Segmentation fault (core dumped)
RussellSB
Hello, I am trying to apply FAD but facing the Fatal Python error: Segmentation fault here. Currently I am using tf 2.5.0 and apache-beam 2.24.0 with python 3.7.
I am stuck on the step > Compute embeddings and eastimate multivariate Gaussians
python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_statsI found the problem Fatal Python error happened in
def create_pipeline()ofcreate_embeddings_beam.py:Cause I am really not an expert using apache so couldn't figure it out what's going on.
Also by googling it, it seems something wrong with stack memory which may cause the Segmentation fault problem, so I have tried to increase the stack that my operating system allocates for the python process ended by
ulimit -s 262140. But it's still not working.I will be appreciate if someone has any ideas. Thank you!!