search

smousavi05 / EQTransformer

EQTransformer, a python package for earthquake signal detection and phase picking using AI.
https://rebrand.ly/EQT-documentations
MIT License
311 stars 150 forks source link

Dimension incompatible in tutorial example #113

Closed RunningXinLiu closed 2 years ago

RunningXinLiu commented 2 years ago

Hi Seyed,

Thank you for making this interesting work public. However, I encountered some issues when running the tutorial code. My device is mac OS Monterey version 12.2.1 (21D62) chip Apple M1 Pro. Tensorflow version 2. 5. 0, image

When I was running the tutorial code from the EQTransformer via Google Colab, at the detection part,

from EQTransformer.core.mseed_predictor import mseed_predictor

mseed_predictor(input_dir='/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/downloads_mseeds',
                input_model='/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/ModelsAndSampleData/EqT_model.h5',
                stations_json='json/station_list.json',
                output_dir='detection_results',
                detection_threshold=0.2,                
                P_threshold=0.1,
                S_threshold=0.1, 
                number_of_plots=10,
                plot_mode='time_frequency',
                batch_size=500,
                overlap=0.3)

I got the incompatible issue. The detailed information is shown below:


> 03-29 [1](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=0)3:01 [INFO] [EQTransformer] Running EqTransformer  0.1.61
> 03-29 13:01 [INFO] [EQTransformer] *** Loading the model ...
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_2 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.WARNING:tensorflow:Layer lstm_2 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_2 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> WARNING:tensorflow:Layer lstm_4 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_4 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> WARNING:tensorflow:Layer lstm_5 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [WARNING] [tensorflow] Layer lstm_5 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
> 03-29 13:01 [INFO] [EQTransformer] *** Loading is complete!
> 03-29 13:01 [INFO] [EQTransformer] There are files for 3 stations in /Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/downloads_mseeds directory.
> 03-29 13:01 [INFO] [EQTransformer] Started working on B921, 1 out of 3 ...
> 03-29 13:01 [INFO] [EQTransformer] 20190901T000000Z__20190902T000000Z.mseed
> 2022-03-29 13:01:09.196025: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> /Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb Cell 19' in <cell line: 3>()
>       1[ from EQTransformer.core.mseed_predictor import mseed_predictor
> ----> ]()[3](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=2)[ mseed_predictor(input_dir='/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/downloads_mseeds',
>       ]()[4](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=3)[                 input_model='/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/ModelsAndSampleData/EqT_model.h5',
>       ]()[5](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=4)[                 stations_json='json/station_list.json',
>       ]()[6](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=5)[                 output_dir='detection_results',
>       ]()[7](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=6)[                 detection_threshold=0.2,                
>       ]()[8](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=7)[                 P_threshold=0.1,
>       ]()[9](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=8)[                 S_threshold=0.1, 
>      ]()[10](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=9)[                 number_of_plots=10,
>      ]()[11](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=10)[                 plot_mode='time_frequency',
>      ]()[12](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=11)[                 batch_size=500,
>      ]()[13](vscode-notebook-cell:/Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformerviaGoogleColab.ipynb#ch0000018?line=12)[                 overlap=0.3)
> 
> File ~/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py:332, in mseed_predictor(input_dir, input_model, stations_json, output_dir, detection_threshold, P_threshold, S_threshold, number_of_plots, plot_mode, loss_weights, loss_types, normalization_mode, batch_size, overlap, gpuid, gpu_limit, overwrite, output_probabilities)
>     ]()[329](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=328)[                 HDF_PROB.flush()
>     ]()[331](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=330)[             if plt_n < args['number_of_plots'] and post_write > pre_write:
> --> ]()[332](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=331)[                 _plotter_prediction(data_set[meta["trace_start_time"][ix]], args, save_figs, predD[ix][:, 0], predP[ix][:, 0], predS[ix][:, 0], meta["trace_start_time"][ix], matches)
>     ]()[333](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=332)[                 plt_n += 1            
>     ]()[335](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=334)[ end_Predicting = time.time() 
> 
> File ~/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py:1147, in _plotter_prediction(data, args, save_figs, yh1, yh2, yh3, evi, matches)
>    ]()[1145](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=1144)[ f, t, Pxx = signal.stft(data[:, 0], fs=100, nperseg=80)
>    ]()[1146](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=1145)[ Pxx = np.abs(Pxx)                       
> -> ]()[1147](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=1146)[ plt.pcolormesh(t, f, Pxx, alpha=None, cmap='hot', shading='flat', antialiased=True)
>    ]()[1148](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=1147)[ plt.ylim(0, 40)
>    ]()[1149](file:///Users/liuxin/Documents/projects/tensorflow_env/eq_transformer/EQTransformer/EQTransformer/core/mseed_predictor.py?line=1148)[ plt.text(1, 1, 'STFT', fontdict=font0)
> 
> File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py:2718, in pcolormesh(alpha, norm, cmap, vmin, vmax, shading, antialiased, data, *args, **kwargs)
>    ]()[2713](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2712)[ @_copy_docstring_and_deprecators(Axes.pcolormesh)
>    ]()[2714](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2713)[ def pcolormesh(
>    ]()[2715](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2714)[         *args, alpha=None, norm=None, cmap=None, vmin=None,
>    ]()[2716](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2715)[         vmax=None, shading=None, antialiased=False, data=None,
>    ]()[2717](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2716)[         **kwargs):
> -> ]()[2718](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2717)[     __ret = gca().pcolormesh(
>    ]()[2719](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2718)[         *args, alpha=alpha, norm=norm, cmap=cmap, vmin=vmin,
>    ]()[2720](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2719)[         vmax=vmax, shading=shading, antialiased=antialiased,
>    ]()[2721](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2720)[         **({"data": data} if data is not None else {}), **kwargs)
>    ]()[2722](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2721)[     sci(__ret)
>    ]()[2723](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/pyplot.py?line=2722)[     return __ret
> 
> File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py:1412, in _preprocess_data.<locals>.inner(ax, data, *args, **kwargs)
>    ]()[1409](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1408)[ @functools.wraps(func)
>    ]()[1410](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1409)[ def inner(ax, *args, data=None, **kwargs):
>    ]()[1411](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1410)[     if data is None:
> -> ]()[1412](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1411)[         return func(ax, *map(sanitize_sequence, args), **kwargs)
>    ]()[1414](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1413)[     bound = new_sig.bind(ax, *args, **kwargs)
>    ]()[1415](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1414)[     auto_label = (bound.arguments.get(label_namer)
>    ]()[1416](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/__init__.py?line=1415)[                   or bound.kwargs.get(label_namer))
> 
> File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py:6062, in Axes.pcolormesh(self, alpha, norm, cmap, vmin, vmax, shading, antialiased, *args, **kwargs)
>    ]()[6059](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6058)[ shading = shading.lower()
>    ]()[6060](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6059)[ kwargs.setdefault('edgecolors', 'none')
> -> ]()[6062](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6061)[ X, Y, C, shading = self._pcolorargs('pcolormesh', *args,
>    ]()[6063](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6062)[                                     shading=shading, kwargs=kwargs)
>    ]()[6064](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6063)[ coords = np.stack([X, Y], axis=-1)
>    ]()[6065](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=6064)[ # convert to one dimensional array
> 
> File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py:5575, in Axes._pcolorargs(self, funcname, shading, *args, **kwargs)
>    ]()[5573](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5572)[ if shading == 'flat':
>    ]()[5574](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5573)[     if (Nx, Ny) != (ncols + 1, nrows + 1):
> -> ]()[5575](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5574)[         raise TypeError('Dimensions of C %s are incompatible with'
>    ]()[5576](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5575)[                         ' X (%d) and/or Y (%d); see help(%s)' % (
>    ]()[5577](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5576)[                             C.shape, Nx, Ny, funcname))
>    ]()[5578](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5577)[ else:    # ['nearest', 'gouraud']:
>    ]()[5579](file:///opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/matplotlib/axes/_axes.py?line=5578)[     if (Nx, Ny) != (ncols, nrows):
> 
**> TypeError: Dimensions of C (41, 151) are incompatible with X (151) and/or Y (41); see help(pcolormesh)]()**
RunningXinLiu commented 2 years ago

My python version is 3.9.12

smousavi05 commented 2 years ago

@RunningXinLiu It seems the issue is related to the plot generation (which itself can be due to your python or scipy version). Anyway, it seems the code has been already ran and detected some events and just has problem generating some auxiliary figures. So before diagnosing the issue more closely, could you either turn off the plot generation (i.e. number_of_plots=0) or change its mode (i.e. plot_mode='time') and see what will happen?

RunningXinLiu commented 2 years ago

@RunningXinLiu It seems the issue is related to the plot generation (which itself can be due to your python or scipy version). Anyway, it seems the code has been already ran and detected some events and just has problem generating some auxiliary figures. So before diagnosing the issue more closely, could you either turn off the plot generation (i.e. number_of_plots=0) or change its mode (i.e. plot_mode='time') and see what will happen?

Thank you Seyed, it worked! Sorry for the late reply. I am playing with your code to see how to visualize it (Because now zero figures are genereated).