search

seismo-live / seismo_live

Live Jupyter Notebooks for Seismology
http://seismo-live.org
78 stars 73 forks source link

unable to build the notebooks #41

Open Thomas-Ulrich opened 4 years ago

Thomas-Ulrich commented 4 years ago

Hi,

For some reason (we were trying to create a new seismology exercise derived for rupture_2D), I tried to build the notebooks. And it turns out that I cannot build them.

Here is what I did:

pip install virtualenv
python -m venv envSeismoLive
source envSeismoLive/bin/activate
pip install --upgrade pip
pip install numpy
pip install scipy matplotlib jupyter jupytext nbconvert obspy pandas seaborn
python conf/convert_to_ipynb.py notebooks built_notebooks

(Python 3.7.4)

and here is what I get:

(envSeismoLive) (base) ulrich@ulrich-ThinkPad-T490s:~/Downloads/seismo_live$ python conf/convert_to_ipynb.py notebooks built_notebooks
Convert to .ipynb file.
[jupytext] Reading built_notebooks/notebooks/Data Visualization/Intro_to_visualization.py
[jupytext] Writing built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb (destination file replaced)
Running .ipynb file: built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 795726 bytes to built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
Converting to HTML: built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb to html
[NbConvertApp] Writing 1109632 bytes to built_notebooks/html/Data Visualization/Intro_to_visualization.html
Convert to .ipynb file.
[jupytext] Reading built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.py
[jupytext] Writing built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
Running .ipynb file: built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Error while converting 'built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb'
Traceback (most recent call last):
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 197, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/notebook.py", line 32, in from_notebook_node
    nb_copy, resources = super(NotebookExporter, self).from_notebook_node(nb, resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 139, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 316, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 448, in preprocess_cell
    raise CellExecutionError.from_cell_and_msg(cell, out)
nbconvert.preprocessors.execute.CellExecutionError: An error occurred while executing the following cell:
------------------
# Preprocessing 2
st = stp.copy()                            # copy stream

for tr in st:
    tr = normalize(tr, norm_method="1bit")
    tr = whiten(tr, 0.1, 0.2)
print ('done!')
------------------

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/numpy/core/function_base.py in linspace(start, stop, num, endpoint, retstep, dtype, axis)
    116     try:
--> 117         num = operator.index(num)
    118     except TypeError:

TypeError: 'float' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-5-290f7feaa7bd> in <module>
      4 for tr in st:
      5     tr = normalize(tr, norm_method="1bit")
----> 6     tr = whiten(tr, 0.1, 0.2)
      7 print ('done!')

<ipython-input-4-39b7c3512fe4> in whiten(tr, freqmin, freqmax)
     82 
     83         # Apodization to the right with cos^2 (to smooth the discontinuities)
---> 84         smo2 = (np.cos(np.linspace(0., np.pi/2., nsmo+1.))**2.)
     85         espo = np.exp(1j * np.angle(FFTs[JJ[-1]-nsmo:JJ[-1]+1]))
     86         FFTsW[JJ[-1]-nsmo:JJ[-1]+1] = smo2 * espo

<__array_function__ internals> in linspace(*args, **kwargs)

~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/numpy/core/function_base.py in linspace(start, stop, num, endpoint, retstep, dtype, axis)
    119         raise TypeError(
    120             "object of type {} cannot be safely interpreted as an integer."
--> 121                 .format(type(num)))
    122 
    123     if num < 0:

TypeError: object of type <class 'float'> cannot be safely interpreted as an integer.
TypeError: object of type <class 'float'> cannot be safely interpreted as an integer.

Traceback (most recent call last):
  File "conf/convert_to_ipynb.py", line 249, in <module>
    html_folder=html_folder,
  File "conf/convert_to_ipynb.py", line 223, in convert_folder
    html_folder=html_folder,
  File "conf/convert_to_ipynb.py", line 186, in convert_file
    check=True,
  File "/home/ulrich/miniconda3/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['jupyter', 'nbconvert', '--to', 'notebook', '--ExecutePreprocessor.timeout=600', '--execute', '--inplace', 'built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb']' returned non-zero exit status 1.

Any idea what could be the problem? Seems like a problem in obspy?

krischer commented 4 years ago

Hi Thomas,

this bug is due to newer numpy versions no longer accepting floating point numbers in the np.linspace call for the number of desired array elements.

This has to be changed in the whiten() function in the Ambient Seismic Noise/NoiseCorrelation notebook.

Thomas-Ulrich commented 4 years ago

ok thx, I can write a pull request. Next error:

------------------
from obspy import read, read_inventory

st = read("data/GR.FUR..BHN.D.2015.361")
inv = read_inventory("data/station_FUR.stationxml")

print(st)
print(inv)
inv.plot(projection="ortho");
------------------

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-5bbf216cc9f8> in <module>
      6 print(st)
      7 print(inv)
----> 8 inv.plot(projection="ortho");

~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/obspy/core/inventory/inventory.py in plot(self, projection, resolution, continent_fill_color, water_fill_color, marker, size, label, color, color_per_network, colormap, legend, time, show, outfile, method, fig, **kwargs)
   1003                        water_fill_color=water_fill_color,
   1004                        colormap=None, colorbar=False, marker=marker,
-> 1005                        title=None, show=False, fig=fig, **kwargs)
   1006 
   1007         if legend is not None and color_per_network:

~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/obspy/imaging/maps.py in plot_map(method, *args, **kwargs)
    793             return plot_basemap(*args, **kwargs)
    794         elif HAS_CARTOPY:
--> 795             return plot_cartopy(*args, **kwargs)
    796         else:
    797             raise ImportError('Neither Basemap nor Cartopy could be imported.')

~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/obspy/imaging/maps.py in plot_cartopy(lons, lats, size, color, labels, projection, resolution, continent_fill_color, water_fill_color, colormap, colorbar, marker, title, colorbar_ticklabel_format, show, proj_kwargs, **kwargs)
    735     scatter = map_ax.scatter(lons, lats, marker=marker, s=size, c=color,
    736                              zorder=10, cmap=colormap,
--> 737                              transform=ccrs.Geodetic())
    738 
    739     if title:

~/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/cartopy/mpl/geoaxes.py in wrapper(self, *args, **kwargs)
    305             raise ValueError('Invalid transform: Spherical {} '
    306                              'is not supported - consider using '
--> 307                              'PlateCarree/RotatedPole.'.format(func.__name__))
    308 
    309         kwargs['transform'] = transform

ValueError: Invalid transform: Spherical scatter is not supported - consider using PlateCarree/RotatedPole.
ValueError: Invalid transform: Spherical scatter is not supported - consider using PlateCarree/RotatedPole.

I guess this notebook was written for basemap.

krischer commented 4 years ago

Hmm - what is your basemap version?

Edit: Sorry not basemap but cartopy.

I feel like this should work.

Thomas-Ulrich commented 4 years ago

0.18.0

krischer commented 4 years ago

@megies Do you know what's going on here?

Thomas-Ulrich commented 4 years ago

Just wanted to see if I can complete the build to the end using basemap: But no, I get another error (time-out?).

(envSeismoLive) (base) ulrich@ulrich-ThinkPad-T490s:~/Downloads/seismo_live$ python conf/convert_to_ipynb.py notebooks built_notebooks
Convert to .ipynb file.
[jupytext] Reading built_notebooks/notebooks/Data Visualization/Intro_to_visualization.py
[jupytext] Writing built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
Running .ipynb file: built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 791190 bytes to built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
Converting to HTML: built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Data Visualization/Intro_to_visualization.ipynb to html
[NbConvertApp] Writing 1105096 bytes to built_notebooks/html/Data Visualization/Intro_to_visualization.html
Convert to .ipynb file.
[jupytext] Reading built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.py
[jupytext] Writing built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
Running .ipynb file: built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 109734 bytes to built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
Converting to HTML: built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Ambient Seismic Noise/NoiseCorrelation.ipynb to html
[NbConvertApp] Writing 427469 bytes to built_notebooks/html/Ambient Seismic Noise/NoiseCorrelation.html
Convert to .ipynb file.
[jupytext] Reading built_notebooks/notebooks/Ambient Seismic Noise/Probabilistic Power Spectral Densities.py
[jupytext] Writing built_notebooks/notebooks/Ambient Seismic Noise/Probabilistic Power Spectral Densities.ipynb
Running .ipynb file: built_notebooks/notebooks/Ambient Seismic Noise/Probabilistic Power Spectral Densities.ipynb
[NbConvertApp] Converting notebook built_notebooks/notebooks/Ambient Seismic Noise/Probabilistic Power Spectral Densities.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Kernel died while waiting for execute reply.
Traceback (most recent call last):
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 478, in _poll_for_reply
    msg = self.kc.shell_channel.get_msg(timeout=timeout)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/bin/jupyter-nbconvert", line 8, in <module>
    sys.exit(main())
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/jupyter_core/application.py", line 270, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/traitlets/config/application.py", line 664, in launch_instance
    app.start()
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 340, in start
    self.convert_notebooks()
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 510, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 481, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 197, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/notebook.py", line 32, in from_notebook_node
    nb_copy, resources = super(NotebookExporter, self).from_notebook_node(nb, resources, **kw)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 139, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 316, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 438, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 578, in run_cell
    exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 483, in _poll_for_reply
    self._check_alive()
  File "/home/ulrich/Downloads/seismo_live/envSeismoLive/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 510, in _check_alive
    raise DeadKernelError("Kernel died")
nbconvert.preprocessors.execute.DeadKernelError: Kernel died
Traceback (most recent call last):
  File "conf/convert_to_ipynb.py", line 249, in <module>
    html_folder=html_folder,
  File "conf/convert_to_ipynb.py", line 223, in convert_folder
    html_folder=html_folder,
  File "conf/convert_to_ipynb.py", line 186, in convert_file
    check=True,
  File "/home/ulrich/miniconda3/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['jupyter', 'nbconvert', '--to', 'notebook', '--ExecutePreprocessor.timeout=600', '--execute', '--inplace', 'built_notebooks/notebooks/Ambient Seismic Noise/Probabilistic Power Spectral Densities.ipynb']' returned non-zero exit status 1.
krischer commented 4 years ago

You can try to increase the timeout in the script: just set --ExecutePreprocessor.timeout=600 to something higher.

Thomas-Ulrich commented 4 years ago

no it does not work. Tried with --ExecutePreprocessor.timeout=6000000000

megies commented 4 years ago

@megies Do you know what's going on here?

sorry, never seen that and still doing most of my stuff in basemap.. slow on the transition :blush: