iris-edu / noise-toolkit

A code repository for the IRIS DMC Noise Toolkit data product.
https://ds.iris.edu/ds/products/noise-toolkit/
32 stars 10 forks source link

ntk_computePSD.py list slicing error #4

Closed elidana closed 1 year ago

elidana commented 1 year ago

Dear EarthScope data management center,

I am trying to use one of your great tools to perform some noise analysis of seismic data. I am interested in using the PDF/PSD package of the noise-toolkit, and I was successfully using that last year.

I was trying again today, and the main change since last time I was using is some upgrades on my local host.

When I run the "default" command suggested in the wiki quickinstall, I get the following error, which I think is related my version of matplotlilb.

This is the command and error

[elidana@fedora noise-toolkit]$ python bin/ntk_computePSD.py net=TA sta=O18A loc=DASH chan=BHZ start=2008-08-14T12:00:00 end=2008-08-14T13:00:00 xtype=period plot=1
[INFO] script: ntk_computePSD.py v.2.1.0 8 args: ['bin/ntk_computePSD.py', 'net=TA', 'sta=O18A', 'loc=DASH', 'chan=BHZ', 'start=2008-08-14T12:00:00', 'end=2008-08-14T13:00:00', 'xtype=period', 'plot=1']
[INFO] Use Internet to get metadata: True
[INFO] accessing logon client as nobody@iris.edu

******************** START ********************

[INFO] Requesting TA.--.O18A.BHZ from  2008-08-14T12:00:00  to  2008-08-14T13:00:00
[INFO] sending request to fedcatalog: http://service.iris.edu/irisws/fedcatalog/1/query?net=TA&sta=O18A&loc=--&cha=BHZ&targetservice=dataselect&level=channel&format=request&start=2008-08-14T12:00:00&end=2008-08-14T13:00:00&includeoverlaps=false&nodata=404
[INFO] from the IRISDMC,http://ds.iris.edu data center
[INFO] Initiating fedcatalog request for IRISDMC
[INFO] dataselect service is http://service.iris.edu/fdsnws/dataselect/1/
[INFO] Requesting data from IRISDMC_0 via http://service.iris.edu

[INFO] Reading TA.O18A.--.BHZ from 2008-08-14 12:00:00.0 to 2008-08-14 13:00:00.0 from FDSN stream
[INFO] ntk_computePSD.py 1 Trace(s) in Stream:
TA.O18A..BHZ | 2008-08-14T12:00:00.000000Z - 2008-08-14T13:00:00.000000Z | 40.0 Hz, 144001 samples
16384
Traceback (most recent call last):
  File "/home/elidana/tmp/aws-workshop/noise-toolkit/bin/ntk_computePSD.py", line 610, in <module>
    power, freq = csd(tr.data, tr.data, NFFT=nfft, noverlap=nfft * windlap, Fs=1. / delta, scale_by_freq=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 640, in csd
    Pxy, freqs, _ = _spectral_helper(x=x, y=y, NFFT=NFFT, Fs=Fs,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 382, in _spectral_helper
    result = _stride_windows(x, NFFT, noverlap)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 258, in _stride_windows
    return np.lib.stride_tricks.sliding_window_view(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: slice indices must be integers or None or have an __index__ method

I am using matplotlib 3.7.1 and Python 3.11.3 version.

Happy to provide further details if this is of any help.

Thanks a lot for any advice you are happy to share!

Elisabetta

manoch-iris commented 1 year ago

Dear Elisabetta,

Thank you for your contacting us. That is definitely a package conflict issue. Unfortunately I will be out until July 5th and I will check it then. Would you please make sure all your Python packages are up to date? Also, please confirm that you have not changed any of the parameter values in the param file. After these checks, if you still have the same issue, please reconfirm the versions of the Python modules and we will test and get back to you after July 4th. Thanks

On Wed, Jun 28, 2023 at 8:34 PM elidana @.***> wrote:

Dear EarthScope data management center,

I am trying to use one of your great tools to perform some noise analysis of our data. I am interested in using the PDF/PSD package of the noise-toolkit, and I was successfully using that last year.

I was trying again today, and the main change since last time I was using is some upgrades on my local host.

When I run the "default" command suggested in the wiki quickinstall, I get the following error, which I think is related to the version of matplotlilb.

This is the command and error

@. noise-toolkit]$ python bin/ntk_computePSD.py net=TA sta=O18A loc=DASH chan=BHZ start=2008-08-14T12:00:00 end=2008-08-14T13:00:00 xtype=period plot=1 [INFO] script: ntk_computePSD.py v.2.1.0 8 args: ['bin/ntk_computePSD.py', 'net=TA', 'sta=O18A', 'loc=DASH', 'chan=BHZ', 'start=2008-08-14T12:00:00', 'end=2008-08-14T13:00:00', 'xtype=period', 'plot=1'] [INFO] Use Internet to get metadata: True [INFO] accessing logon client as @.

**** START ****

[INFO] Requesting TA.--.O18A.BHZ from 2008-08-14T12:00:00 to 2008-08-14T13:00:00 [INFO] sending request to fedcatalog: http://service.iris.edu/irisws/fedcatalog/1/query?net=TA&sta=O18A&loc=--&cha=BHZ&targetservice=dataselect&level=channel&format=request&start=2008-08-14T12:00:00&end=2008-08-14T13:00:00&includeoverlaps=false&nodata=404 [INFO] from the IRISDMC,http://ds.iris.edu data center [INFO] Initiating fedcatalog request for IRISDMC [INFO] dataselect service is http://service.iris.edu/fdsnws/dataselect/1/ [INFO] Requesting data from IRISDMC_0 via http://service.iris.edu

[INFO] Reading TA.O18A.--.BHZ from 2008-08-14 12:00:00.0 to 2008-08-14 13:00:00.0 from FDSN stream [INFO] ntk_computePSD.py 1 Trace(s) in Stream: TA.O18A..BHZ | 2008-08-14T12:00:00.000000Z - 2008-08-14T13:00:00.000000Z | 40.0 Hz, 144001 samples 16384 Traceback (most recent call last): File "/home/elidana/tmp/aws-workshop/noise-toolkit/bin/ntk_computePSD.py", line 610, in power, freq = csd(tr.data, tr.data, NFFT=nfft, noverlap=nfft * windlap, Fs=1. / delta, scale_byfreq=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 640, in csd Pxy, freqs, = _spectral_helper(x=x, y=y, NFFT=NFFT, Fs=Fs, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 382, in _spectral_helper result = _stride_windows(x, NFFT, noverlap) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/elidana/.local/lib/python3.11/site-packages/matplotlib/mlab.py", line 258, in _stride_windows return np.lib.stride_tricks.sliding_window_view( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: slice indices must be integers or None or have an index method

I am using matplotlib 3.7.1 and Python 3.11.3 version.

Happy to provide further details if this is of any help.

Thanks a lot for any advice you are happy to share!

Elisabetta

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/noise-toolkit/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL5L2UC5NH77OTIMGCGUYDXNTEQJANCNFSM6AAAAAAZX2F2I4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Manochehr Bahavar Data Scientist @.***

https://www.earthscope.org

elidana commented 1 year ago

Dear @manoch-iris ,

thanks a lot for your super quick reply, really appreciate! No problem at all, I can definitively wait until you're back.

When I was trying yesterday, I did clone the noise-toolkit repo from scratch, so I confirm I did not changed any value in the param file.

I have tried again following these steps:

Below some details on my host and python and related packages version.

Once you're back, please let me know if I can provide further details!


host details: Linux fedora 6.3.7-100.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Jun 9 15:21:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

python version: Python 3.11.3

python packages

[elidana@fedora noise-toolkit]$ pip3 freeze
argcomplete==2.0.0
awscli==2.11.18
awscrt==0.16.21
Beaker==1.12.1
beautifulsoup4==4.12.2
blivet==3.5.0
blivet-gui==2.4.1
boto3==1.26.100
botocore==1.29.100
Brlapi==0.8.4
certifi==2023.5.7
cffi==1.15.1
chardet==5.1.0
charset-normalizer==2.1.0
click==8.1.3
colorama==0.4.6
contourpy==1.0.7
cryptography==41.0.1
cupshelpers==1.0
cycler==0.11.0
dasbus==1.6
dbus-python==1.3.2
decorator==5.1.1
distro==1.7.0
docutils==0.18.1
fb-re2==1.0.7
fedora-third-party==0.10
file-magic==0.4.0
fonttools==4.38.0
fros==1.1
gpg==1.17.0
greenlet==2.0.2
humanize==3.13.1
idna==3.3
jmespath==1.0.0
kiwisolver==1.4.4
langtable==0.0.62
libcomps==0.1.18
lxml==4.9.1
Mako==1.1.4
MarkupSafe==2.1.1
matplotlib==3.7.1
mercurial==6.2.3
nftables==0.1
numpy==1.24.2
obspy==1.4.0
olefile==0.46
packaging==21.3
panda==0.3.1
Paste==3.5.0
pexpect==4.8.0
pid==2.2.3
Pillow==9.2.0
ply==3.11
productmd==1.35
prompt-toolkit==3.0.29
ptyprocess==0.6.0
pwquality==1.4.5
pyasn1==0.4.8
pycairo==1.21.0
pycparser==2.20
pycrypto==2.6.1
pycups==2.0.1
pycurl==7.45.1
pyenchant==3.2.2
Pygments==2.12.0
PyGObject==3.42.2
pykickstart==3.48
pyOpenSSL==23.2.0
pyparsing==3.1.0
pyparted==3.12.0
PySocks==1.7.1
python-augeas==1.1.0
python-dateutil==2.8.2
python-meh==0.50
pyudev==0.24.1
pyxdg==0.28
PyYAML==6.0
regex==2023.6.3
requests==2.31.0
requests-file==1.5.1
requests-ftp==0.3.1
rpm==4.18.1
rsa==4.9
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
s3transfer==0.6.1
scipy==1.11.1
selinux==3.5
sepolicy==3.5
setools==4.4.0
simpleaudio==1.0.4
simpleline==1.9.0
six==1.16.0
sos==4.4
soupsieve==2.4.1
SQLAlchemy==2.0.17
systemd-python==235
Tempita==0.5.2
typing_extensions==4.7.0
urllib3==1.26.16
wcwidth==0.2.6
manoch-iris commented 1 year ago

Dear @elidana,

Thank you for reporting the issue. We reproduced the error you reported and released a patch for it. Please download the latest version and let us know if the issue is not resolved. Thanks!

elidana commented 1 year ago

Dear @manoch-iris ,

apologies for the late reply but I confirm that the noise toolkit is now working fine for me!

Thank you so much, really appreciate the super quick resolution!!