scottcoughlin2014
commented
6 years ago Hey Iris,
Alright that error is insanely not helpful. The problem is that you are not providing the method with any idea of how to access the timeseries it should be using to do the classifications.
In your case I would recommend starting with a smaller subset of the table (242000 is a lot) and also from one ifo, then you can pass channel name and frametype key word arguments (see example below) and it will find that Livingston data on the fly for you. I will work on the error message right now so that it lets you know you did not provide any information on how to find the timeseries and not give you that uninformative error.
Looks good thoguh when i just ran it here
In [12]: downsample = triggers[triggers['ifo'] == 'L1'][0:5]
In [13]: downsample
Out[13]:
<Events masked=True length=5>
Gamma0 Gamma1 bank_chisq bank_chisq_dof chisq chisq_dof coa_phase eff_distance end_time end_time_ns event_id ifo ... gravityspy_id image_status data_quality upload_flag citizen_score links_subjects url1 url2 url3 url4 event_time
float32 float32 float32 int32 float32 int32 float32 float32 int32 int32 object str2 ... str10 str7 str7 int64 float64 int64 str1 str1 str1 str1 float64
-------- ------- ---------- -------------- --------- --------- ------------ ------------ ---------- ----------- -------------------------- ---- ... ------------- ------------ ------------ ----------- ------------- -------------- ---- ---- ---- ---- --------------------
500014.0 0.0 0.0 0 1.8513647 1 -1.3555727 -- 1169213344 416265849 sngl_inspiral:event_id:421 L1 ... MBFPxbkjIX testing no_flag 0 0.0 0 1169213344.416265965
503461.0 0.0 0.0 0 0.8534533 1 -0.757856 -- 1169213527 157940997 sngl_inspiral:event_id:440 L1 ... fYufgu1nB0 testing no_flag 0 0.0 0 1169213527.157941103
505753.0 0.0 0.0 0 1.5253383 1 -1.8398995 -- 1169214145 259769771 sngl_inspiral:event_id:532 L1 ... FvtKl0NJKB testing no_flag 0 0.0 0 1169214145.259769678
505827.0 0.0 0.0 0 1.6399359 1 -0.040860385 -- 1169214699 354646835 sngl_inspiral:event_id:601 L1 ... Gt9wR6AwOR testing no_flag 0 0.0 0 1169214699.354646921
502610.0 0.0 0.0 0 0.8289764 1 -0.5243332 -- 1169216181 406872915 sngl_inspiral:event_id:807 L1 ... TN9Vj6eZJl testing no_flag 0 0.0 0 1169216181.406872988
In [14]: tmp = downsample.classify(path_to_cnn='/home/gravityspy/GravitySpy/models/multi_view_classifier.h5', channel_name='L1:GDS-CALIB_STRAIN', frametype='L1_HOFT_C00')
/home/gravityspy/opt/GravitySpy-py36-Development/lib/python3.6/site-packages/skimage/transform/_warps.py:24: UserWarning: The default multichannel argument (None) is deprecated. Please specify either True or False explicitly. multichannel will default to False starting with release 0.16.
warn('The default multichannel argument (None) is deprecated. Please '
/home/gravityspy/opt/GravitySpy-py36-Development/lib/python3.6/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images.
warn("Anti-aliasing will be enabled by default in skimage 0.15 to "
In [15]: tmp
Out[15]:
<Events masked=True length=5>
1080Lines 1400Ripples Air_Compressor Blip Chirp Extremely_Loud Helix Koi_Fish Light_Modulation Low_Frequency_Burst ... data_quality upload_flag citizen_score links_subjects url1 url2 url3 url4 event_time q_value
float32 float32 float32 float32 float32 float32 float32 float32 float32 float32 ... str7 float64 float64 float64 str1 str1 str1 str1 float64 float64
------------- ------------- -------------- ------------- ------------- -------------- ------------- ------------- ---------------- ------------------- ... ------------ ----------- ------------- -------------- ---- ---- ---- ---- ------------------ ------------------
0.0002722652 0.00068926497 0.00017827445 0.0003645657 4.0709783e-05 1.788409e-06 0.0013431425 7.159639e-05 0.00012706574 0.3605168 ... no_flag 0.0 0.0 0.0 1169214145.2597697 22.627416997969522
1.5008998e-08 2.0107538e-07 2.425204e-08 6.372219e-06 1.4761399e-09 2.0642229e-07 8.109753e-11 2.3767308e-07 1.5942669e-05 0.007948713 ... no_flag 0.0 0.0 0.0 1169214699.354647 45.25483399593904
0.028634015 0.00015153381 0.00016503473 0.00081625616 0.0001472924 9.587992e-05 0.0006770451 1.7178703e-05 1.9254543e-05 0.00019112768 ... no_flag 0.0 0.0 0.0 1169213344.416266 45.25483399593904
0.0001591614 0.0002688605 1.9412546e-05 0.00020471802 6.2337385e-06 1.1169099e-05 0.00015485396 2.4428195e-05 0.000381595 0.26364598 ... no_flag 0.0 0.0 0.0 1169216181.406873 11.31370849898476
3.1220665e-05 1.589899e-05 6.787829e-07 1.999569e-05 2.677202e-07 2.3677474e-07 1.3981353e-06 2.0006736e-07 1.7251818e-07 0.0001908571 ... no_flag 0.0 0.0 0.0 1169213527.157941 45.25483399593904
Hi Scott, I implemented some changes in events.py to be able to run over a gstlal data file. The diff is pasted below. 51,52c51 < #etg = kwargs.pop('etg', 'OMICRON') < etg = kwargs.pop('etg','GSTLAL') --- > etg = kwargs.pop('etg', 'OMICRON') 71,75d69 < if etg == 'GSTLAL': < #print(type(tab['event_id'])) < tab['event_id'] = tab['event_id'].apply(int) < tab['process_id'] = tab['process_id'].apply(int) < 82,86d75 < < if etg == 'GSTLAL': < tab['event_time'] = (tab['end_time'] + < (0.000000001)*tab['end_time_ns']) < tab['event_time'].format = '%.9f' I was able to run: >>>from gravityspy.table import Events >>>triggers=Events.read('/home/iris.ruiter/projects/gravity_spy/test/H1L1-ALL_LLOID-1169107218-1066800.xml', tablename='sngl_inspiral') which seemed to work. The next step failed >>>results = triggers.classify(path_to_cnn='/home/iris.ruiter/g_spy_src/GravitySpy/models/multi_view_classifier.h5') with the following error message, related to the setting of nproc. results = triggers.classify(path_to_cnn='/home/iris.ruiter/g_spy_src/GravitySpy/models/multi_view_classifier.h5', nproc=3) Traceback (most recent call last): File "", line 1, in
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gravityspy/table/events.py", line 139, in classify
inputs)
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gwpy/utils/mp.py", line 171, in multiprocess_with_queues
raise res
NameError: name 'nproc' is not defined
>>> results = triggers.classify(path_to_cnn='/home/iris.ruiter/g_spy_src/GravitySpy/models/multi_view_classifier.h5')
Traceback (most recent call last):
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gravityspy/table/events.py", line 539, in _make_single_qscan
utils.save_q_scans(plot_directory, specsgrams,
UnboundLocalError: local variable 'specsgrams' referenced before assignment
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gravityspy/table/events.py", line 139, in classify
inputs)
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gwpy/utils/mp.py", line 131, in multiprocess_with_queues
return list(map(_inner, inputs))
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gwpy/utils/mp.py", line 118, in _inner
return func(x)
File "/home/iris.ruiter/opt/GravitySpy-py36/lib/python3.6/site-packages/gravityspy/table/events.py", line 545, in _make_single_qscan
if nproc == 1:
NameError: name 'nproc' is not defined
I'd appreciate any feedback on the diff and also advice on how to run the classify step, ie. which machine would you recommend and any insight into this error.
Best,
Iris