search

MetPX / sarracenia

https://MetPX.github.io/sarracenia
GNU General Public License v2.0
44 stars 22 forks source link

sr_subscribe edit error #251

Open scottkehler opened 5 years ago

scottkehler commented 5 years ago

When I run the following command per the subscribe instructions, I get an error:

sr_subscribe edit swob.conf

2019-08-28 21:12:52,637 [INFO] using vi. Set EDITOR variable pick another one. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sarra/sr_config.py", line 1194, in run_command subprocess.run(cmd_list,check=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.6/subprocess.py", line 729, in init restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child restore_signals, start_new_session, preexec_fn) TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/sr_subscribe", line 11, in load_entry_point('metpx-sarracenia==2.19.4b2', 'console_scripts', 'sr_subscribe')() File "/usr/lib/python3/dist-packages/sarra/sr_subscribe.py", line 2219, in main subscribe.exec_action(action,old) File "/usr/lib/python3/dist-packages/sarra/sr_instances.py", line 287, in exec_action elif action == 'edit' : self.exec_action_on_config(action) File "/usr/lib/python3/dist-packages/sarra/sr_instances.py", line 441, in exec_action_on_config self.run_command([ editor, edit_fil] ) File "/usr/lib/python3/dist-packages/sarra/sr_config.py", line 1195, in run_command except: self.logger.error("trying run command %s " % ' '.join(cmd_list) ) TypeError: sequence item 1: expected str instance, NoneType found

I have tried installing sarracenia on mac (using pip) and on linux using the pip and apt-get methods. In all cases I get the error above when trying to add/edit a new file.

petersilva commented 5 years ago

OK, that is a bug. we will work on it. To get you going for now...

It is looking for the vi editor (which is a very nerdy option, but normal for Linux.) You need to tell it where to find the text editor you prefer to use:

export EDITOR=/path/to/editor/of/your/choice

the other possibility is that the configuration file does not exist... to get a file to edit, use add first:

sr_subscribe add swob.conf

then try editing... hope this will get you going for now.

scottkehler commented 5 years ago

I am able to edit some files using the default editor (vi). E.g. if I do sr_subscribe edit credentials.conf I am able to open the credentials file.

I tried setting EDITOR to nano and that got rid of this error:

2019-08-28 21:12:52,637 [INFO] using vi. Set EDITOR variable pick another one.

But the remainder of the error message is the same.

petersilva commented 5 years ago

this is odd... did you try the add?

you can try:

mkdir ~/.config/sarra/subscribe # in case it isn't there yet. nano ~/.config/sarra/subscribe/swob.conf

then try edit again, and if the file exists it ought to work. I guess the dumb question: what version are you running?
petersilva commented 5 years ago

ignore the version question I noticed it is the latest in reading above.

petersilva commented 5 years ago

fwiw I can reproduce it... really odd.

scottkehler commented 5 years ago

The ~/.config/sarra/subscribe did not exist, so I made it. But it still doesn't work because it seems to be trying to use the directory shown below:

(scott_p372) Scotts-MBP:real_data_download scottkehler$ sr_subscribe add swob.conf 2019-08-28 20:38:20,377 [ERROR] could not add swob.conf to /Users/scottkehler/Library/Application Support/sarra/subscribe

If I try doing sr_subscribe edit swob.conf I get the longer error message I posted before.

petersilva commented 5 years ago

yeah once the file exists, it is fine. on linux... Oh, I thought you were on linux... if you are on Mac it is in a different place. do sr_subscribe list

on my linux box is looks like this:

blacklab% sr_subscribe list

configuration examples: ( /home/peter/src/sarracenia/sarra/examples/subscribe ) WMO_Sketch_2mqtt.conf WMO_Sketch_2v3.conf WMO_mesh_CMC.conf WMO_mesh_Peer.conf amqp_f30.conf aws-nexrad.conf cclean_f91.conf cdnld_f21.conf cfile_f44.conf cp_f61.conf dd_2mqtt.conf dd_all.conf dd_amis.conf dd_aqhi.conf dd_cacn_bulletins.conf dd_citypage.conf dd_cmml.conf dd_gdps.conf dd_ping.conf dd_radar.conf dd_rdps.conf dd_swob.conf ddc_cap-xml.conf ddc_normal.conf downloademail.conf ec_ninjo-a.conf ftp_f70.conf hpfx_amis.conf local_sub.conf pitcher_pull.conf q_f71.conf rabbitmqtt_f31.conf sci2ec.conf subnoaa.conf subsoapshc.conf subusgs.conf u_sftp_f60.conf

general: ( /home/peter/.config/sarra ) admin.conf credentials.conf default.conf

user configurations: ( /home/peter/.config/sarra/subscribe ) amqp_f30.conf amqp_f30.conf.off cclean_f91.conf cdnld_f21.conf cfile_f44.conf cp_f61.conf ftp_f70.conf local_sub.conf q_f71.conf rabbitmqtt_f31.conf u_sftp_f60.conf

logs are in: /home/peter/.cache/sarra/log

blacklab%

you need to make sure the user configurations directory exists. and create swob.conf in that directory.

petersilva commented 5 years ago

try mkdir /Users/scottkehler/Library/Application Support/sarra/subscribe

scottkehler commented 5 years ago

I tried manually putting the swob.conf file in /Users/scottkehler/Library/Application Support/sarra directory and it will start from there and I can edit it.

petersilva commented 5 years ago

you probably need quotes or a backslash to deal with the space.

petersilva commented 5 years ago

off by one... need to mkdir subscribe in there and move swob.conf inside that.

scottkehler commented 5 years ago

This is the result of sr_subscribe list:

(scott_p372) Scotts-MBP:sarra scottkehler$ sr_subscribe list

configuration examples: ( /Users/scottkehler/anaconda3/envs/scott_p372/lib/python3.6/site-packages/sarra/examples/subscribe ) WMO_Sketch.conf WMO_Sketch_2mqtt.conf WMO_Sketch_2v3.conf WMO_mesh_CMC.conf WMO_mesh_Peer.conf aws-nexrad.conf cclean_f91.conf cdnld_f21.conf cfile_f44.conf cp_f61.conf dd_2mqtt.conf dd_all.conf dd_amis.conf dd_aqhi.conf dd_cacn_bulletins.conf dd_citypage.conf dd_cmml.conf dd_gdps.conf dd_ping.conf dd_radar.conf dd_rdps.conf dd_swob.conf ddc_cap-xml.conf ddc_normal.conf downloademail.conf ec_ninjo-a.conf ftp_f70.conf local_sub.conf pitcher_pull.conf q_f71.conf sci2ec.conf subnoaa.conf subsoapshc.conf subusgs.conf t_f30.conf u_sftp_f60.conf

general: ( /Users/scottkehler/Library/Application Support/sarra ) credentials.conf default.conf

user configurations: ( /Users/scottkehler/Library/Application Support/sarra/subscribe ) swob.conf

logs are in: /Users/scottkehler/Library/Logs/sarra

--

The script seems to work fine if I add to sarra/subscribe manually, so maybe there's some permission issue preventing the sr_subscribe command from adding files?

petersilva commented 5 years ago

that looks good... yeah, it's like it didn't create the directory to add the config too. That's one problem. The other problem is that I think if it knows a file doesn't exist, it gives a reasonable error message:


blacklab% 
blacklab% sr_subscribe edit hoho.conf
2019-08-28 21:53:19,123 [ERROR] could not identify file to edit: hoho.conf
blacklab%

The problem seems to be that there is a file in the templates provided called dd_swob.conf, and that partial match seems to be confusing things. if I do sr_subscribe list swob.conf It prints out a config file... it is actually mismatching and printing out the dd_swob.conf file... pretty odd. So if there is a similar file, it gets confused.

petersilva commented 5 years ago

hopefully you can proceed for now.

petersilva commented 5 years ago

@iMacadan salut Daniel, t'est le seul avec accès quotidien à un mac... ca sera bien de reproduire son bobo.

scottkehler commented 5 years ago

Yes, I'm able to proceed for the time being. I suddenly have a large number of swob files on my computer!

Thanks for your help.

petersilva commented 5 years ago

great! good luck!

iMacadan commented 5 years ago

Ok , I'll have a look at it.

iMacadan commented 5 years ago

Here my first attempt (I installed Sarracenia for the first time on my iMac - v. 2.19.04b2)

configuration examples: ( /usr/local/lib/python3.6/site-packages/sarra/examples/subscribe ) WMO_Sketch.conf WMO_Sketch_2mqtt.conf WMO_Sketch_2v3.conf WMO_mesh_CMC.conf WMO_mesh_Peer.conf aws-nexrad.conf cclean_f91.conf cdnld_f21.conf cfile_f44.conf cp_f61.conf dd_2mqtt.conf dd_all.conf dd_amis.conf dd_aqhi.conf dd_cacn_bulletins.conf dd_citypage.conf dd_cmml.conf dd_gdps.conf dd_ping.conf dd_radar.conf dd_rdps.conf dd_swob.conf ddc_cap-xml.conf ddc_normal.conf downloademail.conf ec_ninjo-a.conf ftp_f70.conf local_sub.conf pitcher_pull.conf q_f71.conf sci2ec.conf subnoaa.conf subsoapshc.conf subusgs.conf t_f30.conf u_sftp_f60.conf

general: ( /Users/user/Library/Application Support/sarra ) credentials.conf default.conf

user configurations: ( /Users/user/Library/Application Support/sarra/subscribe )

logs are in: /Users/user/Library/Logs/sarra


I get somewhat the same error  when trying to edit swob.conf

$ sr_subscribe edit swob.conf 2019-09-15 19:11:34,707 [INFO] using vi. Set EDITOR variable pick another one. Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/sarra/sr_config.py", line 1194, in run_command subprocess.run(cmd_list,check=True) File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 403, in run with Popen(*popenargs, **kwargs) as process: File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 709, in init restore_signals, start_new_session) File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 1275, in _execute_child restore_signals, start_new_session, preexec_fn) TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/sr_subscribe", line 11, in sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/sarra/sr_subscribe.py", line 2219, in main subscribe.exec_action(action,old) File "/usr/local/lib/python3.6/site-packages/sarra/sr_instances.py", line 287, in exec_action elif action == 'edit' : self.exec_action_on_config(action) File "/usr/local/lib/python3.6/site-packages/sarra/sr_instances.py", line 441, in exec_action_on_config self.run_command([ editor, edit_fil] ) File "/usr/local/lib/python3.6/site-packages/sarra/sr_config.py", line 1195, in run_command except: self.logger.error("trying run command %s " % ' '.join(cmd_list) ) TypeError: sequence item 1: expected str instance, NoneType found


It's because the swob.conf isn't there in the first place:

$ sr_subscribe list swob.conf 2019-09-15 19:18:30,810 [ERROR] file not found swob.conf

Fichiers d'observations en format SWOB

Pour plus d'information sur ce produit, veuillez consulter la documentation a

http://dd.meteo.gc.ca/observations/doc/README_SWOB.txt

broker amqps://dd.weather.gc.ca/

Observation files in SWOB format

For more information about this product, please consult the documentation at

http://dd.weather.gc.ca/observations/doc/README_SWOB.txt

instances:

quantité de processus qui téléchargent simultanément.

instances 2

instances: number of downloaders running. If you see lag in the log, increase.

expire, en usage opérationnel, c´est mieux 1d (1 jour) pour celui-ci. Il faut

que ça soit plus long que l´interru ption le plus long qu´on veut

tolérer sans perte.

expire 10m

expire, in operations, needs to be longer than the longest expected interruption

All stations

subtopic observations.swob-ml.#

Only station CYQL

subtopic observations.swob-ml.*.CYQL.

accept .*