Code not working - Githubissues

farinexile commented 4 years ago

Am I missing something? This is my code and error:

import paramiko import amqp import sarra

blacklab% sr_subscribe list

error: File "", line 1 blacklab% sr_subscribe list ^ SyntaxError: invalid syntax

t

Thanks.

petersilva commented 4 years ago

that is cool! but yes the problem is that you are trying to call it as python directly. Sarracenia is written in python, but it isn't a python library. You need a bash shell environment somehow. blacklab% is the bash prompt on my laptop, (named blacklab)*

Take a look at: https://github.com/MetPX/sarracenia/blob/master/doc/Install.rst

For now, it is a command-line tool. Do you think having it work as a callable library woud be a good thing?

petersilva commented 4 years ago

I wrote bash, but bash would be the shell on Linux, it can be used in a variety of shells on different operating system. the thing is it is something run in a terminal window... I don't know what operating system you are using.

farinexile commented 4 years ago

Thanks Peter, I have followed installation instructions and then tried again: $ blacklab% git tag but getting error: blacklab%: command not found

petersilva commented 4 years ago

You do not seem familiar with conventions for command line work, and may want to find someone with more command line experience to help you with the instructions. but we will try a bit more, and see if we can get you going. Firstly, you seem to have pasted blacklab% git tag into your terminal window. You need to know that blacklab% is the prompt (the text which is to the left of any command typed.) The prompt should not be included in any text you are pasting. In your window, your prompt is farniexile@FarinaLaptop:~/sarracenia$. If you see the example instruction: blacklab% git tag you should only copy/paste the git tag part.

I guess you are trying to install from source? It looks like a linux terminal screen. OK so far. (for this issue, I am omitting the prompts so you can copy/paste more easily) so the next two commands would be:

python3 setup.py bdist_wheel
pip3 install dist/metpx_sarracenia-*.whl

After that you can enter::

   sr_subscribe list

and what you see on the screen should be something like:


blacklab% sr_subscribe list

configuration examples: ( /home/peter/src/sarracenia/sarra/examples/subscribe )
WMO_Sketch_2mqtt.conf  WMO_Sketch_2v3.conf    WMO_mesh_CMC.conf   WMO_mesh_Peer.conf      aws-nexrad.conf 
       dd_2mqtt.conf          dd_all.conf         dd_amis.conf         dd_aqhi.conf dd_cacn_bulletins.conf 
    dd_citypage.conf         dd_cmml.conf         dd_gdps.conf         dd_ping.conf        dd_radar.conf 
        dd_rdps.conf         dd_swob.conf     ddc_cap-xml.conf      ddc_normal.conf   downloademail.conf 
     ec_ninjo-a.conf       hpfx_amis.conf       local_sub.conf    pitcher_pull.conf          sci2ec.conf 
        subnoaa.conf      subsoapshc.conf         subusgs.conf 

general: ( /home/peter/.config/sarra )
          admin.conf     credentials.conf         default.conf 

user configurations: ( /home/peter/.config/sarra/subscribe )
       amqp_f30.conf      cclean_f91.conf       cdnld_f21.conf       cfile_f44.conf          cp_f61.conf 
        ftp_f70.conf           q_f71.conf  rabbitmqtt_f31.conf      u_sftp_f60.conf 

logs are in: /home/peter/.cache/sarra/log

blacklab%

(where blacklab% would be replaced by farniexile@FarinaLaptop:~/sarracenia$ )

Try that, and we'll see how it goes...

farinexile commented 4 years ago

Thanks Peter, I have got this far.

petersilva commented 4 years ago

OK, What sort of data are you interested in? If you just want to get a continuous stream to sort of kick the tires, then I would suggest hpfx_amis.conf as an example. You could do: ` sr_subscribe add hpfx_amis.conf

`

now it might not be usable right out of the box. You will need to edit the file to change the directory setting to where you want to put stuff on your laptop. the file to edit is: ~/.config/sarra/subscribe/hpfx_amis.conf you change the path listed as the directory setting. I would use vi for this sort of thing, but you probably want a friendlier editor, perhaps nano? nano ~/.config/sarra/subscribe/hpfx_amis.conf or... if your choice of editor is configured in your terminal (something like export EDITOR=nano ) then you can use sr_subscribe edit hpfx_amis.conf Once it is set to your liking, you can then start it up in foreground mode:

sr_subscribe foreground hpfx_amis

and it will start copying files, likely several per second. In foreground mode, you can use Control-C to stop the downloads.

So that's a first example.

petersilva commented 4 years ago

I just played with the example, it has some debug stuff in it. I am fixing it it git, so you might not need these changes, but just in case:

When you go into the configuration file you should also make the following changes:

on the line with broker, remove anonymous ( this seems to be a bug, it will guess anonymous if anoymous isn't written, but it if is written, more configuration is needed.)
either comment out the on_message line by putting a # at the beginning, or delete it... it's debug stuff.

farinexile commented 4 years ago

Hi Peter, I am running ubuntu as a windows subsystem. Have copied the config files in subscribe folder for alert and citypage. But getting the following errors:


farinexile@FarhanaLaptop:~$ sr_subscribe status
2020-02-18 12:11:59,777 [INFO] sr_subscribe amqp_cap-xml 01 instance missing (pid=477)
2020-02-18 12:12:00,692 [INFO] sr_subscribe amqp_citypage 01 instance missing (pid=526)
2020-02-18 12:12:00,694 [INFO] sr_subscribe amqp_citypage 02 instance missing (pid=527)
farinexile@FarhanaLaptop:~$ sudo sr_subscribe status amqp_citypage
2020-02-18 12:12:12,478 [INFO] sr_subscribe amqp_citypage 01 instance missing (pid=526)
2020-02-18 12:12:12,480 [INFO] sr_subscribe amqp_citypage 02 instance missing (pid=527)
farinexile@FarhanaLaptop:~$

How do I resolve this?

petersilva commented 4 years ago

sr_subscribe status is telling you that you haven´t started it::

sr_subscribe foreground amqp_citypage

Will start it...

farinexile commented 4 years ago

Thanks Peter, when I used the code, received the following error:

farinexile@FarhanaLaptop:~$ sr_subscribe foreground amqp_citypage 2020-02-18 13:39:40,965 [INFO] sr_subscribe amqp_citypage start 2020-02-18 13:39:40,966 [INFO] log settings start for sr_subscribe (version: 2.20.02b1): 2020-02-18 13:39:40,966 [INFO] inflight=.tmp events=create|delete|link|modify use_pika=False topic_prefix=v02.post 2020-02-18 13:39:40,966 [INFO] inflight=.tmp events=create|delete|link|modify use_amqplib=False topic_prefix=v02.post 2020-02-18 13:39:40,967 [INFO] suppress_duplicates=False basis=path retry_mode=True retry_ttl=600000ms tls_rigour=normal 2020-02-18 13:39:40,967 [INFO] expire=600000ms reset=False message_ttl=None prefetch=25 accept_unmatch=False delete=False poll_without_vip=True 2020-02-18 13:39:40,967 [INFO] heartbeat=300 sanity_log_dead=450 default_mode=000 default_mode_dir=775 default_mode_log=600 discard=False durable=True 2020-02-18 13:39:40,968 [INFO] post_on_start=True preserve_mode=True preserve_time=True realpath_post=False base_dir=None follow_symlinks=False 2020-02-18 13:39:40,968 [INFO] mirror=True flatten=/ realpath_post=False strip=0 base_dir=None report_back=True log_reject=False 2020-02-18 13:39:40,968 [INFO] Plugins configured: 2020-02-18 13:39:40,969 [INFO] do_download: 2020-02-18 13:39:40,969 [INFO] do_get : 2020-02-18 13:39:40,969 [INFO] on_message: 2020-02-18 13:39:40,970 [INFO] on_data: 2020-02-18 13:39:40,970 [INFO] on_part: 2020-02-18 13:39:40,970 [INFO] on_file: File_Log 2020-02-18 13:39:40,971 [INFO] on_post: Post_Log 2020-02-18 13:39:40,971 [INFO] on_heartbeat: Hb_Log Hb_Memory Hb_Pulse RETRY 2020-02-18 13:39:40,971 [INFO] on_report: 2020-02-18 13:39:40,972 [INFO] on_start: 2020-02-18 13:39:40,972 [INFO] on_stop: 2020-02-18 13:39:40,972 [INFO] log_settings end. 2020-02-18 13:39:40,973 [INFO] sr_subscribe run 2020-02-18 13:39:40,975 [INFO] AMQP broker(dd.weather.gc.ca) user(anonymous) vhost() 2020-02-18 13:39:40,975 [INFO] Using amqp module (AMQP 0-9-1) 2020-02-18 13:39:41,074 [ERROR] Unexpected error: [Errno 92] Protocol not available /usr/lib/python3/dist-packages/amqp/connection.py:292: AMQPDeprecationWarning: The .frame_writer attribute on the connection was accessed before the connection was established. This is supported for now, but will be deprecated in amqp 2.2.0.

Since amqp 2.0 you have to explicitly call Connection.connect() before using the connection.

W_FORCE_CONNECT.format(attr=attr))) Traceback (most recent call last): File "/usr/lib/python3/dist-packages/amqp/connection.py", line 459, in channel return self.channels[channel_id] KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/sr_subscribe", line 11, in load_entry_point('metpx-sarracenia==2.20.2b1', 'console_scripts', 'sr_subscribe')() File "/usr/lib/python3/dist-packages/sarra/sr_subscribe.py", line 2199, in main subscribe.exec_action(action,old) File "/usr/lib/python3/dist-packages/sarra/sr_instances.py", line 298, in exec_action if action == 'foreground' : self.foreground_parent() File "/usr/lib/python3/dist-packages/sarra/sr_instances.py", line 551, in foreground_parent self.start() File "/usr/lib/python3/dist-packages/sarra/sr_subscribe.py", line 1929, in start self.run() File "/usr/lib/python3/dist-packages/sarra/sr_subscribe.py", line 1699, in run self.connect() File "/usr/lib/python3/dist-packages/sarra/sr_subscribe.py", line 212, in connect self.consumer = sr_consumer(self) File "/usr/lib/python3/dist-packages/sarra/sr_consumer.py", line 80, in init self.build_consumer() File "/usr/lib/python3/dist-packages/sarra/sr_consumer.py", line 104, in build_consumer self.consumer.build() File "/usr/lib/python3/dist-packages/sarra/sr_amqp.py", line 276, in build self.channel = self.hc.new_channel() File "/usr/lib/python3/dist-packages/sarra/sr_amqp.py", line 198, in new_channel channel = self.connection.channel() File "/usr/lib/python3/dist-packages/amqp/connection.py", line 462, in channel channel.open() File "/usr/lib/python3/dist-packages/amqp/channel.py", line 434, in open spec.Channel.Open, 's', ('',), wait=spec.Channel.OpenOk, File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 50, in send_method conn.frame_writer(1, self.channel_id, sig, args, content) File "/usr/lib/python3/dist-packages/amqp/connection.py", line 320, in frame_writer self.connect() File "/usr/lib/python3/dist-packages/amqp/connection.py", line 282, in connect self.transport.connect() File "/usr/lib/python3/dist-packages/amqp/transport.py", line 111, in connect self.socket_settings, self.read_timeout, self.write_timeout, File "/usr/lib/python3/dist-packages/amqp/transport.py", line 163, in _init_socket self._set_socket_options(socket_settings) File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _set_socket_options tcp_opts = self._get_tcp_socket_defaults(self.sock) File "/usr/lib/python3/dist-packages/amqp/transport.py", line 183, in _get_tcp_socket_defaults opt: sock.getsockopt(SOL_TCP, opt) for opt in TCP_OPTS File "/usr/lib/python3/dist-packages/amqp/transport.py", line 183, in opt: sock.getsockopt(SOL_TCP, opt) for opt in TCP_OPTS OSError: [Errno 92] Protocol not available farinexile@FarhanaLaptop:~$

farinexile commented 4 years ago

Also if it helps understand the issue: My platform is Linux2 according to sys.platform

petersilva commented 4 years ago

so... I'm not sure why you picked the source version. It requires to most work to get going. You seem to have a missing dependency.

There are also pre-built binary packages for ubuntu. Do you want to hack on the source, or is a pre-build binary ok? To get a working binary, you would do something like:

Please look at the first option here:

https://github.com/MetPX/sarracenia/blob/master/doc/Install.rst

Install the ubuntu packages. It's about four lines (skip the blacklab% )

That will have the happy side-effect of getting the right libraries installed for your source version also. so then you will have two versions working.

petersilva commented 4 years ago

the other thing that might be an issue... ~/.config/sarra/credentials.conf you might need an entry for the anonymous user in the credentials file. the following line should do that:

echo amqps://anonymous:anonymous@hpfx.collab.science.gc.ca >>~/.config/sarra/credentials.conf

farinexile commented 4 years ago

Hi Peter, I used the following codes to install in the beginning : sudo add-apt-repository ppa:ssc-hpc-chp-spc/metpx sudo apt-get update sudo apt-get install python3-paramiko # required SFTP support. sudo apt-get install metpx-sarracenia # only supports HTTP/HTTPS sudo apt-get install sarrac # optional C client.

I uninstalled all of them using sudo, then reinstalled with pip and now it is working! thanks a lot.

petersilva commented 4 years ago

Great! Glad that you got it working! I'm not sure what you are doing, but just so you know, there is also a windows version. one click self-extracting windows binary. You don't need linux if you don't want it.

Good luck!

MetPX / sarracenia

Code not working #307