Closed mkounkel closed 4 years ago
I have the same problem!
Thanks @mkounkel and @deleenm ! I am hoping to address this later today.
Could you tell me what version of jupyter you are using and if you are using jupyter notebook or jupyter-lab? Thanks!
I am using Jupyter_Hub server 1.0.0 to serve a standard jupyter notebook. With the following conda environmnet for the ipython kernel:
ipykernel 5.3.4 py38h23f93f0_0 conda-forge ipython 7.18.1 py38h1cdfbd6_0 conda-forge
And this conda environment for the jupyter_hub server itself:
jupyter 1.0.0 py37_7
jupyter_client 5.3.4 py37_0
jupyter_console 6.1.0 py_0
jupyter_core 4.6.1 py37_0
jupyterhub 1.0.0 py37_0
jupyterlab 1.2.6 pyhf63ae98_0
jupyterlab_server 1.0.6 py_0
Thanks! Very helpful!
On Thu, Oct 1, 2020, 4:39 PM Nathan De Lee notifications@github.com wrote:
I am using Jupyter_Hub server 1.0.0 to serve a standard jupyter notebook. With the following conda environmnet for the ipython kernel:
ipykernel 5.3.4 py38h23f93f0_0 conda-forge ipython 7.18.1 py38h1cdfbd6_0 conda-forge
And this conda environment for the jupyter_hub server itself: jupyter 1.0.0 py37_7 jupyter_client 5.3.4 py37_0 jupyter_console 6.1.0 py_0 jupyter_core 4.6.1 py37_0 jupyterhub 1.0.0 py37_0 jupyterlab 1.2.6 pyhf63ae98_0 jupyterlab_server 1.0.6 py_0
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jcbird/ppv/issues/1#issuecomment-702412369, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRADE7VFDIVI3QZX74YUPTSITZI5ANCNFSM4R2QBA2Q .
Similar on my end, running jupyter notebook, not lab. Python 3.7, jupyter 1.0, jupyter-core 4.5.0, jupyter-client 5.3.3, jupyter-console 6.0.0, ipython 7.8
This may be a useful way to get passwords in jupyter notebooks: import getpass password = getpass.getpass()
Once you have the password in a variable, you can feed it to rsync etc. Also, you should only need to do this once per kernel session with the notebook, since you can keep the password in memory.
Thanks all. @deleenm appreciate the suggestion. Unfortunately, rsync won't accept passwords on the command line. It can use ssh to do the authentication, but Utah won't allow identity keys (BOOOOO). I will get a separate script up today that you can run in a terminal just once before an analysis session. The script will update the plateplans.par file and ALL available plugholes files (in case those were revised).
Going forward, I am going to incorporate https://github.com/sdss/sdss_access from SDSS . That way, you will only need a .netrc file and will never have to input your password again.
Another option would be to use Popen and feed the password into rsync via sshpass. Here is a website with a possible version of that: https://stackoverflow.com/questions/21379185/python-subprocess-rsync-using-sshpass-and-specify-port
@jcbird Here is maybe an even easier option using an environmental variable or password file to run rsync: https://stackoverflow.com/questions/13306842/how-to-parse-server-password-with-subprocess-popen-in-django/13309144
Thanks! I had already been playing with the shell=True option. Your last link looks good. I don't LOVE it, and I think i will try to incorporate sdss_access soon, but this will work for now. Stay tuned!
That was my feeling as well. I don't like that you basically have to write the password in plain text to either an environment variable or file, but it is good enough for the moment. I can't believe that rsync won't just let you pass it a password on the command line. It feels like one of those security "features" that forces you to do less secure things.
Thanks for all the feedback @deleenm and @mkounkel !
I am using the pexpect package now. This is the default package that ipython uses for this sort of thing. SO all should be well now.
Check out the new ppv.ppv.update_platefiles() command (see ipynb tutorial and README) . It will grab ALL the latest from Utah in one go too!
Please comment back here after trying the latest commit so I can close the issue. Thanks!
Looks like it worked for me!
I'm going to close this one. @mkounkel , let me know if you have any problems.
Sorry, I think the update broke it for me completely. Now I can get the password prompt, but it just never does anything after it, either in jupyter or in the terminal. I can run the rsync command it shows outside of python environment, but not within it for whatever reason
I was able to run both tutorials without issue. The rsync definitely ran for me. I even deleted all of the files in my plates directory, and the notebook repopulated them. Note I did have to enter my password twice once for: import ppv and once for: ppv.ppv.update_platefiles()
I've deleted the files, and it's stuck for me on import ppv. It's been trying to run for over 10 minutes now, I assume it shouldn't take this long. When I had older files, that part worked fine without a password prompt, but the moment it did prompt for a password, it got stuck again.
I guess I should post an error log - after entering the password, and then interrupting, this is what it spits out:
An error occurred. /Users/mkounkel/research/cluster/targeting/ppv/plate/platePlans_sdss5.fits does not exist
AND an error occured when trying to execute either
util.download.plate_plans OR _parse_plate_plans
FileNotFoundError Traceback (most recent call last)
There is one bug of minor annoyance for rsync - if I run it from the terminal, it's fine, because it waits for Utah password. Jupyter notebook, on the other hand, skips entering the password, so it crashes when trying to run maybe_observed = planet_targets.available_in(plate_run_2020_08_c)