search

sot / chandra_cmd_states

Interface for creation and maintenance of the Chandra commanded states database.
https://sot.github.io/cmd_states
BSD 3-Clause "New" or "Revised" License
3 stars 1 forks source link

Consistent problem exiting from lucky ftp transfer #18

Open taldcroft opened 10 years ago

taldcroft commented 10 years ago 
WARNING - '/proj/sot/ska/share/cmd_states/update_cmd_states.py --dbi 'sybase' --server 'sybase' --h5file /proj/sot/ska/data/cmd_states/cmd_states.h5' returned non-zero status: 256
<SNIP>
get_cmds: got 13 commands from /data/mpcrit1/mplogs/2014/OCT0614/oflsa/CR278_1906.backstop
insert_cmds_db: inserting 13 cmds to commands tables
udpate_states_db: removed HDF5 cmd_states rows from 247660 to 247660
udpate_states_db: DELETE FROM cmd_states WHERE datestart >= '2014:278:19:07:57.848'
udpate_states_db: inserting states[708:1282] to HDF5 cmd_states
udpate_states_db: inserting states[708:1282] to database cmd_states
Connected (version 2.0, client OpenSSH_6.1)
Authentication (password) successful!
Secsh channel 1 opened.
[chan 1] Opened sftp connection (server version 3)
Ska.ftp: log in to lucky as taldcroft
Ska.ftp: cd /home/taldcroft
Ska.ftp: ls .
Ska.ftp: ls cmd_states
Ska.ftp: put cmd_states.h5 as 3abbf1f6-4772-4513-9591-3762aa9e1cf7
Ska.ftp: rename 3abbf1f6-4772-4513-9591-3762aa9e1cf7 cmd_states/cmd_states.h5
Traceback (most recent call last):
  File "/proj/sot/ska/share/cmd_states/update_cmd_states.py", line 9, in <module>
    update_cmd_states.main()
  File "/proj/sot/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/Chandra.cmd_states-0.09-py2.7.egg/Chandra/cmd_states/update_cmd_states.py", line 463, in main
    occweb.ftp_put_to_lucky(ftp_dirname, [opt.h5file], logger=logging)
  File "/proj/sot/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/kadi-0.11.1-py2.7.egg/kadi/occweb.py", line 124, in ftp_put_to_lucky
    ftp.rename(ftp_file, destination_file)
  File "/proj/sot/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/proj/sot/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/Ska.File-0.03-py2.7.egg/Ska/File.py", line 28, in chdir
    os.chdir(curdir)
OSError: [Errno 2] No such file or directory: ''

Closing remaining open files: /proj/sot/ska/data/cmd_states/cmd_states.h5... done
[chan 1] sftp session closed.

The FTP transfer is completing successfully, but the bug is that somehow curdir is an empty string. That is set previously in kadi.occweb.ftp_put_to_lucky as the first part of an absolute local path to the file being transferred. So it doesn't make any sense that curdir can be ''.

This is probably an upstream bug, or something related to our system NFS.

jeanconn commented 10 years ago

This is probably an upstream bug, or something related to our system NFS.

Possibly related to the aca home directory moves? (syshelp mentioned that unagi ended up stuck with it on numa instead of remus and then switched it back) Doesn't quite line up with these errors in time.

jeanconn commented 10 years ago

The FTP transfer is completing successfully, but the bug is that somehow curdir is an empty string. That is set previously in kadi.occweb.ftp_put_to_lucky https://github.com/sot/kadi/blob/0e4260c31ee1aa5eb28798beed9b64ef788d2b4a/kadi/occweb.py#L115 as the first part of an absolute local path to the file being transferred. So it doesn't make any sense that curdir can be ''.

I thought curdir is the CWD before Ska.File does the chdir to file_dir (the first part of the absolute local path to the file).

taldcroft commented 10 years ago

I thought curdir is the CWD before Ska.File does the chdir to file_dir (the first part of the absolute local path to the file).

Duh, of course it is! So get_cwd() is failing, which is more believable.

jeanconn commented 10 years ago

Though I disagree with the "sporadic" in the issue name. It has failed this way every time it has run this week.​