Setting trajectory group

[hududed]

Hi @newville

I am trying to set a simple line trajectory.

I have the axes set as such:

xps = NewportXPS(hostname)
# XPS host:         192.168.254.254 (XPS-1B81)
# Firmware:         XPS-D-N13019
# Current Time:     Wed Aug  5 14:52:22 2020
# Last Reboot:      Sun Aug  2 17:27:49 2020
# Trajectory Group: None
# Groups and Stages
Z (singleaxisinuse), Status: Not referenced state
   Z.Z-MFA (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - ZM low level
      Positioner Errors: OK
XY (xyinuse), Status: Not referenced state
   XY.X-MFA (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - ZM low level
      Positioner Errors: OK
   XY.Y-MFA (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - ZM low level
      Positioner Errors: OK

If I try to define the line trajectory for group Z:

>>>xps.define_line_trajectories("Z")
Must define a trajectory group first!

If I set the trajectory group I get the error:

>>>xps.set_trajectory_group("Z")
XPSException: 'OrderedDict([('category', 'xyinuse'), ('positioners', ['X-MFA', 'Y-MFA'])])' cannot be a trajectory group, must be one of 

How do I properly set the trajectory group?

[newville]

@hududed It looks to me like you have an XYGroup defined but not a MultipleAxes group defined.

Currently, this newportxps module doesn't support trajectory scans with XYGroups, but only supports PVT trajectories with MultipleAxes groups. I have used XYGroups in the distant past, and recall that there were some subtle differences, but I don't recall the details. So, in principle, support for this this could be added, I just have not needed that and don't have one of those set up at the moment.

The simplest thing would be to switch from XYGroup to MultipleAxesGroup. In fact, it looks like you want to do a trajectory with your 'Z-MFA' stage, which is a single-axis group and cannot (I believe) do trajectories. [aside: with one of the XPSes I use for trajectory scans, almost all the trajectories are really along one axis at a time, but it might be one of two or 3 axes, and on rare occasions, I do diagonal scans. With other XPSes, we really do more complex trajectories where only PVT makes sense]. So, perhaps you want to switch to a system.ini that puts any axis that you will want to do a trajectory with into a MultiAxes group, something like this (totally writing this on the fly!!):

[GROUPS]
SingleAxisInUse = 
SpindleInUse =
XYInUse =
XYZInUse =
MultipleAxesInUse = XYZ

[XYZ]
PositionerNumber = 3
PositionerInUse = X-MFA, Y-MFA, Z-MFA
InitializationAndHomeSearchSequence = Together

[XYZ.X-MFA]
PlugNumber = 1
StageName = MFA@MFA-CC@XPS-DRV11
TimeFlasherBaseFrequency = 40e6  ; Hz

[XYZ.Y-MFA]
PlugNumber = 2
StageName = MFA@MFA-CC@XPS-DRV11
TimeFlasherBaseFrequency = 40e6  ; Hz

[XYZ.Z-MFA]
PlugNumber = 3
StageName = MFA@MFA-CC@XPS-DRV11
TimeFlasherBaseFrequency = 40e6  ; Hz

with that, you could define complex trajectories in x, y, z, or just scan along one axis. Does that make sense and seem like a reasonable approach?

Also: you will have to home and initialize the groups before you can run a trajectory ;).

FWIW, if you're committed to using the XYGroup approach, I don't think it would be that hard to alter this code to support that.

[hududed]

@newville the XYZ multiple axes works thank you.

# Current Time:     Thu Aug  6 09:55:24 2020
# Last Reboot:      Thu Aug  6 09:47:49 2020
# Trajectory Group: XYZ
# Groups and Stages
XYZ (multipleaxesinuse), Status: Not referenced state
   XYZ.X (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - ZM low level
      Positioner Errors: OK
   XYZ.Y (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - ZM low level
      Positioner Errors: OK
   XYZ.Z (MFA@MFA-CC@XPS-DRV11)
      Hardware Status: First driver powered on - Minus end of run activated - ZM high level
      Positioner Errors: OK

Now I get a file not found error when defining the trajectory:

>>>xps.initialize_allgroups()
>>>xps.set_trajectory_group('XYZ')
>>>xps.define_line_trajectories('Z')
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
~\Anaconda3\envs\controller2\lib\site-packages\newportxps-0.1-py3.7.egg\newportxps\newportxps.py in define_line_trajectories(self, axis, group, start, stop, step, scantime, accel, upload, verbose)
    740                 self.upload_trajectory('forwardx.trj',
--> 741                                        self.linear_template % fore)
    742                 self.upload_trajectory('backwardx.trj',

~\Anaconda3\envs\controller2\lib\site-packages\newportxps-0.1-py3.7.egg\newportxps\newportxps.py in upload_trajectory(self, filename, text)
    236         self.ftpconn.connect(**self.ftpargs)
--> 237         self.ftpconn.cwd(os.path.join(self.ftphome, 'Public', 'Trajectories'))
    238         self.ftpconn.put(text, filename)

~\Anaconda3\envs\controller2\lib\site-packages\newportxps-0.1-py3.7.egg\newportxps\ftp_wrapper.py in cwd(self, remotedir)
     41     def cwd(self, remotedir):
---> 42         self._conn.cwd(remotedir)
     43 

~\Anaconda3\envs\controller2\lib\site-packages\pysftp-0.2.9-py3.7.egg\pysftp\__init__.py in chdir(self, remotepath)
    523         self._sftp_connect()
--> 524         self._sftp.chdir(remotepath)
    525 

~\Anaconda3\envs\controller2\lib\site-packages\paramiko\sftp_client.py in chdir(self, path)
    658             return
--> 659         if not stat.S_ISDIR(self.stat(path).st_mode):
    660             code = errno.ENOTDIR

~\Anaconda3\envs\controller2\lib\site-packages\paramiko\sftp_client.py in stat(self, path)
    492         self._log(DEBUG, "stat({!r})".format(path))
--> 493         t, msg = self._request(CMD_STAT, path)
    494         if t != CMD_ATTRS:

~\Anaconda3\envs\controller2\lib\site-packages\paramiko\sftp_client.py in _request(self, t, *arg)
    812         num = self._async_request(type(None), t, *arg)
--> 813         return self._read_response(num)
    814 

~\Anaconda3\envs\controller2\lib\site-packages\paramiko\sftp_client.py in _read_response(self, waitfor)
    864                 if t == CMD_STATUS:
--> 865                     self._convert_status(msg)
    866                 return t, msg

~\Anaconda3\envs\controller2\lib\site-packages\paramiko\sftp_client.py in _convert_status(self, msg)
    893         elif code == SFTP_NO_SUCH_FILE:
    893             # clever idea from john a. meinel: map the error codes to errno
--> 894             raise IOError(errno.ENOENT, text)
    895         elif code == SFTP_PERMISSION_DENIED:

FileNotFoundError: [Errno 2] No such file

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-13-4653e1efeb94> in <module>
----> 1 xps.define_line_trajectories('Z')

~\Anaconda3\envs\controller2\lib\site-packages\newportxps-0.1-py3.7.egg\newportxps\newportxps.py in wrapper(self, *args, **kwargs)
     24         if self._sid is None or len(self.groups) < 1 or len(self.stages) < 1:
     25             self.connect()
---> 26         return fcn(self, *args, **kwargs)
     27     wrapper.__doc__ = fcn.__doc__
     28     wrapper.__name__ = fcn.__name__

~\Anaconda3\envs\controller2\lib\site-packages\newportxps-0.1-py3.7.egg\newportxps\newportxps.py in define_line_trajectories(self, axis, group, start, stop, step, scantime, accel, upload, verbose)
    744                 ret = True
    745             except:
--> 746                 raise ValueError("error uploading trajectory")
    747         return ret
    748 

ValueError: error uploading trajectory

So there are foreward and backward .trj files preloaded on my system forwardx.trj backwardx.trj so I naively changed the filenames in newportxps.py but still got this error. Any ideas how to solve?

[newville]

@hududed That traceback says the problem is when the (S)FTP instance tries to set the current working directory to

 os.path.join(self.ftphome, 'Public', 'Trajectories')

so you might want to investigate that. Can you manually use the sftp CLI program to get to your controller:

~> sftp Administrator@YOUR_XPS_IPADDRESS

From there, you should be able to do cd Public/Trajectories. That is really all the python code is doing here.

It still seems to me like something is not set up correctly with sftp between your workstation and XPS controller.

[hududed]

@newville I can confirm the data are there

>>>sftp Administrator@192.168.254.254
Administrator@192.168.254.254's password:
Connected to Administrator@192.168.254.254.
sftp> ls
Config                        DefaultConfig                 DefaultUserOptionalModules    Firmware                                                                                                                                              Public                        UserOptionalModules
sftp> cd Public
sftp> ls
CoreDump       Gathering      Log            Scripts        Trajectories   WebsiteConfig  toolchain
sftp> cd Trajectories
sftp> ls
backwardx.trj  forwardx.trj    

[newville]

Um, why do the filenames have "x" in them?

[hududed]

They are preloaded that way. Do you have a simple working trajectory file example that I can test with the code?

[newville]

The files 'forewardx.trj' and 'backwardx.trj' are not uploaded by this module.

The files uploaded here should be named 'foreward.trj' and 'backward.trj'. I suspect that you are not able to write to this Trajectories folder, though I don't know why that would be the case.

A simple working trajectory file for a 3-axis group would be

0.010000,0.0020,0.2000,0,0,0,0
1.010000,0.2020,0.2000,0,0,0,0
0.010000,0.0020,0.000,0,0,0,0

A simple script to define and run a trajectory might be

from newportxps import NewportXPS
IPADDR = '192.168.254.100'

xps = NewportXPS(IPADDR)
xps.define_line_trajectories(start=0.0, stop=0.2, step=0.002, scantime=1.0, axis='X')
xps.run_trajectory('foreward')

But, again, I think the problem you were having was with sftp.

[hududed]

@newville So, if I understand this correctly.

xps.define_line_trajectories(start=0.0, stop=0.2, step=0.002, scantime=1.0, axis='X')
xps.run_trajectory('foreward')

basically writes 0.000000,0.0020,0.2000,0,0,0,0 in the foreward.trj file to Public/Trajectories.

I used the sftp directly in the notebook as how I have it modified in ftp_wrapper.py and check if I can write a text file in the XPS:

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
sftp = pysftp.Connection(host=xpsd_remoteip,
                         username='Administrator',
                         password='Administrator',
                         cnopts=cnopts
                         )
>>>sftp.pwd
'/Admin/Public//Trajectories//'
>>>sftp.put('./test.txt')
<SFTPAttributes: [ size=69 uid=101 gid=101 mode=0o100644 atime=1596776149 mtime=1596776149 ]>
>>>sftp.listdir()
['backwardx.trj', 'forwardx.trj', 'test.txt']

I also double-checked to see I have full read/write permissions and that the text file is there through the command line:

[Administrator@XPS-1b81 ~/Public/Trajectories]$ ls -ld ./
drwxrwxrwx  2 root      root           4096 Aug 07 06:30 ./
[Administrator@XPS-1b81 ~/Public/Trajectories]$ ls
backwardx.trj      forwardx.trj      test.txt

It seems I have the permissions fine. The upload is written through linear_template in newportxps.py. Should the second line at 378 be there?

377        self.linear_template = '\n'.join(['', trajline1, trajline2, trajline3])
388        self.linear_template = '\n'.join(['', trajline1, trajline2, trajline3,''])

[hududed]

So I think the problem is the backlash in Windows. Since I am running on windows I replaced the\\ with / throughout newportxps.py e.g.:

    def upload_trajectory(self, filename,  text):
        """upload text of trajectory file

        Arguments:
        ----------
           filename (str):  name of trajectory file
           text  (str):   full text of trajectory file
        """
        self.ftpconn.connect(**self.ftpargs)
        self.ftpconn.cwd(os.path.join(self.ftphome, 'Public', 'Trajectories').replace("\\", "/"))

So now if I run:

>>>xps.set_trajectory_group('XYZ')
>>>xps.define_line_trajectories(start=0.0, stop=0.5, step=0.01, scantime=1.0, axis='X')
True
>>>xps.trajectories
{'foreward': {'start': [-0.155],
  'stop': [0.655],
  'axes': ['X'],
  'pixeltime': 0.02,
  'npulses': 51,
  'nsegments': 3},
 'backward': {'start': [0.655],
  'stop': [-0.155],
  'axes': ['X'],
  'pixeltime': 0.02,
  'npulses': 51,
  'nsegments': 3}}
>>>xps.arm_trajectory('foreward')
>>>xps.run_trajectory('foreward')
XPSError: message= PVT Execute, error=-22, description=Not allowed action
Had to do repeat XPS Gathering:  0 0 500000
Had to do repeat XPS Gathering:  0 0 500000
Had to do repeat XPS Gathering:  0 0 500000
Had to do repeat XPS Gathering:  0 0 500000
Had to do repeat XPS Gathering:  0 0 500000
looks like something is wrong with the XPS!

Am I missing something?

[newville]

@hududed

It seems I have the permissions fine.

Hm, OK not sure... it was failing in Python when setting the working directory in sftp.

The upload is written through linear_template in newportxps.py. Should the second line at 378 be there?

377 self.linear_template = '\n'.join(['', trajline1, trajline2, trajline3]) 378 self.linear_template = '\n'.join(['', trajline1, trajline2, trajline3,''])

Oh, good catch - it should be the second version to ensure that there is a trailing new line.

So I think the problem is the backlash in Windows. Since I am running on windows I replaced the\ with / throughout newportxps.py e.g.:

Ah, that makes more sense. Bitten by "os.path.join()" again (well, and the damn Windows backslash!).

Right, the path is on the sftp server, not on the client-side. So we should fix it the code to force forward slashes, as with your replace().

xps.trajectories {'foreward': {'start': [-0.155], 'stop': [0.655], 'axes': ['X'], 'pixeltime': 0.02, 'npulses': 51, 'nsegments': 3}, 'backward': {'start': [0.655], 'stop': [-0.155], 'axes': ['X'], 'pixeltime': 0.02, 'npulses': 51, 'nsegments': 3}} ps.arm_trajectory('foreward') xps.run_trajectory('foreward') XPSError: message= PVT Execute, error=-22, description=Not allowed a

Not sure. Is your XYZ.X stage initialized, homed, and working? Are those valid ranges and speeds? If I recall correctly, these were MFA stages - which ought to be able to do 2 mm/sec. Maybe the controller web page gives some more hints and help decode that error number.

[hududed]

@newville pardon my ignorance. I didn't home the stages. It's working now!

[newville]

@hududed Great! Glad to hear it's working. OK, so I think the main action items here are:

a) fix the path joining to force forward slashes. b) better error messages and/or handling of sftp problems.

Does that seem right and complete to you? Any more suggestions on making this more user friendly (like, maybe more informative error messages)?

[hududed]

Sounds about right. Maybe extend an example with just necessary steps to make it run. So far for me something like this:

from newportxps import NewportXPS
IPADDR = '192.168.254.100' # Update remote/host address here

xps = NewportXPS(IPADDR)
xps.initialize_allgroups()
xps.home_allgroups()
print(xps.status_report())
xps.define_line_trajectories(start=0.0, stop=0.2, step=0.002, scantime=1.0, axis='X')
xps.trajectories
xps.run_trajectory('foreward')

Thanks again and feel free to close this comment.

[newville]

@hududed OK, I think these are now fixed in v 0.2