Updated python-based config generator

[brendanburkhart]

Replace existing Matlab GUI config generator with a console-only Python config generator. Supports both XML and JSON output. calParser.py is used to parse the Matlab .cal files, and dvrk-config-generator.py does the actual config generator. Can be invoked as ./dvrk-config-generator.py -a <arm-name> -c <cal-file> -f <JSON|XML>, for example, ./dvrk-config-generator.py -a MTML -c ./jhu-dVRK/cal-files/jhu-mtml-m22723.cal -f XML.

[adeguet1]

I'd like to keep the MATLAB based config generator for at least one minor revision to avoid a rough transition for some users. Can you undo that commit? I should be able to test the config generator in the comming days.

[brendanburkhart]

Good idea, sorry about that.

[adeguet1]

Sorry for the delay reviewing your PR. Config generation worked fine for MTM and PSM, very nice job overall.

• Nice to have:
  • Once the file is generate, provide a message saying something like "Generated config file blah.xml" (or json) *I might move the config generator from share to applications/config-generator to cleanup a bit
• Issues:
  • MTM gripper: Actuator Id should be Id of second board. Tested for MTML and reported 0 instead of 1
  • MTM gripper: BitsToPosSI Scale should be 360.0, not 1.0
  • ECM: tested with cal file share/jhu-daVinci/cal-files/jhu-dv-ecm-29738.cal, got error syntax error. Easy fix for this, boardId should be boardID.
  • ECM: typo in EngagedCurret
  • ECM: In AnalogBrake, EngagedCurrent, ReleaseTime, ReleasedCurrent... should be separated and have a Unit and Value

[adeguet1]

Thank you for the update. There is a few things you can probably fix on your own:

• Add the files pXXXX.cal to the directory jhu-dVRK-S/cal-files
• Set file permissions to 755 for the main python script
• You added time units for consistency. This is a good idea but values are actually in seconds (s), not milliseconds (ms). To note, that c++ parser will just ignore this but since users might edit the configuration file by hand, it would be better to display s so they won't be confused
• Can you change the precision in the XML file. For example, BitToPosSi seems to have only 4 or 5 significant digits and since we have fairly high precision encoders on the new arms, it would be good to have more decimals
• Board and Axis IDs are different on the Si controllers vs Classic controllers. Si have a single board id with 10 axes instead of two boards with 4 axes per board for the Classic. The board Id should be the first one used for the Classic system (i.e. 6 for PSM1, 8 for PSM2 and 10 for PSM3). Axis IDs are 0 to 6 for the motors and 7 to 9 for the brakes. Brake "7" is on axis "0", brake "8" is on axis "1" and brake "9" is on axis "2".

The following issues might be a bit harder to fix since we need to figure out where the numbers should be coming from:.

• BitsToFeedbackAmp, scale and offset don't seem to match the values in the config file we created empirically
• PotToPosSI, the current code sets both the scale and offset to 0. Ideally these values should come from the .cal file (at least the offset, scale might be a constant)

[adeguet1]

I tried to generate a couple of config files for Si PSMs today and ran into some issues. I'm using Ubuntu 20.04, default Python 3 with the command line:

> pwd 
/home/adeguet1/catkin_ws/src/cisst-saw/sawIntuitiveResearchKit/applications/config-generator
> ./dvrk-config-generator.py -a PSM1 -g Si -c ../../share/jhu-dVRK-S/cal-files/jhu-psm-s-334809.cal

The first issue I ran into was related to pad and I applied the following patch (though not sure it's the way to proceed):

         # Zero-pad numeric arrays, None-pad object arrays
         numeric_array = isinstance(value, np.ndarray) and value.dtype != np.dtype(object)
         if numeric_array:
-            return np.pad(data, ((0, padRows), (0, padColumns)))
+           return np.pad(data, ((0, padRows), (0, padColumns)), mode='constant')
         else:
             data = data.astype(object)
-            return np.pad(data, ((0, padRows), (0, padColumns)), constant_values=None)
+           return np.pad(data, ((0, padRows), (0, padColumns)), constant_values=None, mode='constant')

With that "fix", I now get the following error/stack:

Traceback (most recent call last):
  File "./dvrk-config-generator.py", line 1060, in <module>
    generateConfig(args.cal, args.arm, args.generation, outputFormat)
  File "./dvrk-config-generator.py", line 1008, in generateConfig
    calData = parser.parseFile(calFileName)
  File "/home/adeguet1/catkin_ws/src/cisst-saw/sawIntuitiveResearchKit/applications/config-generator/calParser.py", line 386, in parseFile
    assignmentPath.assignTo(data, value)
  File "/home/adeguet1/catkin_ws/src/cisst-saw/sawIntuitiveResearchKit/applications/config-generator/calParser.py", line 187, in assignTo
    self.parent.assignTo(variables, data)
  File "/home/adeguet1/catkin_ws/src/cisst-saw/sawIntuitiveResearchKit/applications/config-generator/calParser.py", line 220, in assignTo
    data = self._padToFit(data, value)
  File "/home/adeguet1/catkin_ws/src/cisst-saw/sawIntuitiveResearchKit/applications/config-generator/calParser.py", line 263, in _padToFit
    return np.pad(data, ((0, padRows), (0, padColumns)), constant_values=None, mode='constant')
  File "/usr/lib/python3/dist-packages/numpy/lib/arraypad.py", line 1371, in pad
    newmat = _append_const(newmat, pad_after, after_val, axis)
  File "/usr/lib/python3/dist-packages/numpy/lib/arraypad.py", line 142, in _append_const
    (arr, (np.zeros(padshape) + val).astype(arr.dtype)), axis=axis)
TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'

Thoughts?

[brendanburkhart]

It looks like versions of NumPy older than v1.14 don't let you pad an array with None. I tested the fix with NumPy 1.13 and Python 3.6.9, let me know if it works for you.