geom ops: handle four-element 'points' in feature geometry list

[caver456]

Apparently, each element of a feature's geometry list (each 'point') can be either a two-element list or a four-element list:

[lon,lat,elevation(meters),timestamp]

or

[lon,lat]

Apptracks (and maybe GPSIO tracks) use the four-element variety. Drawing features from the web page creates two-element 'points'.

Regardless, crop dies on the four-element variety. Apparently we have not tried to crop the four-element variety before - seems odd but believable.

10:00:26 [sartopo_python:1893:INFO] c1
10:00:26 [sartopo_python:621:INFO] refresh requested for map P1H: 1038ms since last completed sync; shorter than syncInterval; forceImmediate not specified: not syncing now
10:00:26 [sartopo_python:1909:INFO] c2
10:00:26 [sartopo_python:1918:INFO] tgc before:[[-120.91254159, 39.31020279, 1196, 1636773757000], [-120.91254572, 39.31019563, 1196, 1636773758000], [-120.91255035, 39.31017004, 1200, 1636773763000], [-120.91256382, 39.31016305, 1199, 1636773768000], [-120.91256467, 39.31016243, 1199, 1636773773000], [-120.9125648, 39.3101624, 1199, 1636773778000], [-120.9125649, 39.31016221, 1198, 1636773783000], [-120.91256548, 39.31016168, 1198, 1636773788000], [-120.91256656, 39.31016063, 1198, 1636773793000], [-120.91256717, 39.31015965, 1198, 1636773798000], [-120.91256727, 39.31015913, 1198, 1636773803000], [-120.91256696, 39.31015885, 1198, 1636773808000], [-120.9125664, 39.31015875, 1198, 1636773813000], [-120.91256607, 39.31015885, 1198, 1636773818000], [-120.91256589, 39.31015889, 1198, 1636773823000], [-120.91256633, 39.31015836, 1198, 1636773828000], [-120.91256677, 39.31015774, 1199, 1636773833000], [-120.91256692, 39.3101576, 1199, 1636773838000]]
10:00:26 [sartopo_python:1920:INFO] tgc after:[[-120.91254159, 39.31020279, 1196, 1636773757000], [-120.91254572, 39.31019563, 1196, 1636773758000], [-120.91255035, 39.31017004, 1200, 1636773763000], [-120.91256382, 39.31016305, 1199, 1636773768000], [-120.91256467, 39.31016243, 1199, 1636773773000], [-120.9125648, 39.3101624, 1199, 1636773778000], [-120.9125649, 39.31016221, 1198, 1636773783000], [-120.91256548, 39.31016168, 1198, 1636773788000], [-120.91256656, 39.31016063, 1198, 1636773793000], [-120.91256717, 39.31015965, 1198, 1636773798000], [-120.91256727, 39.31015913, 1198, 1636773803000], [-120.91256696, 39.31015885, 1198, 1636773808000], [-120.9125664, 39.31015875, 1198, 1636773813000], [-120.91256607, 39.31015885, 1198, 1636773818000], [-120.91256589, 39.31015889, 1198, 1636773823000], [-120.91256633, 39.31015836, 1198, 1636773828000], [-120.91256677, 39.31015774, 1199, 1636773833000], [-120.91256692, 39.3101576, 1199, 1636773838000]]
10:00:26 [plans_console:280:CRITICAL] Uncaught exception
Traceback (most recent call last):
  File "shapely\speedups\_speedups.pyx", line 90, in shapely.speedups._speedups.geos_linestring_from_py
AttributeError: 'list' object has no attribute '__array_interface__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\caver\Documents\GitHub\plans_console\plans_console.py", line 792, in debriefButtonClicked
    self.dmg=DebriefMapGenerator(self,self.sts,self.debriefURL)
  File "C:\Users\caver\Documents\GitHub\plans_console\sartopo_bg.py", line 468, in __init__
    self.newFeatureCallback(f)
  File "C:\Users\caver\Documents\GitHub\plans_console\sartopo_bg.py", line 1646, in newFeatureCallback
    self.addOuting(f)
  File "C:\Users\caver\Documents\GitHub\plans_console\sartopo_bg.py", line 1330, in addOuting
    self.cropUncroppedTracks()
  File "C:\Users\caver\Documents\GitHub\plans_console\sartopo_bg.py", line 1517, in cropUncroppedTracks
    croppedTrackLines=self.sts2.crop(utid,bid,beyond=cropDegrees)
  File "C:\Users\caver\Documents\GitHub\plans_console\sartopo_python.py", line 1921, in crop
    targetGeom=LineString(tgc)
  File "C:\Program Files (x86)\Python310-32\lib\site-packages\shapely\geometry\linestring.py", line 46, in __init__
    ret = geos_linestring_from_py(coordinates)
  File "shapely\speedups\_speedups.pyx", line 174, in shapely.speedups._speedups.geos_linestring_from_py
AssertionError

    def crop(self,target,boundary,beyond=0.0001,deleteBoundary=False,useResultNameSuffix=False):
        logging.info('c1')
        if isinstance(target,str): # if string, find feature by name; if id, find feature by id
            targetStr=target
            if len(target)==36: # id
                targetShape=self.getFeature(id=target)
            else:
                targetShape=self.getFeature(title=target,featureClassExcludeList=['Folder','OperationalPeriod'])
        else:
            targetShape=target
            targetStr='NO TITLE'
            if isinstance(targetShape,dict):
                targetStr=targetShape.get('title','NO TITLE')
        if not targetShape:
            logging.warning('Target shape '+targetStr+' not found; operation aborted.')
            return False

        logging.info('c2')
        tg=targetShape['geometry']
        targetType=tg['type']
        if targetType=='Polygon':
            tgc=tg['coordinates'][0]
            tgc=self.removeSpurs(tgc)
            targetGeom=Polygon(tgc) # Shapely object
        elif targetType=='LineString':
            tgc=tg['coordinates']
            logging.info('tgc before:'+str(tgc))
            tgc=self.removeSpurs(tgc)
            logging.info('tgc after:'+str(tgc))
            targetGeom=LineString(tgc) # line 1921
        else:
            logging.warning('crop: target feature '+targetStr+' is not a polygon or line: '+targetType)
            return False

So, the geom ops code needs to be redone to deal with four-element points as well as two-element points, and to preserve all four elements if possible.

[caver456]

The obvious easy failsafe option is: don't try to keep the 3rd or 4th elements of each vertex; just return the two-element-vertex feature. We should explore options for keeping the 4-element lists before giving up:

1) do all the shapely operations on the 2-element vertices, then try to match them back up again afterwards 2) overload a bunch of shapely guts to deal with 4-element vertices - probably ugly!

Either way, getting a 2-element list from a either-2-or-4-element list is easy:

>>> a=[[1,2,3,4],[2,3,4,5],[3,4,5,6]]
>>> b=[x[0:2] for x in a] 
>>> b
[[1, 2], [2, 3], [3, 4]]
>>> [x[0:2] for x in b]   
[[1, 2], [2, 3], [3, 4]]

and, searching is easy - just a question of how slow it is:

>>> [x for x in a if x[0:2]==[1,2]]
[[1, 2, 3, 4]]
>>> [x for x in a if x[0:2]==[1,3]] 
[]

[caver456]

During testing on CTD 4231, crop of apptrack just returns 200 with no json. This doesn't cause any exception or abort or traceback. Is this new? Is it related to this issue (4-point vs 2-point)? Do we need to make a line before cropping? If it's not directly related to this issue, a new issue should be created.

2022-12-17 17:50:28,459 [INFO] crop: target=CW209B  boundary=cropper
2022-12-17 17:50:28,962 [INFO] getUsedSuffixList: base=CW209B  rval=[]
2022-12-17 17:50:28,994 [INFO] editFeature called
2022-12-17 17:50:28,995 [INFO]  id specified: 33a63b2f-b41b-48e6-90ba-8a3eb8275a8a
2022-12-17 17:50:28,995 [INFO] sending post to http://localhost:8080/api/v1/map/HN7/Apptrack/33a63b2f-b41b-48e6-90ba-8a3eb8275a8a
2022-12-17 17:50:28,996 [INFO] {'json': '{"type": "Feature", "id": "33a63b2f-b41b-48e6-90ba-8a3eb8275a8a", "geometry": {"size": 56, "coordinates": "56 points", "incremental": true, "type": "LineString"}}'}
2022-12-17 17:50:29,026 [ERROR] sendRequest: response had no decodable json:<Response [200]>

[caver456]

Confirmed that cropping a line made of four-element vertices works fine. The line CW209Ba was made from Copy As Line in the GUI; it still had four-element vertices before the crop operation, and the crop did work visually.

                  ...
                  ...
                  [
                     -121.17803321231266,
                     39.27680332217298,
                     405,
                     1668202204061
                  ],
                  [
                     -121.17806355480214,
                     39.27681245844744,
                     406,
                     1668202209059
                  ]
               ],
               "incremental": true,
               "type": "LineString"
            },
            "id": "ca823f3f-3a45-43f4-9be2-5e1afe7b36a4",
            "type": "Feature",
            "properties": {
               "stroke-opacity": 1,
               "creator": "11UEE9",
               "pattern": "solid",
               "description": "",
               "stroke-width": 2,
               "fill": "#0000FF",
               "title": "CW209Ba",
               "stroke": "#0000FF",
               "class": "Shape",
               "updated": 0,
               "timestamp": 1671342667436,
               "gpstype": "TRACK"
            }
         }
      ]
   }
}
2022-12-17 21:52:00,397 [INFO] crop: target=CW209Ba  boundary=cropper
2022-12-17 21:52:00,779 [INFO] getUsedSuffixList: base=CW209Ba  rval=[]
2022-12-17 21:52:00,825 [INFO] editFeature called
2022-12-17 21:52:00,825 [INFO]  id specified: ca823f3f-3a45-43f4-9be2-5e1afe7b36a4
2022-12-17 21:52:00,825 [INFO] sending post to http://localhost:8080/api/v1/map/HN7/Shape/ca823f3f-3a45-43f4-9be2-5e1afe7b36a4
2022-12-17 21:52:00,825 [INFO] {'json': '{"type": "Feature", "id": "ca823f3f-3a45-43f4-9be2-5e1afe7b36a4", "geometry": {"size": 56, "coordinates": "56 points", "incremental": true, "type": "LineString"}}'}
2022-12-17 21:52:04,715 [INFO]   processing 1 feature(s):['ca823f3f-3a45-43f4-9be2-5e1afe7b36a4']
2022-12-17 21:52:04,715 [INFO]   response contained geometry for Shape:CW209Ba but it matched the cache, so no cache update or callback is performed

So, this is probably just a case of inability to crop apptracks. This kind of makes sense, because the apptrack is still in process, by definition - so cropping the middle would make no sense. The solution is probably to make a copy of the apptrack as a line, in code, and then crop that. Creating a separate issue to handle cropping of apptracks, and closing this issue since it was fixed at some point in the past, during work in the plans_console repo.