search

pacificclimate / pdp

The PCIC Data Portal - Server software to run the entire web application
GNU General Public License v3.0
1 stars 2 forks source link

HTTP 500 on Portal and Data - Parse Error In Geometry #92

Open matthewbenstead opened 6 years ago

matthewbenstead commented 6 years ago

I observed a handful of HTTP 500's on August 24th. Some of the HTTP requests are: 

/data/pcds/agg/?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=MULTIPOLYGON%28%28%28-125.98478862538049+54.73931765174576%2C-126.19199673053424+54.73420005007077%2C-125.98478862538049+54.73931765174576%29%29%29&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=xlsx&download-climatology=Climatology

/portal/pcds/count_stations?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=MULTIPOLYGON(((-125.98478862538049+54.73931765174576%2C-126.19199673053424+54.73420005007077%2C-125.98478862538049+54.73931765174576)))&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=nc&metadata-format=csv&metadata-format=&openid-provider=https%3A%2F%2Flogin.launchpad.net%2F

/portal/pcds/record_length?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=MULTIPOLYGON(((-125.98478862538049+54.73931765174576%2C-126.19199673053424+54.73420005007077%2C-125.98478862538049+54.73931765174576)))&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=nc&metadata-format=csv&metadata-format=&openid-provider=https%3A%2F%2Flogin.launchpad.net%2F

They occurred around August 24th 2018 05:11.

The requests in the front-end and back-end both gave similar error. Front end: 

2018-08-24 12:10:12 [30] [ERROR] Exception raised during streamed response: ('(psycopg2.InternalError) geometry requires more points\nHINT:  "...25.98478862538049 54.73931765174576))" <-- parse error at position 130 within geometry\n',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 68, in run_for_one
    self.accept(listener)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 27, in accept
    client, addr = listener.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
error: [Errno 11] Resource temporarily unavailable

2018-08-24 12:10:12 [30] [INFO] XXX.XXX.XXX.XXX - - [24/Aug/2018:12:10:12 +0000] "GET /pcds/count_stations?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=MULTIPOLYGON(((-125.98478862538049+54.73931765174576%2C-126.19199673053424+54.73420005007077%2C-125.98478862538049+54.73931765174576)))&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=nc&metadata-format=csv&metadata-format=&openid-provider=https%3A%2F%2Flogin.launchpad.net%2F HTTP/1.1" 500 664 "https://data.pacificclimate.org/portal/pcds/map/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
2018-08-24 12:10:12 [30] [ERROR] Exception raised during streamed response: ('(psycopg2.InternalError) geometry requires more points\nHINT:  "...25.98478862538049 54.73931765174576))" <-- parse error at position 130 within geometry\n',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 68, in run_for_one
    self.accept(listener)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 27, in accept
    client, addr = listener.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
error: [Errno 11] Resource temporarily unavailable

2018-08-24 12:10:12 [30] [INFO] XXX.XXX.XXX.XXX - - [24/Aug/2018:12:10:12 +0000] "GET /pcds/record_length?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=MULTIPOLYGON(((-125.98478862538049+54.73931765174576%2C-126.19199673053424+54.73420005007077%2C-125.98478862538049+54.73931765174576)))&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=nc&metadata-format=csv&metadata-format=&openid-provider=https%3A%2F%2Flogin.launchpad.net%2F HTTP/1.1" 500 664 "https://data.pacificclimate.org/portal/pcds/map/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
2018-08-24 12:10:26 [30] [ERROR] Exception raised during streamed response: ('(psycopg2.InternalError) geometry requires more points\nHINT:  "...25.98478862538049 54.73931765174576))" <-- parse error at position 130 within geometry\n',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 68, in run_for_one
    self.accept(listener)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 27, in accept
    client, addr = listener.accept()
  File "/usr/lib/python2.7/socket.py", line 206, in accept
    sock, addr = self._sock.accept()
error: [Errno 11] Resource temporarily unavailable

Same error in the back-end:

2018-08-24 12:10:04 [22] [INFO] XXX.XXX.XXX.XXX - - [24/Aug/2018:12:10:04 +0000] "GET /pcds/agg/?from-date=2018%2F06%2F01&to-date=2018%2F08%2F24&input-polygon=&input-var=thickness_of_rainfall_amount&network-name=&input-freq=daily&data-format=nc&download-climatology=Climatology HTTP/1.1" 200 120782 "https://data.pacificclimate.org/portal/pcds/map/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
2018-08-24 12:11:56 [15] [ERROR] Exception raised during streamed response: ('(psycopg2.InternalError) geometry requires more points\nHINT:  "...25.98478862538049 54.73931765174576))" <-- parse error at position 130 within geometry\n',)
None

The error seemed unusual given there appeared to be 3 points in the polygon.

corviday commented 6 years ago

Confusingly, there are actually only two points in this polygon. WKT, the format the PDP and climate explorer polygons are expressed in, requires restating the first point to "close" the polygon; the first and last points are always identical in a WKT polygon.

Possibly the front end needs to check whether a polygon has more than two points and alert the user if so, instead of sending a two-point "polygon" to the backend.

jameshiebert commented 6 years ago

Good catch! I totally missed that.

corviday commented 6 years ago

I was able to reproduce this error by clicking the draw polygon button, clicking on one point, and double-clicking on another to make a two-point polygon. While it should be pretty obvious to the user what's gone wrong when this occurs, it might be worth fixing on the basis of how dramatic it looks - a two point polygon causes the vector map to fail to load:

screenshot from 2018-08-30 15-11-02

A popup error box that says "You must select at least three points" and clearing the polygon before passing it off to any code that will try to extract the space it represents should be a pretty straightforward fix.