search

PiRSquared17 / morisoliver

Automatically exported from code.google.com/p/morisoliver
0 stars 0 forks source link

Allow export of data by a drawn polygon for selection shape #15

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Currently export is a select by the bounding box of the current map view.
In older Java Web Start OLIVER a shape could be drawn that would define the 
export select.  (Data is still selected, not clipped if features fall over the 
line.  Clip is not being asked for).  GeoServer WFS supports using a 
non-rectangular shape as a filter for export.  The application just needs to 
offer this functionality in the UI.

Original issue reported on code.google.com by Aleda.Fr...@state.ma.us on 24 Feb 2011 at 7:11

GoogleCodeExporter commented 9 years ago

Original comment by Aleda.Fr...@state.ma.us on 24 Feb 2011 at 8:58

GoogleCodeExporter commented 9 years ago 
If I can nail down this issue, it should help w/ issue 16.  So I'm trying to 
start simple by sticking w/ a bbox but treating it as a polygon to do the 
extraction.  I'm running into problems.

Aleda, I'm not sure why the following WFS "hits" query it returning 0.

<wfs:GetFeature resultType="hits" xmlns:wfs="http://www.opengis.net/wfs" 
service="WFS" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs 
http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<wfs:Query typeName="massgis:GISDATA.TOWNSSURVEY_ARC" srsName="EPSG:26986" 
xmlns:massgis="http://massgis.state.ma.us/featuretype">
<ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:Intersects>
<ogc:PropertyName>SHAPE</ogc:PropertyName>
<gml:Polygon xmlns:gml="http://www.opengis.net/gml" srsName="EPSG:26986">
<gml:outerBoundaryIs>
<gml:LinearRing>
<gml:coordinates decimal="." cs="," ts=" ">316515.57489637,855555.06639038 
324791.24501706,855555.06639038 324791.24501706,863294.08896059 
316515.57489637,863294.08896059 
316515.57489637,855555.06639038</gml:coordinates>
</gml:LinearRing>
</gml:outerBoundaryIs>
</gml:Polygon>
</ogc:Intersects>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>

Its original bbox counterpart returns stuff OK.  The following paste is not for 
the exact same BBOX, but the idea is there.

<wfs:GetFeature resultType="hits" xmlns:wfs="http://www.opengis.net/wfs" 
service="WFS" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs 
http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<wfs:Query typeName="massgis:GISDATA.TOWNSSURVEY_ARC" srsName="EPSG:26986" 
xmlns:massgis="http://massgis.state.ma.us/featuretype">
<ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:Intersects>
<ogc:PropertyName>SHAPE</ogc:PropertyName>
<gml:Envelope xmlns:gml="http://www.opengis.net/gml" srsName="EPSG:26986">
<gml:lowerCorner>310265.59212458 867525.68217948</gml:lowerCorner>
<gml:upperCorner>312801.79196632 869608.98919233</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>

Any ideas?

Original comment by cpl...@gmail.com on 14 Sep 2011 at 3:04

GoogleCodeExporter commented 9 years ago 
The syntax is going to be different. Take a look at this example:
https://wiki.state.ma.us/confluence/display/massgis/get+feature+by+shape

I have ported the wiki to a new place and it's easier to navigate now I think.
This is the front page: https://wiki.state.ma.us/confluence/display/massgis

Original comment by Aleda.Fr...@state.ma.us on 14 Sep 2011 at 3:10

GoogleCodeExporter commented 9 years ago 
Thanks for the pointer.  The only thing majorly different between your example 
and my 1st POST was Intersects vs. Within.  I'll keep digging.

Original comment by cpl...@gmail.com on 14 Sep 2011 at 3:20

GoogleCodeExporter commented 9 years ago 
It looks like this describes a box, counterclockwise, repeating the first point 
(all this is fine) and it should intersect some arcs in 
massgis:GISDATA.TOWNSSURVEY_ARC.  (see attachment).  I submit the request and 
see what happens.

Original comment by Aleda.Fr...@state.ma.us on 14 Sep 2011 at 3:40

Attachments:

GoogleCodeExporter commented 9 years ago 
Sorry to be dense, but I'm not sure how to parse your last sentence!  You 
submitted the request, and the attachment is what happened?  Or you will submit 
the request and see what happens.

Sorry!

Original comment by cpl...@gmail.com on 14 Sep 2011 at 3:46

GoogleCodeExporter commented 9 years ago 
No, sorry, I plotted your points against a map of the datalayer just to make 
sure that they were supposed to grab something.  Looks like they should.  
However, I get 0 results as well with your request, whether I use Intersects or 
Within.  The wiki example works (of course, that is a different datalayer and 
different shape).  I'm going to experiment a little more.  Maybe I'll change to 
TOWNS_POLY and keep your shape.

Original comment by Aleda.Fr...@state.ma.us on 14 Sep 2011 at 3:49

GoogleCodeExporter commented 9 years ago 
I think this is what you need to do - change the version to 1.0.0 in your 
requset, then it works for me.

Original comment by Aleda.Fr...@state.ma.us on 14 Sep 2011 at 3:57

GoogleCodeExporter commented 9 years ago 
Although when I do that I actually get the features back, not just a count of 
them.

Original comment by Aleda.Fr...@state.ma.us on 14 Sep 2011 at 4:00

GoogleCodeExporter commented 9 years ago 
Great detective work.  That is an interesting find.  But if I can't count the # 
of features before pulling them back, that will be a show stopper.

I just read "And resultType is a WFS 1.1.0 feature."  Ouch!

Is there any way to stick w/ 1.1.0 but still be able to use POLYGON in a filter?

Original comment by cpl...@gmail.com on 14 Sep 2011 at 4:09

GoogleCodeExporter commented 9 years ago 
I think I've got it.

<wfs:GetFeature resultType="hits" xmlns:wfs="http://www.opengis.net/wfs" 
service="WFS" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs 
http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<wfs:Query typeName="massgis:GISDATA.TOWNSSURVEY_ARC" srsName="EPSG:900913" 
xmlns:massgis="http://massgis.state.ma.us/featuretype">
<ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:Within>
<ogc:PropertyName>SHAPE</ogc:PropertyName>
<gml:Polygon xmlns:gml="http://www.opengis.net/gml" srsName="EPSG:900913">
<gml:exterior>
<gml:LinearRing>
<gml:posList>-7829667.183374 5146451.9573434 -7793588.9060299 5146451.9573434 
-7793588.9060299 5185587.7158184 -7829667.183374 5185587.7158184 
-7829667.183374 5146451.9573434</gml:posList>
</gml:LinearRing>
</gml:exterior>
</gml:Polygon>
</ogc:Within>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>

Loosely followed this thread, 
http://osgeo-org.1803224.n2.nabble.com/Spatial-filters-td6105395.html.

The result snippet . . .

<wfs:FeatureCollection numberOfFeatures="70" ...

Original comment by cpl...@gmail.com on 14 Sep 2011 at 4:29

GoogleCodeExporter commented 9 years ago 
I have added an identify poly button to the main menu.

A couple of things . . .

* Now that I'm rereading the issue, an ability to query by polygon wasn't 
really requested.  I needed to go through this exercise to be able to build the 
foundation for export, so it was not a wasted effort.

* If you want to keep this functionality, I need someone (Saul?) to go in and 
make it talk correctly to your toolbar config stuff.  Search for this line to 
see the part that I left out:

// don't know what I need to do here for Poly and topToolBar

* You will probably want to change the icon and the 'IdentifyPoly' label.  Just 
say the word when you have alternates ready.

* If you don't want this functionality, my feelings won't be hurt.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 1:32

GoogleCodeExporter commented 9 years ago 
The last comment folds into:

Committed revision 105.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 1:32

GoogleCodeExporter commented 9 years ago 
Does export by polygon apply to rasters, too?  If so, how does this meld w/ 
coming up w/ whether or not a raster size will be OK to export?

Original comment by cpl...@gmail.com on 15 Sep 2011 at 2:36

GoogleCodeExporter commented 9 years ago 
I would not recommend extracting rasters by polygon, it's too complicated as 
you say with figuring out the file size.  Let's just give them the regular 
bounding box.

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 2:42

GoogleCodeExporter commented 9 years ago 
KML's OK exporting by poly?

Original comment by cpl...@gmail.com on 15 Sep 2011 at 3:09

GoogleCodeExporter commented 9 years ago 
I guess that should work?  The features get selected by poly and then they go 
into the file sort of?  Let's try it :)

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 3:14

GoogleCodeExporter commented 9 years ago 
I selfishly hoped you wouldn't say yes!  Currently, the only thing that gets 
extracted via a FILTER is .shp.  Everything else is via a straight HTTP GET, 
normal bbox query.  I'd like to see if we could pull a KML via FILTER before I 
monkey around w/ a poly.  So I might leave it for now by lumping it into the 
bbox-only bucket w/ rasters.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 3:17

GoogleCodeExporter commented 9 years ago 
Sure, leave it for later if that's easier for now.

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 3:18

GoogleCodeExporter commented 9 years ago 
So here we go!  Running into filtergate.

http://wsgw.mass.gov/geoserver/wfs?request=getfeature&version=1.0.0&outputformat
=SHAPE-ZIP&service=wfs&typename=massgis:MORIS.SLA_ARC&filter=<ogc:Filter 
xmlns:ogc="http://ogc.org" 
xmlns:gml="http://www.opengis.net/gml"><ogc:Within><ogc:PropertyName>SHAPE</ogc:
PropertyName><gml:Polygon xmlns:gml="http://www.opengis.net/gml" 
srsName="EPSG:26986"><gml:exterior><gml:LinearRing><gml:posList>285891.014170433
3 844892.6166250343 332262.01492860704 845606.7742510559 331473.9098411659 
887806.6082367487 285379.2183591436 887096.706037038 285891.0141704333 
844892.6166250343</gml:posList></gml:LinearRing></gml:exterior></gml:Polygon></o
gc:Within></ogc:Filter>&SRSNAME=EPSG:26986

Attempted to construct illegal filter - I dont understand the tag: 
gml:exterior.  HINT: tags are case-sensitive!
Attempted to construct illegal filter - I dont understand the tag: 
gml:exterior.  HINT: tags are case-sensitive!

Original comment by cpl...@gmail.com on 15 Sep 2011 at 3:48

GoogleCodeExporter commented 9 years ago 
Nevermind.  I should be using 1.1.0.  And a quick test passes mustard.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 3:50

GoogleCodeExporter commented 9 years ago 
This is actually coming from GeoServer, not the Gateway.
I tested it against my internal copy of GeoServer and I get the same error.
So there is something about the Filter that GeoServer doesn't like. 
Did you get this example from a working wiki example?  It doesn't seem to like 
the exterior tag.  Maybe it's only in version 1.1.0? That worked for me, I got 
a good .zip file with a good shapefile inside.

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 3:54

GoogleCodeExporter commented 9 years ago 
I thought about how to keep the workflow as friendly as possible, and I came up 
w/ the attached.

Basically there are 3 ways to launch the wizard:

(1) the original way where your map's bbox pretty much drives the equation
(2) performing either a bbox or a polygon identify request will:
  (a) allow you to launch the wizard w/ your drawn bbox / polygon
  (b) allow you to launch the wizard w/ a polygon feature as the constraint

If you launch the wizard using one of the (2) options, the wizard will not have 
the bbox stuff that you normally see on the wizard.

The polygon will be used (if applicable) for shp extracts.  If something else 
is lumped into an extract that started from an identify, the shp's.bounds() 
will be used as the bbox (i.e. not the map's).

I'm running into what I think are some kind of filter max size issues.  If you 
restrict by a small feature (a feature w/ not too many vertices), the query 
seems to go through fine.  If you choose a large feature, e.g. a county, I 
think it is overloading the Filter, and nothing is getting returned.  I may 
need help verifying this.

Although the feature geometries are coming back from geoserver as MULTIPOLYGON 
(when they are not lines or points), I am assuming that there is only one 
polygon being represented in an individual feature.

Let the games begin . . .

Original comment by cpl...@gmail.com on 15 Sep 2011 at 7:32

Attachments:

GoogleCodeExporter commented 9 years ago 
Issue 16 has been merged into this issue.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 7:35

GoogleCodeExporter commented 9 years ago 
Committed revision 109.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 7:37

GoogleCodeExporter commented 9 years ago 
Sometimes the identify polygon that's drawn is under the data.
To reproduce:

1.  Open MORIS, draw a identify polygon
2.  Draw MWRA Towns with Water or Sewer (this is a solid colored polygons).
3.  Now, the identify polygon will draw under the MWRA polygons.

If the MRWA polys were drawn first and then the first identify poly was done it 
would be OK.  It's only data that added to the map after the first identify 
that then draws over and new identify polygons. 

see 2 attached screen shots.

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 8:43

Attachments:

GoogleCodeExporter commented 9 years ago 
I tried a test extract by poly (see screenshot).
I was supposed to get 3 points, 1 line and 2 towns.
I got 3 points, 0 lines and 1 town.
Is it doing Within instead of Intersects?  Behaves like it.
We want anything the Intersects.
Yes I see in your Filter you're using Within - can you switch it?  Thanks!

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 8:52

Attachments:

GoogleCodeExporter commented 9 years ago 
I like the Identify by Poly tool and the UI.

Original comment by Aleda.Fr...@state.ma.us on 15 Sep 2011 at 8:53

GoogleCodeExporter commented 9 years ago 
Perhaps I should clear the query poly when a new layer is added.  After all, 
the query doesn't include the new layer, anyway.

I'll change it to Intersects for the next commit tomorrow.

Original comment by cpl...@gmail.com on 15 Sep 2011 at 9:52

GoogleCodeExporter commented 9 years ago 
Query poly now cleared on layer addition.  Intersects in place of Within.

It looks like big polys that launch the wizard bomb.  I'll see if I can trace a 
POST.

Committed revision 111.

Original comment by cpl...@gmail.com on 16 Sep 2011 at 5:11

GoogleCodeExporter commented 9 years ago 
Maybe the POST limit is just on my end.  I'm running a different proxy here.  
Could you test on your end to see if all is OK?

Original comment by cpl...@gmail.com on 16 Sep 2011 at 5:19

GoogleCodeExporter commented 9 years ago 
Identify by poly first step not getting good results still.  (see screenshot of 
towns and lighthouses - getting too many lighthouses).

<wfs:GetFeature resultType="hits" xmlns:wfs="http://www.opengis.net/wfs" 
service="WFS" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs 
http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<wfs:Query typeName="massgis:GISDATA.LIGHTHOUSES_PT" srsName="EPSG:26986" 
xmlns:massgis="http://massgis.state.ma.us/featuretype">
<ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:Intersects>
<ogc:PropertyName>SHAPE</ogc:PropertyName>
<gml:Polygon xmlns:gml="http://www.opengis.net/gml" srsName="EPSG:26986">
<gml:exterior>
<gml:LinearRing>
<gml:posList>303872.73003439 862746.74987066 315642.78927561 862746.74987066 
315642.78927561 871872.11968724 303872.73003439 871872.11968724 303872.73003439 
862746.74987066</gml:posList>
</gml:LinearRing>
</gml:exterior>
</gml:Polygon>
</ogc:Intersects>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>

I drew what's in the the screenshot "identify_by_poly_step1".

But, if I take what gets posted in terms of coordinates and plot them I get a 
rectangle that looks like the second screenshot "identify_by_poly_posted".

That rectangle would grab the 3 lighthouses.  

Somehow the shape I draw is being turned into a box the first identify step.  
The extract is also flagging 3 to be exported, and 3 are exported.  So it's 
using a bbox there too I guess. 

However, the identify poly is not drawing underneath newly added layers 
anymore, so that seems fixed.

Original comment by Aleda.Fr...@state.ma.us on 16 Sep 2011 at 8:02

Attachments:

GoogleCodeExporter commented 9 years ago 
You said "It looks like big polys that launch the wizard bomb."  But any big 
poly I draw right now is getting changed into a rectangle of 5 coordinates (one 
repeated as expected), so I can't test...

Original comment by Aleda.Fr...@state.ma.us on 16 Sep 2011 at 8:05

GoogleCodeExporter commented 9 years ago 
Sorry.  I accidentally passed around a bbox no matter what for the poly 
extract.  Fixed.

Committed revision 116.

Original comment by cpl...@gmail.com on 16 Sep 2011 at 8:15

GoogleCodeExporter commented 9 years ago 
OK, I see the correct shape being used instead of a rectangle.  The above town 
and lighthouse example works correctly now. I used a shape with 50 vertices 
(see screenshot for proof!) to select and it worked fine - both identify and 
extract.  When you said "big polys that launch the wizard bomb" how many 
vertices were you trying?

Original comment by Aleda.Fr...@state.ma.us on 16 Sep 2011 at 8:48

Attachments:

GoogleCodeExporter commented 9 years ago 
Cool polygon!  When I try using something like a complex county (lots of 
vertices) from another layer as the constraint, it doesn't get any hits.  I'm 
referring to launching the data export from the identify box popup's Feature 
Details list.

So if you ID on a polygon layer and click on its At a Glance to actually see 
the features.  THEN right-click on one of those to launch the data export wiz.

Original comment by cpl...@gmail.com on 16 Sep 2011 at 8:52

GoogleCodeExporter commented 9 years ago 
When I tried to select a county and use it to extract schools I was told I 
would get 576 features.  I got a zip file but it only had a lyr file an xml 
file and an avl file in it.  Is that what you're getting - a zip file but 
without the actual shapefile? 

I'm looking at Firebug and I'm confused.  Are you posting an XML file that is a 
GetFeature to do this extract.  What I see kind of looks like a URL thing.  
I'll post a screenshot.

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 4:40

Attachments:

GoogleCodeExporter commented 9 years ago 
I see the request submitted to GeoServer as a large query but I do not see any 
result or error message.  Maybe this query can't take as large a list of 
coordinates.  But I wonder if it could if it was an XML post type request (if 
it isn't, I'm still not sure).

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 4:52

GoogleCodeExporter commented 9 years ago 
You're getting farther than I did.  I imagine it might not be making it into 
the zip file because I am performing a HTTP GET to pull the file.  It could be 
running into the max # of chars for a URL.  Can you see the request as it hits 
your data server?

Original comment by cpl...@gmail.com on 19 Sep 2011 at 4:52

GoogleCodeExporter commented 9 years ago 
Our messages crossed.  Yes, a GET could be a problem because of the max # of 
characters for a URL.  Although since it seems to get to GeoServer I'm not sure 
about that part of it.  But maybe a POST would be more reliable?  I can try to 
take that list of coords and test a POST XML...

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 4:54

GoogleCodeExporter commented 9 years ago 
The list of coordinates is so large that I'm having trouble pulling it out of 
the GeoeServer log file or Firebug.  Is it easy for you to send me the list for 
a county?  If so I could put it into an XML file, post it to GeoServer via wget 
(although you could do this too) and we'll see if that comes out with a 
shapefile extract.

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 4:59

GoogleCodeExporter commented 9 years ago 
I just slurped some from a WFS request.

Original comment by cpl...@gmail.com on 19 Sep 2011 at 5:06

Attachments:

GoogleCodeExporter commented 9 years ago 
I got 0 features, but I'm not used to extracting by non-state plane.  I used 
900913 as the EPSG is that correct?

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 6:27

GoogleCodeExporter commented 9 years ago 
What proj would you prefer?  Those were 900913 I think.

Original comment by cpl...@gmail.com on 19 Sep 2011 at 6:40

GoogleCodeExporter commented 9 years ago 
26986 if you can

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 6:50

GoogleCodeExporter commented 9 years ago

Original comment by cpl...@gmail.com on 19 Sep 2011 at 6:56

Attachments:

GoogleCodeExporter commented 9 years ago 
I see in the coordinate list that the county is made up of several polygons 
(that can be true).  Maybe that is too complicated a query.  I'm pretty sure 
this used to work but I'm not sure how it used to be implemented.  Maybe it 
didn't use to work that well.  I don't see another way of doing it except 
asking for the coordinates and then giving them back in the next query.

I switched to towns to do something simpler.  Fire stations by town (Maynard) 
worked. But Gloucester doesn't even claim I'll get anything, there are a lot of 
coordinates.

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 8:24

GoogleCodeExporter commented 9 years ago 
Saul says mutlipart polygons are OK, but you'll have to post XML because there 
will be too many vertices for browsers.  I'm trying to get an example to work, 
but I need to get the syntax correct.

Original comment by Aleda.Fr...@state.ma.us on 19 Sep 2011 at 8:59

GoogleCodeExporter commented 9 years ago 
Where are multipolys coming into play?  Comment 23 includes this assumption . . 

"Although the feature geometries are coming back from geoserver as MULTIPOLYGON 
(when they are not lines or points), I am assuming that there is only one 
polygon being represented in an individual feature."

Is that not a valid assumption?

Since I am only allowing you to right-click on one row to launch the wizard, it 
should only launch w/ a polygon (posList set of positions).

I think what you want is more of a query builder than a polygon constraint, 
especially in the case of export-this-layer-by-county.  But that isn't what 
this issue asks for.

Original comment by cpl...@gmail.com on 19 Sep 2011 at 9:09

GoogleCodeExporter commented 9 years ago 
mkzip now makes SHAPE-ZIP requests by POST.  I left the other request types 
(tif, kml, etc.) alone.

Committed revision 127.

Original comment by cpl...@gmail.com on 21 Sep 2011 at 2:08