search

MapServer / MapServer-import

3 stars 2 forks source link

MAP EXTENT truncates GetFeature requests for data outside bounds #1287

Open tbonfort opened 12 years ago

tbonfort commented 12 years ago

Reporter: tomkralidis Date: 2005/03/18 - 15:50

If data in a given layer is outside the bounds of the MAP/EXTENT object, this
data is not included in a blanket GetFeature request, i.e.:

http://[host]/mapserv?version=1.0.0&service=WFS&request=GetFeature&typename=roads

It should be.
tbonfort commented 12 years ago

Author: tomkralidis Date: 2008/03/17 - 04:50 I can confirm that this behaviour still exists.

Thinking about this again, I wonder what our best approach is here. Even though the layer bbox does not overlap MAP/EXTENT, there is data there, in accordance to what a WFS client thinks should be returned (i.e. all data).

I guess this leads us to the question: what relevance does MAP/EXTENT have in WFS Server (mapwfs.c)?

Comments?

tbonfort commented 12 years ago

Author: sdlime Date: 2008/03/17 - 19:42 Depends on the query type. For attributes the extent is used as a pre-filter before any attribute-based filtering is done. Does WFS just wrap the msQueryUsing... functions or does it have it's own processing logic? I assume the latter since it supports spatial filters that the CGI-based query modes do not.

What sorts of OGC filters are we talking about here?

Steve

tbonfort commented 12 years ago

Author: sdlime Date: 2008/03/17 - 19:44 One workaround would be to set the map->extent based on the layers data extent before doing any queries...

Steve

tbonfort commented 12 years ago

Author: assefa Date: 2008/04/15 - 19:55 The user has the ability to use a bbox as part of a wfs query and that would be used to set the extents before doing a query. For me I think it is an acceptable workaround.

tbonfort commented 12 years ago

Author: tomkralidis Date: 2008/04/16 - 13:58 Replying to [comment:5 assefa]:

The user has the ability to use a bbox as part of a wfs query and that would be used to set the extents before doing a query. For me I think it is an acceptable workaround.

Right. What about the following use case: the user invokes the URL as per:

http://[host]/mapserv?version=1.0.0&service=WFS&request=GetFeature&typename=roads

...in this case, the client is expecting all the data to return (i.e. no filters have been applied)

?

tbonfort commented 12 years ago

Author: assefa Date: 2008/04/16 - 15:24 ok convinced :) In that case, what we should I think would be to use the bbox extents that were advertised for the layer (in the getcapabilities) and set the map extent with it. Any bbox settings that are part of the query would overwrite this default behavior.

tbonfort commented 12 years ago

Author: sdlime Date: 2008/04/16 - 17:51 I wish we could do this (Assefa's) logic for the CGI-based attribute query but there's no way to know if the map->extent was modified or if it is the original. Anyway, I think Assefa's idea to use the layer extent(s) is sufficient.

Steve

tbonfort commented 12 years ago

Author: tomkralidis Date: 2008/06/13 - 19:09 Fixed in 5dcfe20dde97a91fa5f8539b784a4168c11acb3c (r7674). Tested with:

$ ~/foss4g/mapserver/trunk/mapserver/mapserv QUERY_STRING="map=config.map&service=WFS&version=1.0.0&request=GetFeature

against:

MAP
 NAME "mstrac-1287"
 STATUS ON
 SIZE 400 300
 #EXTENT -180 -90 180 90
 EXTENT 10 10 20 20  # (does not overlap data)
 IMAGECOLOR 255 255 255
 WEB
  IMAGEPATH "/tmp/"
  IMAGEURL "/ms_tmp/"
  METADATA
   "wfs_onlineresource" "http://foo"
   "wfs_srs"               "EPSG:4326"
  END
 END
 PROJECTION
  "init=epsg:42304"
 END
 LAYER
  NAME obs
  STATUS ON
  TYPE POINT
  DATA "../../trunk/msautotest/wxs/data/obs.shp"
  DUMP TRUE
  METADATA
    "wfs_title"         "obs"
    "wfs_featureid"     "stn_id"
  END
  PROJECTION
    "init=epsg:4326"
  END
  CLASS
   NAME "obs"
   COLOR 200 255 0
   OUTLINECOLOR 120 120 120
  END
 END
 LAYER
  NAME prov
  STATUS ON
  TYPE POINT
  DATA "../../trunk/msautotest/wxs/data/province.shp"
  DUMP TRUE
  METADATA
    "wfs_title"         "province"
    "wfs_featureid"     "name"
  END
  PROJECTION
    "init=epsg:42304"
  END
  CLASS
   NAME "obs"
   COLOR 200 255 0
   OUTLINECOLOR 120 120 120
  END
 END
END

..testing and editing in the mapfile:

If anyone can give this a once over or test, that would be great.

tbonfort commented 12 years ago

Author: bartvde Date: 2008/10/08 - 13:30 Changeset 5dcfe20dde97a91fa5f8539b784a4168c11acb3c (r7674) introduced a new problem, changing the sequence of queries in the WFS GetFeature request can now yield different results, since the map->extent is set to the last query layer's extent. This is done in a separate loop before the queries are actually run. So this can result in one query sequence resulting in 1 result (the expected), whereas changing the sequence of the Query elements may yield 0 results.

tbonfort commented 12 years ago

Author: tomkralidis Date: 2008/10/08 - 21:55 Bart: thanks for the info. Can you post an example of some sort to clarify?

tbonfort commented 12 years ago

Author: bartvde Date: 2008/10/09 - 10:23 Hey Tom, I've uploaded a testcase to my website:

http://www.osgis.nl/download/mapserver/bug1287.tgz

it contains 2 shapefiles, a MAP file and 2 Python scripts, as well as the output which I am getting from the two scripts. As you can see the sequence of the Query tags will change the output (this is the only difference between the two Python scripts).

If I remove the following line in mapwfs.c all is fine:

bbox = map->extent = ext;
tbonfort commented 12 years ago

Author: tomkralidis Date: 2008/10/15 - 04:40 Bart: if I comment out that line in mapwfs.c, the problem as originally specified in this ticket comes back.

I also looked your testcase. The desired behaviour would be to set the bbox on a per layer basis, as the case of querying against > 1 typename in a !GetFeature case, per your testcase.

Assefa: any comments here on approach? Thinking out loud:

tbonfort commented 12 years ago

Author: bartvde Date: 2008/10/15 - 08:46 Hey Tom, indeed I was thinking the same, aggregate the bbox's of the several layers from the request. Though it's kind of a shame that a WFS attribute query will end up with an unnecessary spatial query as well, but I guess there is too much to be done to change that ....

If different BBOX's are specified in the request, does the code not go through msQueryByRect with the bbox of the layer?

if(msQueryByRect(map, -1, bbox) != MS_SUCCESS) {
tbonfort commented 12 years ago

Author: assefa Date: 2008/10/16 - 15:35 Sorry to be late on this. Couple of notes on this:

I feel the extent value we advertise in the capabilities document for each layer should be able to read a map level metadata (ows/wfs_extent) that takes precedence on the layer->extent values being used. This would allow users to limit queries on all the layers to a certain extent. Of course each layer can override this by specifying a layer level ows_extent metadata.

Aggregating could be one way of doing it. Another possibility is to do the msQueryByRect on each layer (using the advertised extents).

Here are the diifrent scenarios:

Comments?

tbonfort commented 12 years ago

Author: assefa Date: 2009/11/20 - 23:14 Added a patch that I believe fixes the initial problem mentioned by tom. It should also work fix the bug reported by Bart. Will commit/add more tests after 5.6

tbonfort commented 12 years ago

Author: assefa Date: 2009/12/03 - 20:50 test case to use for msautotest: results for hits are different when the layer's order is inverted (which should not happen). Note that there is a bug in the hits count #3224 to be fixed before testing this.

http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=popplace,province&resulttype=hits

http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=province,popplace&resulttype=hits

tbonfort commented 12 years ago

Author: assefa Date: 2009/12/08 - 03:30 patch added in trunk (adff6a2bca5344b9e6b9394d7cafdab7a4035fd0 (r9597)) msautotest added 98f53c000f3614d498f6acb010976dd6c8b85379 (r9599)

tbonfort commented 12 years ago

Author: bartvde Date: 2009/12/08 - 14:10 Verified, works fine.