Closed GoogleCodeExporter closed 9 years ago
Currently OLIVER uses the Bing geocoding service. According to our counts
approximately 200 geocodes per day go to this service. The service provides
very flexible address parsing. Misspellings are often corrected and landmark
names (Fenway Park) are accepted and valid. The service was chosen to provide
region-wide geocoding as the original sponsors of the OLIVER reworking from
Java Web Start to OpenLayers wanted to use the application across New England.
However, the service is not based on the most up-to-date roads data that
MassGIS provides.
Therefore we would like to provide an alternative geocoding option on the UI.
(presumably one or the other would be turned on although in certain cases both
could be turned on). Since we probably can't provide such flexible parsing
(can we? if Soundex is available it might be good to use it) we propose that
instead of a free-form input box that we add a tool with an appropriate icon
(maybe a pushpin) that pops up a form where the user would be guided to fill
out certain fields: Address (includes street number, street name and street
suffix in any format), City, Zip, State. Address is mandatory and either City
or Zip is mandatory. State is optional. The request is composed into an XML
request and posted to the ArcGIS Server geocoding service described here:
https://wiki.state.ma.us/confluence/display/massgis/ArcGIS+Server+-+Geocoding
using the GeocodeAddress method (which returns one result, not candidates).
Similar to the Bing service a window should be popped up before zooming which
displays the information to the user and offers buttons "zoom to center point",
"zoom to region", "cancel" just as Bing does and with the same logic (distances
or zoom level) as Bing.
Original comment by Aleda.Fr...@state.ma.us
on 17 Dec 2012 at 4:47
Requests to geocode addresses must contain a street address including address
number and either a city or zipcode. The request may contain all 4 elements of
address, city, zipcode, state.
Street types may be written out or abbreviated, with punctuation or without.
Samples:
- 1 Ashburton Pl, 02108
- 1 Ashburton Pl, Boston
- 1 Ashburton Pl, Boston, 02108
- 1 Ashburton Pl, Boston, MA 02108
- 1 Ashburton Place, Boston, MA 02108
- 1 Ashburton Pl., Boston, MA 02108
- 100 Willard Rd, 03071
- 100 Willard Rd, New Ipswich
- 100 Willard Rd, New Ipswich, NH
- 100 Willard Rd, New Ipswich, NH, 03071
- 100 Willard Rd., 03071
- 100 Willard Road, 03071
Currently intersections are not supported (example: Autumn St and Wedgemere Rd
02148)
We may want to configure the form to have the address number in a box which is
mandatory. Have the street name in a box which is mandatory. Have the street
extension in a box which is optional. That way we don't need to worry whether
both number and street name are there.
A "bad" address will either return NaN values for X and Y or an address with a
very low score. It would be great to also show the score on the pop up window
for the user. If NaN occurs the message in the info box should be "No address
found".
Original comment by Aleda.Fr...@state.ma.us
on 17 Dec 2012 at 5:03
We were able to have both a zoom to center point and a zoom to region because
Bing returns both values (a point and a bbox). In the case of the MassGIS
geocoding, it only returns a point. We could either drop the bbox option or we
could create our own bbox which would be centered around the geocoding hit and
set at a certain zoom level.
Original comment by cpl...@gmail.com
on 24 Jun 2013 at 1:48
I don't think the bbox option is necessary actually.
I think this option is sufficient: "we could create our own bbox which would be
centered around the geocoding hit and set at a certain zoom level".
We don't want the user to be right on top of the point.
Original comment by Aleda.Fr...@state.ma.us
on 24 Jun 2013 at 1:55
I think I see this as a separate button, that we can hide/show.
I think most viewers would prefer the Bing geocoding because it can intake so
many more flexible "locations" and interpret them. However, for some viewers
we may prefer to hide the Bing box and substitute the "MassGIS geocoding
button" which I'm thinking will have to pop up a box with:
Address:
Municipality:
State:
Zipcode:
Where Address is required
Either municipality OR zipcode is required
and State is optional
Original comment by Aleda.Fr...@state.ma.us
on 24 Jun 2013 at 2:17
Note - it would be good to have the service posted URL to be configurable.
Because Gabriela Laird, our ArcGIS Server admin will likely roll out other
geocoding services in the future with new names. So we might want to change
from posting to
http://gisprpxy.itd.state.ma.us/MassGISGeocodeServiceApplication/MassGISCustomGe
ocodeService.asmx
to posting to a different service name. This service name does basic geocoding.
She I believe is working on a service to do more complex geocoding (which
cascades trying to use different data sources). For backwards compatibility
she will likely leave
MassGISGeocodeServiceApplication/MassGISCustomGeocodeService.asmx named as is
and make a new service with a new name for new functionality. If she makes a
MassGISCustomGeocodeService_Cascasding.asmx in the future we may want to switch
to using it in OLIVER. I would have the service be a parameter that can be set
viewer-by-viewer (in other words not hard-coded in the map.js).
Original comment by Aleda.Fr...@state.ma.us
on 24 Jun 2013 at 3:18
After getting this rolling here, it doesn't make sense to me for us to offer a
zoom-to-region for the hit coming back from this svc. It only returns a point,
so shouldn't it behave like Bing when it finds a point and we zoom to it?
The way it is setup w/ a Bing point zoom is that it centers on the lon,lat and
zooms it to a scale of 1000.
Original comment by cpl...@gmail.com
on 24 Jun 2013 at 4:02
That would be fine.
Original comment by Aleda.Fr...@state.ma.us
on 24 Jun 2013 at 4:11
See comments for r495.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 12:45
OK, trying to test this. First I had to change the geocoding URL, because of
current limitations of which servers the maps server can talk to. Then I had
to add that URL to the proxy.cgi allowedHosts list (at first I was getting "bad
gateway"). Now, I get a 500 unknown error. The posted request looks good:
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GeocodeAddress xmlns="http://tempuri.org/">
<Address>1 Ashburton Pl.</Address>
<City></City>
<ZipCode>02108</ZipCode>
<State></State>
</GeocodeAddress>
</soap:Body>
</soap:Envelope>
so I need to look at the Apache error log.
I get this:
[Tue Jun 25 10:42:07 2013] [error] [client 146.243.44.177] Some unexpected
error occurred using proxy.cgi. Error text was: HTTP Error 415: Unsupported
Media Type, referer: http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 2:42
I'll send you the current proxy.cgi
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 2:46
Do you have this running somewhere that I could hit it?
I didn't write that proxy.cgi, so I may hand this off to Saul.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 2:53
Yes, it's available at
http://maps.massgis.state.ma.us/cgi-bin/proxy.cgi?url=http://170.63.93.155/MassG
ISGeocodeServiceApplication/MassGISCustomGeocodeService.asmx
That's what OLIVER is showing in Firebug
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 2:54
I'm sorry, I meant the mapper.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 2:57
Sorry, http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 2:58
Here is a complete and working wget that is going through your proxy.
wget --header "Content-Type:text/xml" --post-data='<?xml version="1.0"
encoding="utf-8"?> <soap:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body>
<GeocodeAddress xmlns="http://tempuri.org/"> <Address>1 Ashburton Pl</Address>
<City></City> <ZipCode>02108</ZipCode> <State></State> </GeocodeAddress>
</soap:Body> </soap:Envelope>'
'http://maps.massgis.state.ma.us/cgi-bin/proxy.cgi?url=http://gisprpxy.itd.state
.ma.us/MassGISGeocodeServiceApplication/MassGISCustomGeocodeService.asmx' -O
response.xml
I can run that from my box here, and it's fine (a valid response.xml is
created).
Can you run it from maps.massgis.state.ma.us to see if something weird is going
on?
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 3:10
Yes, that was going to be my next move...
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:13
I copied and pasted what you had above to a command line on maps and I get a
good response:
<?xml version="1.0" encoding="utf-8"?><soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GeocodeAddressResponse
xmlns="http://tempuri.org/"><GeocodeAddressResult><X>236097.54354225798</X><Y>90
1058.865187255</Y><MatchedAddress>1 ASHBURTON PL, BOSTON, MA,
02108</MatchedAddress><Score>100</Score></GeocodeAddressResult></GeocodeAddressR
esponse></soap:Body></soap:Envelope>
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:16
Please give r498 a shot.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 3:28
I get the same 500 error when I try
http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php after replacing the
map.js from r498...
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:33
Do you see the change at http://maps.massgis.state.ma.us/map_ol_to5b/js/map.js?
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:34
Is this at all important?
Firebug shows this:
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GeocodeAddress xmlns="http://tempuri.org/">
<Address>69 Autumn St.</Address>
<City></City>
<ZipCode>02148</ZipCode>
<State></State>
</GeocodeAddress>
</soap:Body>
</soap:Envelope>
In the post tab, but not the <?xml version="1.0" encoding="utf-8"?> at the top
of it... should it?
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:37
Well, at the maps command line this also seems to work, without the extra line
at top:
wget --header "Content-Type:text/xml" --post-data='<soap:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body>
<GeocodeAddress xmlns="http://tempuri.org/"> <Address>1 Ashburton Pl</Address>
<City></City> <ZipCode>02108</ZipCode> <State></State> </GeocodeAddress>
</soap:Body> </soap:Envelope>'
'http://maps.massgis.state.ma.us/cgi-bin/proxy.cgi?url=http://gisprpxy.itd.state
.ma.us/MassGISGeocodeServiceApplication/MassGISCustomGeocodeService.asmx' -O
response3.xml
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:40
Yes, I see the change. And it was the <?xml> part that I removed from the
request so that it would match how all the other POST requests were being made.
Try looking at normal query and you'll see that <?xml> isn't part of the POST
itself.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 3:42
Maybe I can add some print to log statements in the proxy.cgi...
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:44
Is there any chance that the proxy is throwing stuff out because of <soap>?
I'll ping Saul and ask him to join in.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 3:53
I asked the proxy to print some stuff into the error log:
[Tue Jun 25 11:55:53 2013] [error] [client 146.243.44.177]
{'HTTP_AUTHORIZATION': '', 'HTTP_REFERER':
'http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php', 'SERVER_SOFTWARE':
'Apache/2.2.3 (Red Hat)', 'SCRIPT_NAME': '/cgi-bin/proxy.cgi',
'SERVER_SIGNATURE': '<address>Apache/2.2.3 (Red Hat) Server at
maps.massgis.state.ma.us Port 80</address>\\n', 'REQUEST_METHOD': 'POST',
'SERVER_PROTOCOL': 'HTTP/1.1', 'QUERY_STRING':
'url=http://gisprpxy.itd.state.ma.us/MassGISGeocodeServiceApplication/MassGISCus
tomGeocodeService.asmx', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin',
'CONTENT_LENGTH': '364', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1;
rv:21.0) Gecko/20100101 Firefox/21.0', 'HTTP_CONNECTION': 'keep-alive',
'SERVER_NAME': 'maps.massgis.state.ma.us', 'REMOTE_ADDR': '146.243.44.177',
'SERVER_PORT': '80', 'SERVER_ADDR': '170.63.93.152', 'DOCUMENT_ROOT':
'/opt/massgis/wwwroot', 'HTTP_PRAGMA': 'no-cache', 'HTTP_X_REQUESTED_WITH':
'XMLHttpRequest', 'SCRIPT_FILENAME': '/opt/massgis/cgi-bin/proxy.cgi',
'SERVER_ADMIN': 'root@localhost', 'SCRIPT_URI':
'http://maps.massgis.state.ma.us/cgi-bin/proxy.cgi', 'HTTP_HOST':
'maps.massgis.state.ma.us', 'SCRIPT_URL': '/cgi-bin/proxy.cgi',
'HTTP_CACHE_CONTROL': 'no-cache', 'REQUEST_URI':
'/cgi-bin/proxy.cgi?url=http://gisprpxy.itd.state.ma.us/MassGISGeocodeServiceApp
lication/MassGISCustomGeocodeService.asmx', 'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'GATEWAY_INTERFACE': 'CGI/1.1', 'REMOTE_PORT': '57332', 'HTTP_ACCEPT_LANGUAGE':
'en-US,en;q=0.5', 'CONTENT_TYPE': 'application/xml; charset=UTF-8',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate'}body: <soap:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body>
<GeocodeAddress xmlns="http://tempuri.org/"> <Address>1 Ashburton Pl.</Address>
<City></City> <ZipCode>02108</ZipCode> <State></State> </GeocodeAddress>
</soap:Body> </soap:Envelope>, referer:
http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php
[Tue Jun 25 11:55:53 2013] [error] [client 146.243.44.177] Some unexpected
error occurred using proxy.cgi. Error text was: HTTP Error 415: Unsupported
Media Type, referer: http://maps.massgis.state.ma.us/map_ol_to5b/oliver.php
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:57
Above coming from
sys.stderr.write(str(os.environ))
and
sys.stderr.write("body: " + body)
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 3:58
Could you try changing line 6883 in map.js from
,headers : {'Content-Type':'application/xml; charset=UTF-8'}
to
,headers : {'Content-Type':'text/xml; charset=UTF-8'}
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 4:02
That worked! My map zooms in and shows a blue marker.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:05
Amazing. That was so totally a shot in the dark, but I'll take it!
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 4:07
The change was submitted in r499.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 4:09
Could you change the text to match the field names. In other words, capitalize
Street and Municipality in "The street field is required along with either the
municipality or ZIP code field. State is optional."
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:10
I tried many different types of addresses and the geocoder seems to work fine.
I'm going to have some other people test it...
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:25
As long as those "somebodies" do not include Emily. She's good at poking
holes! :)
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 4:27
I had only tested Firefox.
Emily tried Chrome and found a bug.
Using Chrome (I have v. 27) I get a window titled "The page at
maps.massgis.state.ma.us says:" with content of "phi2z has NoConvergence"
Then the map zooms to 0 Maple St in Douglas, regardless of the address entered.
We'll test next with IE and Safari.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:43
Same error with IE 10.0
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:45
Same error with Safari (5.1.7)
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:46
If I use IE developer tools it looks like the response is good.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 4:49
Sorry about that. r501 please.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 5:11
Works in all 4 browsers now.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 5:15
I think this would benefit from a "Location search results" window display
before zoom like the Bing does. We could display the MatchedAddress and Score
values. The reason is that there are 2 Salems - one in MA, one in NH - if I
put in Address "20 High St"
Municipality "Salem" and don't fill in State or Zip code which aren't required
how do I know what it did?
The MassGIS geocoding service found the following location:
1 ASHBURTON PL, BOSTON, MA, 0210
Score: 100
"Zoom to center point" "Cancel"
Just no Zoom to region.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 5:21
Please see r504.
Original comment by cpl...@gmail.com
on 25 Jun 2013 at 5:52
I like it, passing to users.
Original comment by Aleda.Fr...@state.ma.us
on 25 Jun 2013 at 6:43
Could we have this new button be false by default?
What I mean is, shouldn't it NOT show up if this paragraph is NOT in
toolConfig.js?
toolSettings.massgisAddressSearch = {};
toolSettings.massgisAddressSearch.status = 'show';
toolSettings.massgisAddressSearch.keyMap = {
// not required
"ctrl":true, // not required if false
"alt": true, // not required if false
"key": "M"
};
toolSettings.massgisAddressSearch.url =
'http://gisprpxy.itd.state.ma.us/MassGISGeocodeServiceApplication/MassGISCustomG
eocodeService.asmx';
Otherwise I have to edit many, many toolConfig.js to say I want to "hide" the
tool. I thought the default was, no mention of tool in toolConfig.js = no tool.
Original comment by Aleda.Fr...@state.ma.us
on 26 Jun 2013 at 2:23
Clarification on #45 - in viewers that don't have a Bing search such as
http://maps.massgis.state.ma.us/map_ol_to5b/911.php, the new MassGIS address
red pin tool doesn't show up. However, in viewer that have the Bing search
(most) such as http://maps.massgis.state.ma.us/map_ol_to5b/moris.php the new
MassGIS address red pin does show up. Even though the only toolConfig I have
added the new paragraph to is OLIVER's.
Original comment by Aleda.Fr...@state.ma.us
on 26 Jun 2013 at 3:34
Sorry, Aleda. I mistakenly lumped the massgis geocoder in w/ Bing. Got Bing?
Got MassGIS. I'll pull them apart.
Original comment by cpl...@gmail.com
on 26 Jun 2013 at 5:54
I think you'll be happier w/ r507.
I moved the eraser button to appear to the right of one or both geolocation
options. If you have at least one geolocation option (either bing or massgis),
the button will appear.
Original comment by cpl...@gmail.com
on 26 Jun 2013 at 6:52
We're getting closer - I see the eraser has moved to the right of one or both
geolocation options - that's fine. I ask for bing only, I get eraser, I ask for
massgis only, I get eraser, I ask for both, I get eraser.
The issue I still have is ... if I have a viewer toolConfig which I don't
touch, don't add any mention of this new massgis geocoding button - I don't
want it to appear in the viewer. I don't want to have to add the paragraph and
say "hide" for 100 files. Isn't that how it has worked in the past - for
example, the external WMS button doesn't appear if I don't talk about it at all
in the toolConfig.
Here's an example - http://maps.massgis.state.ma.us/map_ol_to5b/moris.php and
it's
http://maps.massgis.state.ma.us/map_ol_to5b/js/toolConfig_moris.js
Look! There's a red pin button in MORIS now even though I have not touched
http://maps.massgis.state.ma.us/map_ol_to5b/js/toolConfig_moris.js
Original comment by Aleda.Fr...@state.ma.us
on 26 Jun 2013 at 7:19
I understand what you're saying, but I believe these geocoding buttons behave
like the normal identify:
if (!toolSettings || !toolSettings.identify || toolSettings.identify.status == 'show') {
if (!toolSettings || !toolSettings.massgisAddressSearch || toolSettings.massgisAddressSearch.status == 'show') {
Basically, for each, if you don't have any toolSettings or you don't have a
toolSettings.theWidget or you have it listed as "show", then it shows.
I could change it to behave like you are asking, but it wouldn't be consistent.
Original comment by cpl...@gmail.com
on 26 Jun 2013 at 7:25
Original issue reported on code.google.com by
Aleda.Fr...@state.ma.us
on 17 Dec 2012 at 4:09