Schools directory

[U-238]

In the admin section here (superuser access required), there is a directory of nearly all Australian schools: https://my.robogals.org/topsecretarea/rgteaching/directoryschool/

We'd like to make this available in the front-end, to any exec user (is_staff). The link should be under Workshops > Schools Directory.

The schools directory can be searched (for names of schools or suburbs) as well as filtered by type, level and gender (like in the admin section currently).

Chapters should be able to "star" schools (just like "starring" emails in Gmail). This is at the chapter-level: e.g. If a Melbourne exec stars Laurison Girls School, is appears as starred to all Melbourne exec but not Sydney (or any other) exec.

There should also be a link that one can click to copy the school's details into the already-existing list of schools (Workshops > Schools)

To make it more advanced, it could be made possible to search for schools within X kilometres of a certain postcode. This would be very helpful for organising rural and regional trips. But this feature can be added later, after the basic version has already been released.

[U-238]

Show 50 schools per page

[yfcheung]

There should also be a link that one can click to copy the school's details into the already-existing list of schools (Workshops > Schools)

Q: rgteaching/models:

class School: ... chapter = models.ForeignKey(Group) ...

what should the new school.chapter should be when converting a DirectorySchool to a School?

If school.chapter should be request.user.chapter then two users from different chapters may copy the same school twice to two different chapter. (It seems ok)

If school.chapter should be derieved from address, then ... (may be google map api?) user from chapter a can not copy schools from chapter other than a.

[yfcheung]

Advanced features: Earth shape: Ellipsoids

convert difference of latitude to distance

start with the equation for ellipsoid in 2D, (x,y), theta is = arctan(y/x) where y and x is a point on ellipsoid devise: x = f(theta) y = f(theta)

I think: square(dx / d(theta)) + square(dy / d(theta)) = square(something) Integrate(something) over the theta difference = something2 (possibly distance) over theta difference

theta can be written in terms of latitude.

Similar approach can be use to calculate shortest path between two point in a 3d ellipsoids, once shortest path is known (Need to refresh on calculus)

[U-238]

Answer to your first question: school.chapter should be request.user.chapter

Answer to your second question: I think 2D space would suffice for finding schools nearby a certain place, especially if the latitude and longitude is definitely available.

[yfcheung]

Yeah, I just realise latitude and longitude, that are precise to a degree of practical use, is classified as national security and hence not available. Instead google provide the following url for this purpose for good will for the public:

http://maps.googleapis.com/maps/api/distancematrix/json?origins=Vancouver+BC|Seattle&destinations=San+Francisco|Victoria+BC&mode=bicycling&language=fr-FR&sensor=false

I intend to use in the following ways:

    distance = int(request.GET['distance'])
    origin = request.GET['origin']
    mode = 'driving'        
    sensor = 'false'        
    data = {}               
    data['origins'] = origin                                                                                                                                                                                                                                              
    data['destinations'] = 'Claremont TAS|Abbotsford VIC' 
    data['mode'] = mode     
    data['sensor'] = sensor 
    url_values = urllib.urlencode(data)
    url = 'http://maps.googleapis.com/maps/api/distancematrix/json'
    full_url = url + '?' + url_values
    data = urllib2.urlopen(full_url)
    result = json.loads(data.read())

[yfcheung]

Oh yeah, I strongly think, given the efforts invested nowadays, 2D would not be sufficient.

I am sorry for resorting to ugly analogy:

Sir Humpphrey Appleby said: To excel in a trivial task, a highly trained philosopher is needed. To occupy an untrained mind, any meaningless task will do.

While I am not looking for an exact cambridge solution, non am I looking for a fast approximating oxford solution, but then the primitive 3D idealistic solution is nevertheless necessary, and I think that is reasonable.

(By the way, I believe numerical method is the king)

[yfcheung]

data = urllib2.urlopen(full_url)

problem: When network is not reachable urllib2.urlopen() block the whole server.

suggestion: data = urllib2.urlopen(full_url, timeout = 1)

refinement: what should the timeout value be?

[U-238]

Distance between two points: http://www.codecodex.com/wiki/Calculate_distance_between_two_points_on_a_globe

Use Google API to get the longitude and latitude of the school, and save in the database.

[yfcheung]

http://www.ga.gov.au/earth-monitoring/geodesy/geodetic-techniques/distance-calculation-algorithms.html

provide the following algorithm to calculate two point on earth: L1 = latitude at the first point (degrees) L2 = latitude at the second point (degrees) G1 = longitude at the first point (degrees) G2 = longitude at the second point (degrees) DG = longitude of the second point minus longitude of the first point (degrees) DL = latitude of the second point minus latitude of the first point (degrees) D = computed distance (km)

TERM1 = 111.08956 * (DL + 0.000001) TERM2 = COS(L1 + (DL/2) TERM3 = (DG + 0.000001) / (DL + 0.000001) D = TERM1 / COS(ARCTAN(TERM2 * TERM3))

plan: 1: Filter out schools with D (derived from above) greater than the search distance. 2: Number of schools left can be fit into a single query to google for finding individual driving distance. 3: sent it to google 4: display the real driving distance from google

url max length problem: divide and conquer

overall performance is slow!!!!!!!!!!! suspect:

        schools_list = schools_list.filter(id__in=sch_list)
        l = [] 
        for key in sorted(sch_ordering, key=sch_ordering.get):
            l.append(schools_list.get(pk=key))      <-------------------
        schools_list = l

Features: 1: Returns the top 50 results.

[U-238]

Use Google only to get lat/lon initially

Increase timeout

Ensure that "filllatlngschdir" will only send max 100 queries in 10 seconds - maybe, delay 0.1 seconds after each one

Only get lon/lat in save() if it isn't already there

In admin, allow manual editing of lon/lat

Remove the distance column when its not relevant

Put comment where you use the algorithm: this algorithm is specific to Australia; will need to be changed if other countries are added to the schools directory

[yfcheung]

Google map added just below the control panel (in an ad-hoc manner: style="position: absolute;")

Quota description: 25 000 free static map requests per application per day. https://developers.google.com/maps/documentation/staticmaps/index#Limits

It seems that the quota applies to individual clients, not the server. If that is the case, the top 20 results can be pinpointed in the same map (due to max url size restriction)

[U-238]

Yeah seeing the schools on a map would be very cool!

[yfcheung]

That is funny, I was coming around to that view point myself.

Will do.

[yfcheung]

Added the google static map.

Will look into the Google Maps JavaScript API v3. (Need a bit more time, meanwhile the static map can be a temporary fixed)

[yfcheung]

Confession:

target suburb latlng url look up: done at the server with limit (25000 per day) map instances: 25000 per day (done at client, tracked by key)

suburb latlng lookup can be switch to client to neutralise one bottle neck by using Json. (I think I will take a rest before attempting this).

[U-238]

I think keeping the suburb latlng lookup on the server-side is ok, because there's not going to be 25,000 requests per day.

Merged and tested!

This is VERY cool! I really like the map, and it's even possible to click on the school on the map to see its details. Just so so awesome! Great work!