Display Bicikelj information

[mitar]

Display information about Bicikelj locations, free spaces and available bicycles on each location. Both on map and in a panel.

[lazarm]

Is it okay to fetch this information from these sites : http://www.bicikelj.si/service/stationdetails/ljubljana/27 http://www.bicikelj.si/service/carto

The good thing is it's xml file, so information is easily fetched, but the problem could be if URL changes (it already did once from .../stationdetails/27 to .../stationdetails/ljubljana/27) ?

[mitar]

Perfect! I didn't know that they publish this in XML. Is this official API? Is there any official documentation for it? Do you know what is "ticket" and "open"? I don't think URL changed (as far as I read around the Internet about the API). Anyway, don't be concerned about that. You cannot do much about it. But you should make sure that your code fails if URL changes so that we will notice. (Aha, ticket means "if it supports the option to rent by credit card".)

Is this information real-time? So the question which opens is: should we prefetch this information in background or should clients access it directly through JavaScript? Can clients access it? I think not because XML is on other server and they don't support JSONP (or do they)?

But it is so visible that it was developed for Paris. "arrondissements". :-)))

Useful reading (searching Google for "service/stationdetails" helps): *https://ocfnash.wordpress.com/2009/10/31/dublin-bikes/

So, make a background task which fetches data every minute (yes, polling, great for them!). And then display this data in the panel and on the map for bike station around nodes.

Also add pushing of new free/empty station data to users. So the main information we want to display is not where are locations but what is state there. And we should update this in as real-time as we can. (You have timestamp in station details so you can know if something changed and you have to push.)

One more thing. Store all previous states of the stations in the database. Don't delete them. It could be interesting statistics out of it through some time. :-) (You can of course convert timestamp to data time data type.)

Check how horoscope is done. It is very similar what you are doing here to that. It also has XML parsing and background job. And it also stores old data.

[lazarm]

Yes, it's their official API, I've found it on some apple forum site :) "open" has a binary value, 1 if the station is open and 0 if it's closed (apparently, this can happen). My question is: how many bike station markers should be displayed on the map and should i calculate k nearest stations from longitude and latitude between the node and each station available (and also define max distance, so it wouldn't display a station in Siska if the user is using a node somewhere in Kranj). Should there be just markers of the stations on the map and if the user clicks it, the information would be displayed on the panel or should the panel automatically display information from the nearest station? Or what would be the best solution?

[mitar]

"open" has a binary value, 1 if the station is open and 0 if it's closed (apparently, this can happen).

OK. Then also use this information. We can two types of icons. And also display that station is closed.

The idea of our panels is, that we know where the user is and we display relevant information for the user. In this case, which stations are around the user and what is state of free spots and available bicycles there. As a bonus, we display this also on the map. (This is a bit tricky because API for this is not yet done. You should not modify map panel, map panel should just provide the map and you should hook into it. The problem is, that student working on this has not yet finished it.)

The map we are displaying in the panel is a static one, user cannot move around or zoom. So it is exactly known which are is displayed. And you can use exactly the same are to limit displayed content in the panel to.

You don't have to calculate this yourself, but you can use GIS queries to MongoDB.

Later on, we will also have a full screen map (#267), but don't concern yourself with that. This can be some other ticket later on.

how many bike station markers should be displayed on the map and should i calculate k nearest stations from longitude and latitude between the node and each station available

No. Not k nearest, but all in the bounds of the map, sorted by distance (with distance displayed in the panel).

Should there be just markers of the stations on the map and if the user clicks it, the information would be displayed on the panel or should the panel automatically display information from the nearest station? Or what would be the best solution?

No, display close stations in the panel and in the map automatically. You can do some GUI effect that if user hovers over the entry in the panel the marker on the map is highlighted and vice-versa. Click on the marker or panel entry could open official page for that station, if it exists?

[lazarm]

Is there an easy way to get the bounds of the map with specific zoom level (distance between the node and the farthest point on the map in latitude and longitude)? And yes, I did search for an answer for more than an hour, unsuccessfully :)

[mitar]

In JavaScript yes. But in Python, not really. Because this is Google API and Google's zoom levels. (Maybe they are standard, but I doubt.)

So, as map has a static zoom and this means that bounds are fixed, too, you could measure them once and hard-code them into the code. The best way would be to move zoom-level and measured bounds to top-level constants of map panel Python code. And then use that in your panel.

The other way is to go in the other direction. Where you define bounds we want that map is in in Python code, pass this to JavaScript, and then use those values to render the the map within those bounds with Google API (fitBounds method). The problem with this is, that it displayed bounds will be larger than requested ones. So there will be some gap around. And we would like that there is no gap. (Because we will be providing markers only for area inside of the requested bounds, so gap would be empty, useless.)

So the best approach would be probably to combine this two approach, to fake a bit. To define bounds in Python (bounds which are 100m or something wide, the idea is that the static map can display/encompass an usable part of the town, for example, Presern's square or something like that, play a bit, but I think current zoom level is somewhere there already) and then use them to set bounds and zoom and not hardcode the zoom. But that those bounds you define are defined in a way you measured from what Google displays, so that you know that there will be no gap. You can iterate few times until you get that. You can also define the expected zoom level you check in JavaScript after fitting zoom so that you make sure things are OK (for example, if somebody resizes the map canvas in CSS, fitting would start to work differently). Also make a comment in the Python code how you got those bounds values.

[lazarm]

Okay, I'll deal with this later. But I have found a formula for computing bounds and I think it works quite good: https://groups.google.com/forum/?fromgroups=#!topic/google-maps-js-api-v3/Q5WHkOr-A38

So if we know the zoom level, pixel width (in our case it's 450) and we take constant_at_60_degrees, the difference between maxLat and minLat is constant_at_60_degrees*450/(2^zoom_level). And then we halve this distance to get the difference between nodeLat and boundLat. I checked the formula for trebinjska 4 node and it was mistaken for about 0.001 of latitude, but I hope that's not so much. There are also inconsistencies in our node database (it says that latitude of trebinjska 4 is 46.0793969971, but when you type the address in google maps, it says it's 46.07796 ?).

Anyway, my next question is how the data should be stored. BicikeljStation model should have attributes that never change (so name, address, number, position), and the ones that change when we are updating information about stations (so available, free, open, ticket and timestamp). But when and where should the main data (the one that never changes) be stored in database? Does horoscope panel store data and where? Because the panel shows data error and I can't retrieve any horoscope objects. So should I make a file data.py like in nodes app, with a list of all stations (which are - like nodes - class-based) and their static attributes. And then when fetching info about a certain station, if updating fails because that station is not in our database, we would create an object with attributes of the station in the data.py with the same number, and also with dynamic attributes, which we have just fetched and then save the object in database? But I don't know, this part of code would be executed only 32 times (the number of stations), so probably this is not so good idea.

Is it a good way to make a background task, which in fixed time intervals fetches info of all stations in neighbourhood and updates the attributes of the stations which have a timestamp that is different from the fetched one. Can i declare a global variable NODE in panel file, which gets its value inside get_context, where I can retrieve a node with self.request.node? I would then use this variable in a task, so it can calculate nearest stations.

Sorry for so many questions, I just think it's a lot of coding and I don't want to make a pull request with 90% of code totally wrong :)

[mitar]

So if we know the zoom level, pixel width (in our case it's 450) and we take constant_at_60_degrees, the difference between maxLat and minLat is constant_at_60_degrees*450/(2^zoom_level). And then we halve this distance to get the difference between nodeLat and boundLat. I checked the formula for trebinjska 4 node and it was mistaken for about 0.001 of latitude, but I hope that's not so much.

You can use this formula to help you get proper hard-coded values for bounds you then use to "fit" the map into. But please leave to Google Maps API to fit into it.

There are also inconsistencies in our node database (it says that latitude of trebinjska 4 is 46.0793969971, but when you type the address in google maps, it says it's 46.07796 ?).

Because locations are not of the address but of the node. Building can be large on the same address and there are many balconies, window shelfs and roof locations where node can be. We mostly put nodes on the map based on the satellite image, not address.

Does horoscope panel store data and where?

Of course, see models.py.

Because the panel shows data error and I can't retrieve any horoscope objects.

Have you run manage.py script to populate the database? Or wait for one day that background job did that? :-)

So should I make a file data.py like in nodes app, with a list of all stations (which are - like nodes - class-based) and their static attributes.

No, you should definitely not. This is done like that just for testing purposes, so that you do not have to have access do real-time database data when developing.

And then when fetching info about a certain station, if updating fails because that station is not in our database, we would create an object with attributes of the station in the data.py with the same number, and also with dynamic attributes, which we have just fetched and then save the object in database?

OK. This is simply a bad design. If you have things which change, they should be in the database to begin with. Why data.py?

Is it a good way to make a background task, which in fixed time intervals fetches info of all stations in neighbourhood and updates the attributes of the stations which have a timestamp that is different from the fetched one.

Yes. But not update it in the database, but store a new entry into the database with new data. In this way we can also use this data for statistics how the stations are being used.

Can i declare a global variable NODE in panel file, which gets its value inside get_context, where I can retrieve a node with self.request.node? I would then use this variable in a task, so it can calculate nearest stations.

I think you have some concepts/understanding very wrong. You are mixing apples and bikes here. Global variable in panel file means a variable global to one process. And this probably a process serving the frontend, the client. Even if you store anything in that variable, how will the other process (maybe running on a completely other server at other datacenter) access that variable?

Please read code of horoscope panel again and again until you understand it completely. Of course, the prerequisite is to make it working for you. So does it work for you? If not, you don't understand it.

[mitar]

So: main point is: don't be afraid of storing data in the database. This is what they are for. We want all data, we want to store it, current data, past data, everything. So you simply make a background task which regularly fetches status of all bicycle stands and store all that. And this is done for all stands and is not important where any client connected to our website is. You simply transfer data to our database, so that we can easier do queries to it.

And then, in the frontend, when user connects, you based on her location, fetches the most recent data from our database and display that.

This is it.

[lazarm]

How do you quote a comment?

Do I have to run manage.py updatehoroscope --verbosity=2 ? I keep getting an error: Updating horoscopes... ... ... raise ImproperlyConfigured("settings.DATABASES is improperly configured. " django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly co nfigured. Please supply the ENGINE value. Check settings documentation for more details.

I also forgot how to run celery :-). "manage.py celeryd --loglevel=info --concurrency=1 --maxtasksperchild=10 --beat" doesn't do it for windows, because -B option doesn't work, and it says I should run celerybeat as a seperate service. However, when I do it, so "manage.py celerybeat", it says celerybeat.pid already exists.

Okay, I understand now, because upsert=True, the horoscope data isn't always updated, but also stored, if the object with given attributes doesn't exist. That's why I was confused, I didn't know where the data is stored.

[mitar]

How do you quote a comment?

Like in e-mails. :-)

> How do you quote a comment?

I keep getting an error:

Are you using exactly those versions of packages that are in the requirements.txt file?

You have to be running celery daemon for updatehoroscope to work.

However, when I do it, so "manage.py celerybeat", it says celerybeat.pid already exists.

Probably from some time before. You can try to delete it. But make sure that you are not really already running it.

[lazarm]

When pushing new users in get_online_users, we sent an update with action JOIN or PART. If I understand correctly, JOIN creates a new "li" tag in html file inside "ul"userlist "/ul" tag with provided user info, and PART removes it. In bicikelj panel we don't need to add or remove a user from a list, but just replace an old data (available and free) with a new data. Should I also put "available" and "free" inside "ul" tags and then, when the new data arrives, first remove (so PART) "li" tag with old data and then add a new tag with JOIN?

[mitar]

So you will display in a list all stations in vicinity of nodes. And you will be sending to "home" channel all updates about stations' states. When update will come, you will update relevant entry in the displayed list.

How it is done with online users is not really the best way. I would not copy too much from there.

But first come to the stage where updates will be coming to the client. How will that be displayed/how you will manipulate display, this is quite easy then. (I propose to use jQuery data() to attach information about which station is displayed where in the list and then update just that station.)