Generalize map service code to handle non-venues

[benglass]

Right now all the map server code is Venue oriented. There isn't really a reason this needs to be the case and if we want the code to be re-usable for other directories it should be generic.

Instead there should be some kind of ResultInterface that the map can handle which would have methods such as

getType // enum Venue, Event, etc. getLabel getLat getLon

We would also need some concept of a renderer for different types of results that could build the summary and detail html for the hover (summary) and click (detail) popups.

[eflynn]

I did some work with Google Maps powered by a Google Fusion Table. That was very interesting and amazing technology. I also deduped the venue data and removed some of the businesses that have gone out of business.

However, my main concern that stalled me is the server situation. Last week I got access to a VM that Brian Dewey set up. There is very little that is running on it. I set up an Apache virtual host, appropriate permissions, and installed Wordpress. But here's the problem: many times the pages take 20 seconds or more to load. I emailed Brian Dewey and he wrote back, "I think it's just the firewall at cctv doing some qos and other stuff."

No one can expect to have a kickass website with a server that is put together with duct tape. There are many hosting providers out there that would be more than happy to provide a VPS for probably the same amount it costs to power the server rack at Big Heavy World. I think maybe it's a good time to impress on that if anyone wants the project to move forward.

[benglass]

Yeah that is unfortunate regarding the load time. The way they have their apache configured is not ideal either as everything is under /var/www when really it should be broken out into separate home directories for each hosting account/subdomain with unique ftp/ssh users instead of having a single root user that is being given out to everybody.

Last night I updated the code to add the ability to draw arbitrary polygons on the map and have that load venues within the polygons you drew. It only works for polygons and squares right now (not circles) but it would probably be easy to get it working with circles since it appears thats something supported by elastic search. It was almost no work to do this because the leaflet draw plugin creates layers with a toGeoJSON method which can then be passed almost directly to an elasticsearch query.

On Thu, Nov 7, 2013 at 11:49 AM, Evan Flynn notifications@github.comwrote:

I did some work with Google Maps powered by a Google Fusion Table. That was very interesting and amazing technology. I also deduped the venue data and removed some of the businesses that have gone out of business.

However, my main concern that stalled me is the server situation. Last week I got access to a VM that Brian Dewey set up. There is very little that is running on it. I set up an Apache virtual host, appropriate permissions, and installed Wordpress. But here's the problem: many times the pages take 20 seconds or more to load. I emailed Brian Dewey and he wrote back, "I think it's just the firewall at cctv doing some qos and other stuff."

No one can expect to have a kickass website with a server that is put together with duct tape. There are many hosting providers out there that would be more than happy to provide a VPS for probably the same amount it costs to power the server rack at Big Heavy World. I think maybe it's a good time to impress on that if anyone wants the project to move forward.

— Reply to this email directly or view it on GitHubhttps://github.com/codeforbtv/live-music-venue-directory/issues/52#issuecomment-27982869 .

[eflynn]

What you suggest is certainly one way to do it, but there's no hard and fast rule saying you have to. I set it up so any user belonging to the www group can write to /var/www. That way you don't need root access to write there, which is good. Note that the Apache web process runs with the www-data uid, so some directories need to be chown'd to write to them. I should document that.

[bradley-holt]

While it's still a ways out, hopefully the Civic Cloud will address some of these issues. Stay tuned for some big news on this front. In the meantime, I'll ping Jim about this to see if they can do some troubleshooting.

[acrawford]

Hi all,

Andy over at CCTV here. Hi @eflynn Hi @benglass, I think I might have seen you both briefly at the Hack the Stacks event.

As you may or may not know, the server that BHW is currently using is very powerful (12 cores) with plenty of RAM (64G), however a default VM on that server will likely have 1 gig RAM and 1 "core" of execution or in Intel terms 2 Hyperthreads, which is the equivalent of 1 64bit execution thread. To increase the resources available to a particular VM, it is necessary to configure that VM to use more resources. Doing so will require you to know how much memory and compute your application is likely to generate under load (or at least how much you want!), communicate that to Brian, and Jessica or myself so that we can manually edit the qemu/kvm domain in virsh and restart the VM instance to give you more resources.

Think of the VM images as what you would get with a base install of a Linux distribution, they are not meant for web hosting, they are simply a starting point for computing. That means that yes, you need to install your entire software stack on to the VM, database, webserver... etc. and configure it from the CLI. We have no cPanel or web administration system for these systems. The VM's are intended to be taken from their initial state and configured as needed. This BHW server is a temporary home for these VM's and they will eventually move to the civic cloud, or maybe you will just eventually deploy the code to a VM image optimized for wordpress provisioned via open stack on the civic cloud. We aren't there yet, and everything is still being done at a low level.

As for QOS and bandwidth, we have 100Mbit of symmetrical dedicated bandwidth from BT at our offices. We use a Fair Queue policy, meaning that if nothing else in our office is using that Bandwidth, BHW can use almost all of it. The hypervisor running all the BHW VM's is connected at 1Gbps full duplex, so I have a feeling that your VM is simply under resourced.

I just want to impress on everyone that the VM's you are using are not configured to do anything by default, the benefit of that being that you can configure them exactly how you want them configured.

[eflynn]

@acrawford Thank you for your insightful commentary.

I think using VMs and being able to move them around is nice. I am behind the curve when it comes to virtualization.

I'm very comfortable with CLI server administration. The webserver and database are set up, and the system load is low. I'm seeing 8GB allocated.

I was basing my assumptions on beta.bigheavyworld.com, which frequently has a latency of around 20 seconds. If you could fix that, that would be really nice.

[benglass]

@acrawford thanks for the information Andew, I was not aware of the specifics of the hosting environment so that is great to know. I would defer to others with more experience with systems administration in terms of how much resources the VM needs although it shouldnt be very much to serve up a wordpress blog so it would seem strange that 1gb was insufficient if thats all that was being done with the VM right now.

@eflynn

Regarding this specific issue of generalizing the map service, I have pushed some new commits that remove references to venues and generalize to 'results' but there is still a bit of venue specific code (specifically the business_name field). I have also updated the search code to use the bounds of the city if you are doing a city search and only use a distance from city center in the event that nothing is found in the city (for smaller towns in vermont).

At this point with the exception of adding a little more detail about the venues to the on click popup this current setup has feature parity with the existing functionality at http://bigheavyworld.com/businesses.php?t=0 so at this point i will try to create some issues for the few things that are missing in terms of it being a functional replacement for the existing directory and try to get it working embedded on a page on nate's wordpress site (should be easy since its almost all javascript). Items that come to mind are...

• [ ] Add venue details to list results and onclick bubble
• [ ] Database cleanup and de-dup (evan you mentioned you had some work on this?)
• [ ] Admin interface for adding/editing venues

[eflynn]

@benglass See develop branch for new venue data and import script.