codeforboston / finda

Generic "find-a" app for geographic datasets
http://codeforboston.github.io/finda/
MIT License
57 stars 99 forks source link

Too slow when data set is large #146

Open alexsoble opened 9 years ago

alexsoble commented 9 years ago

When larger datasets come into play, the loading/filtering is too slow for the app to be practical. Child care finder: http://codeforboston.github.io/child-care-finder/. Testing from home, takes around 20 seconds of loading before I can interact and another ~20 seconds when I click on a filter or map. Maybe caching could help here? Or loading less data? The map initializes with a view of the Boston area, but I have a feeling it's pulling in every location in MA before the user can interact with the map.

child-care-finder

radumas commented 9 years ago

Do you now how many datapoints are in that set? I'm having a hard time testing finda with my own dataset (originally 924, now down to 100) and seeing whether the interminable loading is a bug I've introduced or the dataset.

alexsoble commented 9 years ago

Based on GitHub's data.geojson visualization, it looks like there are 8,260 points in the child care finder set.

djq commented 9 years ago

Would something like http://turfjs.org/ help? You'd still need to load the data (somewhere) but it could be filtered to the map bounding-box, which would speed up rendering

paulswartz commented 9 years ago

There is some filtering already; if you let the data load you can see a "limit to map" check box in the filters. The way it works is by loading the data, and then using the map bounds to limit what is displayed by the map/list.

I recently did some work around making the performance better, but I'd love for someone else to look into it too!

radumas commented 9 years ago

A thought: Could it be the enabling text-searching that builds an index when the site first loads that takes a lot of time for larger datasets?

paulswartz commented 9 years ago

There's also the index we build for the filter on the sidebar, and adding that many points to the map in the first place.

On Thu, Mar 12, 2015 at 2:09 PM radumas notifications@github.com wrote:

A thought: Could it be the enabling text-searching that builds an index when the site first loads that takes a lot of time for larger datasets?

— Reply to this email directly or view it on GitHub https://github.com/codeforboston/finda/issues/146#issuecomment-78550848.

alexsoble commented 9 years ago

@paulswartz @radumas -- Will you be at CfB this Tuesday? I'll be there, let's pair on this together if we can both make it!