search

WasabiFan / 3D-US-Data-Map

3D US Data Map is an in-browser 3D map of the US, split by County or State, with geographies extruded and colored based on a user-chosen formula that can access US census data.
Microsoft Public License
2 stars 2 forks source link

math equations are fairly limited #3

Open btbonval opened 10 years ago

btbonval commented 10 years ago

I couldn't get parenthesis to work. Finding the percentage of 1 and 2 person households might look like this, but it won't work: ( geo.onePersonHouseholds + geo.twoPersonHouseholds ) / ( geo.familyHouseholds + geo.familyHouseholds ) * 100

It seems like geo.property * X does not work, but X * geo.property does work. I could be mistaken.

Logs do not appear to be possible. For example: Math.log( geo.totalPopulation )

I haven't tested this enough yet, but it appears as though negative numbers are not plotted, or maybe they are not handled in an obvious way? 2 - geo.aggregateNumberOfVehiclesAvailible

WasabiFan commented 10 years ago

That simple processor script never worked very well; I can re-write it to make it work much better in all respects.

btbonval commented 10 years ago

That'd be great. It sounds like you already know what you're doing.

I have some friends who did a similar math equation editor which is calculated to render some images. If I were going to approach it, I'd have to look at what they did to get an idea of how to approach the problem.

WasabiFan commented 10 years ago

I replaced what was there before with a relatively simple script that's safer (doesn't directly eval() equation), and is much more universal. I won't close this issue because there is definitely still room for improvement, but it's a start.

btbonval commented 10 years ago

Parens are working, math operators (+,-,/, etc) are working, the only thing missing as far as I'm concerned are the built-in Math functions, so I wrote a little hack for them. Pull request coming shortly.

I'm not sure what other features are desired, but I'm quite content now.

WasabiFan commented 10 years ago

I have made it possible to call functions without the Math reference. I did a bunch of testing on it, and with the combination of the ability to do things like log as well as the error checking to make sure that the equation is valid, it's working pretty well. Any other ideas to extend the computational functionality?

btbonval commented 10 years ago

The only idea I have is variables to make data entry easier. "totalPopulation" or "P0010001" are not convenient if one is writing an equation like (x - y) / (x + y): (P0010001 - P0010002) / (P0010001 + P0010002)

It might be nice if there was a way for the user to define variables beyond the raw properties and the hard coded property names that already exist.

I don't know what that would look like. It would probably need to be a different form of data entry than a text box.

btbonval commented 10 years ago

Oh, I have code that extracts all the properties and caches them. I haven't done a pull request because I didn't know if it was valuable. It makes the raw variable checking much easier e.g. (property in USCB.PROPS).

It might be neat if we could search through all the available properties which have been loaded and then assign them variable names dynamically?

btbonval commented 10 years ago

I had another idea that is more mathy. Aggregate functions.

Let's say I want to normalize an attribute like totalPopulation between 0 and 1 across all states or all counties. I need some way to say max(totalPopulation) that will grab the maximum value of totalPopulation across the current data set. Then I could say totalPopulation / max(totalPopulation), which renders values on the closed interval [0, 1].

I think aggregation will be difficult to do efficiently.

WasabiFan commented 10 years ago

I had originally made it possible to do something like foo{P0010002} and it would assign foo to the value from the raw property, but it was buggy and didn't work very well with the small box. I think it might be worth it to make a pop-out so that you could click a button and make it bigger. Then you could write small scripts across multiple lines. If we do make it pop-out with an editor, it might be valuable to look in to solutions that already exist along the lines of mini scripting/math languages and parsers.

btbonval commented 10 years ago

Oh that's what the curly braces were for! I didn't understand that at the time. That would have been handy.

I agree: it would be good to see if a mini math scripting facility exists.

Do you like the idea of being able to dynamically load ACS and SF1 properties? I'm only using it to define which property goes with which dataset (e.g. P and B works in 2010, but 2012 introduces new properties that aren't as easy to work out), but it could be valuable for a math editor to know what variables exist. sort of like an IDE.

Elvara commented 10 years ago

Are you sure that it isn't in there now? I distinctly remember coding it

-Adam

On Feb 18, 2014, at 5:49 PM, WasabiFan notifications@github.com wrote:

I had originally made it possible to do something like foo{P0010002} and it would assign foo to the value from the raw property, but it was buggy and didn't work very well with the small box. I think it might be worth it to make a pop-out so that you could click a button and make it bigger. Then you could write small scripts across multiple lines. If we do make it pop-out with an editor, it might be valuable to look in to solutions that already exist along the lines of mini scripting/math languages and parsers.

— Reply to this email directly or view it on GitHub.

WasabiFan commented 10 years ago

Definitely! If you want to submit a pull request at some point, I'll happily merge it. When I wrote this originally, I wasn't designing it with extensibility in mind... there were a lot of hard-coded values in the URLs and things like that. You untangled a lot of that (thanks!), but there is still work to be done to make it fully work with any Census set of data or maybe even other sources with some sort of custom parser. That would definitely be a start in the right direction!

Ps. Elvara, I remember that too, but I think it's partially dismantled now.

btbonval commented 10 years ago

@WasabiFan There is still work to be done by the Census Bureau as well. I was comparing ACS5 from 2010 and 2012, and they have changed meta data formats describing properties between the years. No matter what we try to do, it's going to be an uphill battle until the API is finalized. Thus I try to make things as flexible as possible. I'll submit a pull request for what I've got now. It might help for finding variables in the equation editor if we can find a sane way to data mine it.

WasabiFan commented 10 years ago

If we want to go down the route of our own mini-language, I think this might be the place to start: http://zaach.github.io/jison/. You might've heard of it; it generates a JavaScript parser given a language definition. We would just have to write a definition file, include the generated JS file, and call it's parse function. It would give the evaluated result without any other code. The one thing that I see that might not be as easy is deciding how to inject our data variables; but I'm sure it's doable. I'm working on writing a test script based on our current set of capabilities to see if it will work.