3D Scatter Well and Google Docs Integration

[theo-armour]

@ladybug-tools/build-well

Scatter Well ~ Google Spreadsheet Data R1

[bwelle]

@theo-armour,

Ok, ready with some details on this 3D Scatter Plot. This will the funnest viz to do, the most compelling to the research advisory board, Phil our CEO, and almost every architect. This is the money visualization. 3D Scatter Well. I'm proposing to something no one has ever done in the AEC industry as far as I know. I’ve updated the google sheet Anton sent. Input values in red are the dynamic inputs (ones that change) to be displayed in pop-up window when hover over design, and in the left hand panel to select for the graph, and as design constraints with duel slider. The blue ones are the outputs that may be shown in the pop-up window, left hand panel to select for the graph, and as performance constraints. The uncolored input values need not be displayed, and are only included so as to make available to you all the inputs needed to regenerate the geometry rather a single point in the scatter plot. At the end of each row in the google sheet is the name of the according SVG image.

On the interface, you will want to list all the inputs vertically in a list with a checkbox next to it, then a dropdown menu called Axis where you can select x,y, or z next to each checkbox. Put all these inputs under the heading "Design Parameters". See slide 3 here…

Then you will want to create a heading "Building Performance". List the outputs, again with a checkbox next to each one and a drop down menu for axis. The user must select one input and two outputs via the checkbox, then assign which of the three axis they want to view the three parameters. See slide 3 above as well.

Now for the actual Scatter Plot. In addition to each point displayed on the chart based on the user checked input and outputs (3 total) and axis, there needs to be a slider under the heading Design Constraints for each input variable. Then under the heading Performance constraints, there needs to a slider for each output. This is where the user can specify lower and upper limits for each input and output to eliminate points from the scatter plot and reduce the size of the point cloud. Rather than making the points disappear that fall outside the constraints, it would be preferable to grey them out to a very light semi-transparent grey. See slide 4 above.

Each slider has the range for the given parameter that exists in the CSV file. Numerical sliders may be true sliders (meaning if range of values is 3,4,5, you can put 3-5 as range, and 0-30 for orientation, even though there are only two valid values, 0 and 30), while discrete inputs such as Window Construction will have several points to jump to. Maybe we want to make numerical value sliders actually be enumerations of valid values. Not sure how best to show construction names in the slider. Output slider constraints may simply be a true slider with all values possible between the ranges (since there are many possible outputs, as opposed to only a limited set of valid inputs0. Each slider has two little control points so they can specify a minimum and maximum. All the above applies to outputs.

Finally, you need to put a slider under the heading "Output Preference". The slider goes from 0-100, and defaults to 50. The two outputs selected appear at either end of the slider, with a toggle switch to minimize or maximize that parameters. Then as you move the slider off of 50 (which is equal preference), closer to one of the objective parameters, the colors of the design points change accordingly (in the 2D axis with both outputs, colors don't change across the input. Somewhere the ranges for colors and output preference values is in the code for most preference shading modules. I'm sure three.js has somthing.

So that takes care of coloring convention for the points, and determine which points are greyed out due to being outside a design or performance constraint. See slide 4 above.

Next, we will want the user to hover a given point in the scatter plot, and have design summary window pop that lists the other input and output values, and has an svg image of the actual geometry below it. The SVG reference for each point is at the end of the CSV.

Inputs (Ignoring units):

1. Footprint Shape
2. Width 1 Y1
3. Num of Floors
4. Floor Height
5. Orientation
6. WWR_S
7. Overhang Depth S
8. Window Construction
9. LPD

Outputs (ignoring units)

1. Annual Cooling Load
2. Annual Heating Load
3. Total Solar Load
4. EUI

Inputs Ranges:

1. Footprint Shape – (L,T,H)
2. Width 1 Y1 – (80,100)
3. Num of Floors – (1,2)
4. Floor Height – (3,4.5)
5. Orientation – (0,30)
6. WWR_S – (0.4,0.8)
7. Overhang Depth S (0.6,0.9)
8. Window Construction (ASHRAE 189.1-2009 EXTWINDOW CLIMATEZONE 3, CBECS 1980-2004 EXTWINDOW CLIMATEZONE 4A, ASHRAE 90.1-2010 EXTWINDOW NONMETAL CLIMATEZONE 7-8)
9. LPD (7,12)

Outputs Ranges

5. Annual Cooling Load - (93195-273677)
6. Annual Heating Load – (317908-725639)
7. Total Solar Load - (218978-1262900)
8. EUI – (116-264)

One final thing. We'd like to show the Pareto Front in 3D, meaning the points where you cannot improve in one objective without worsening in the other. These points should be designated with bold X's. To calculate these points, I found the following JS libraries here and here:

I'd be totally stoked to get to this point. I still have that image in my head of when you zoom in have the points actually show the building geometry of that design (that's why i included the footprint dimensions in the inputs CSV), so if we have time for that, I know that would blow architects away. They've never seen that.

[bwelle]

In general, we want to take what you do, as you wish to do it, then do whatever code modifications and additions beyond that ourselves. How we take your approach/methodology/code and fit it into ME's environment they are building is up to us. We will find out tomorrow for the first time what we can expect for that process to be.

Here's what I currently perceive what your value-added scope is and where it stops....

Taking the inputs and outputs from the google docs sheet and plotting the runs on the interactive 3D plot using some symbol. (Done).

Enabling the symbols to be the actual building geometry given the inputs you already use for the Build Well visualizations.

Helping us appropriate scale the geometries of each data point with the axes units/ranges.

Enabling a HUD when the user hovers over a design that displays some inputs and an SVG of the geometry. You've already done this, and we can agree on an "average" number of inputs/outputs to display on the HUD for sizing purposes. I believe we always want the HUD to show the SVG even though the actual symbol is the same geometry when the user is zoomed out, they will want to see an image of the geometry for certain points before they are able to make out the geometry of that symbol in the plot. Additionally, the SVG will allow the user to clearly see orientation, which we need not represent with the actual symbol geometry. Eventually we will add analysis results SVGs, not just geometry. Assistance with how to ensure that the HUDs are able to be seen within the view window (and not cutoff like they sometimes are now if you're clicking on some points at the bottom section of the window) would be greatly appreciated. Like you said, that's an art along with scaling of geometric symbols, and you are the artist in the group.:)

Ignore enabling the ability for the user to select with inputs and outputs they view and on what axis. That's not critical path. Building off your suggestion, let's always put Annual Cooling Load on the Z-axis, EUI on the X-axis, and # of Floors on the Y-axis (as we always need one axis to be an input, and two an output). I like keeping the outputs on the X and Z axes since we can expect the color preference shading to go in one specific direction.

Let's ignore all the sliders for design and performance constraints that grey out certain symbols. We can do that ourselves. If you want, it would be nice to have one example of this in the open source code for others to use, as it's pretty cool. If you agree with this scope, I propose you make one design constraint only...# of Floors, so we can see how you look up ranges for that input in google docs to put as starting and end points for the slider. This also applies how to look up ranges for an input and auto-scale the plot axes with the appropriate starting and ending grid levels (for example, for # of Floors it will be 1-3 with our current runs, but what if we had # of Floors ranging from 2-7?) With that example, we can do everything else.

I think the preference shading slider is important. We will fix the inputs to it as Annual Cooling Load and EUI, the two fixed outputs for the plot. Somewhere the user clicks whether they want to maximize or minimize that output objective, then they can use the slider to define which one is more important to them, which will then change the color transition on the X and Z plane only.

The last thing we need to provide is the pareto front designs on the chart, the special data points/runs that can get better with one output objective (Annual Cooling Load) without getting worse on the second output objective (EUI). I provide a JS library that takes all the input points, the values of the outputs, and what is considered "better" for each output parameter, i.e. the numerical value is lower or higher. It spits back the points/runs that meet this criteria, then we would want to make those points visually different then the rest of the data points. In a conventional plot, this is done by making the designs an "X" rather than a "point". But in our case, we will have the actual building geometry, so I would recommend just making the color of those point black, which is not an available color on the preference shading.

An example of how to do this on Github would be highly valuable to others wanting to use a 3D scatter plot, and it would provide an example of using an external JS library to process/evaluate the designs within the three.js code, which could be applicable to any type of process they want to invoke.

These are the things we would love to have your support on. It seems to me scoped so that user experience is taken out of the equation, you're not doing any more than one example for each piece, and with this to work with, we can get get to our end goal in a reasonable amount of time.

[theo-armour]

@bwelle

Fingers crossed the next run goes OK.

[And I wonder if by next year we are doing the runs in JavaScript accessing the GPU and running licketty split ;-]

---

I had a lot of fun today. The new menu system I am playing with is smaller, faster and easier then before.

The gbXML Viewer and Build Well should tart have this code in the next day or two. It should make life easier. Code that works in one should work in the other.

High Level Requirements

• Switch between Flatland stuff and 3D models seamlessly
• Maintain 3D context/scope while switching menus
• Enable multiple menus to interactively enable adding, editing and deleting data in the 3D model
• Everything entry-level JavaScript
  • lots of small files / nothing big to slog through
• Apart from primary libraries - Three.js for 3D and Showdown for Markdown - no external dependencies
• All menus also available as standalone, testable HTML files
• And, you must feel assured that no 3D data is lost when looking at 2D text content web pages

[bwelle]

@theo-armour Well now we can do 100 runs in parallel, so can do 1000 runs in about 8 minutes. NREL has given us the info we need to run 10,000 runs in parallel. Hopefully in a few weeks will have this up and running.

Good high level requirements. Will be adding some comments to Build Well issue in next few mins.

[bwelle]

@theo-armour I updated the main google docs tab with new data. I also updated SVGs in SVG folder (there are 864 runs, which all reference 144 OSMs, so I overwrote 144 old OSMs in SVG folder. But nothing shows up in 3D Scatter. Do you know why?

[mdengusiak]

@bwelle I just review your post and check all links and "Design Parameters" 3D Scatter Plot idea looks great! How do you run them? You produce from build-well all options using excel then all are exported to specify location and OS runs from selected folder? Do you store all simulation assumption in OSgbXML? or those come from OS as default values?

[bwelle]

@mdengusiak We actually just use Grasshopper and OS component in Honeybee to produce OS server spreadsheet and OSMS, then use measures in OS. Then we store in Mongo, then query Mongo to generate a csv. Happy to show you sometime the workflow.

[bwelle]

@theo-armour Wow! I updated the spreadsheet to match the last, and look at what happens with the 3D Scatter Plot! Kinda groovy.

[bwelle]

@theo-armour Ahh, nevermind. Found another issue with data need to resolve first before spreadsheet formatting.:(

[bwelle]

@theo-armour I have merged 2 Scatter Plot threads as there was pertinent info info in both and some repetitive, easier for now to have as one thread, then can break out smaller tasks once we get rolling more so they may be closed.

[bwelle]

@theo-armour Addressing Issues from above:

Taking the inputs and outputs from the google docs sheet and plotting the runs on the interactive 3D plot using some symbol.

You had this hard-coded for v1. Let's dig a little deeper to see what inputs we can expect each time. In general, the inputs will vary for every project, but the outputs will stay fixed. So we need a way to accommodate varying input values in the Google Docs spreadsheet, which will have varying ranges project by project. As agreed, we will ignore enabling the ability for the user to select with inputs and outputs they view and on what axis. That's not critical path. Building off your suggestion, let's always put Annual Cooling Load on the Z-axis, EUI on the X-axis, and Num of Floors on the Y-axis (as we always need one axis to be an input, and two an output). I like keeping the outputs on the X and Z axes since we can expect the color preference shading to go in one specific direction. Let's just hard-code this into the axes, meaning these two outputs and one input must always be in the Google Docs spreadsheet.

As Num of Floors, Annual Cooling Load, and EUI are in the spreadsheet, that should give you everything you need to "place" a point, or geometry on the 3D grid.

Next, you will need the geometric inputs necessary to recreate the geometry for each point. Your previous code used: Footprint Shape, Width 1 (m), Num of Floors, Floor Height (m), Orientation (deg), WWR (let's ignore WWR by orientation), Building Area (ft2), Width 2 (m), Length 1 (m), Length (2), and for H and T shape, Width 3 (m) and Length 3 (m), as well as Overhang Depth (Let's ignore Overhang Depth by orientation) and Perimeter Depth. These parameters should allow you to recreate the geometry using the old method for each design in the 3D scatter plot. As detailed in the Build Well post, we are changing the equations and inputs to simplify for geometry, so this will have to be updated once that's up and running, but for now assuming old method is used.

The only thing left is the HUD. Let's assume we simply display the following inputs and outputs in the HUD...Inputs: Footprint Shape, Building Area, Num of Floors, Floor Height, Orientation, WWR, Window Type, Overhang Depth, Wall R-Value/Type, and Roof R-Value/Type. For Outputs, we will display Annual Cooling Load, Annual Heating Load, PV Generation, and EUI. That's 14 lines items, which should keep the HUD to a minimum. Then display the corresponding SVG as you currently do. All other inputs/outputs in the Google Docs spreadsheet you can ignore in the HUD.

Then as identified before, the following are critical path:

Helping us appropriate scale the geometries of each data point with the axes units/ranges.

I propose you make one design constraint only...# of Floors, so we can see how you look up ranges for that input in google docs to put as starting and end points for the slider. This also applies how to look up ranges for an input and auto-scale the plot axes with the appropriate starting and ending grid levels (for example, for # of Floors it will be 1-3 with our current runs, but what if we had # of Floors ranging from 2-7?) With that example, we can do everything else.

I think the preference shading slider is important. We will fix the inputs to it as Annual Cooling Load and EUI, the two fixed outputs for the plot. Somewhere the user clicks whether they want to maximize or minimize that output objective, then they can use the slider to define which one is more important to them, which will then change the color transition on the X and Z plane only.

The last thing we need to provide is the pareto front designs on the chart, the special data points/runs that can get better with one output objective (Annual Cooling Load) without getting worse on the second output objective (EUI). I provide a JS library that takes all the input points, the values of the outputs, and what is considered "better" for each output parameter, i.e. the numerical value is lower or higher. It spits back the points/runs that meet this criteria, then we would want to make those points visually different then the rest of the data points. In a conventional plot, this is done by making the designs an "X" rather than a "point". But in our case, we will have the actual building geometry, so I would recommend just making the color of those point black, which is not an available color on the preference shading.

I'd leave this one for last in case we don't get there. The rest are critical path. If we don't get to the 2D Scatter Plot for demo, that's fine.

How does this sound? Make sense?

[bwelle]

@theo-armour With the new runs, we got some bigger differences than the last set, but there are still some close values (which we are investigating). I'm not sure how to handle these close values so a bunch of buildings don't overlap and you can't make heads or tails of it.

[theo-armour]

@bwelle

Looking much better! Yay!

After just a quick read, I think most everything you suggest is doable. It's going to take a number of revisions to get there. I will know a lot more after a dive into the changes.

[bwelle]

@theo-armour

Actually not. I just switched back to your old spreadsheet since I'm on the phone with MicroExcel showing what you did. I need to switch to new spreadsheet, then nothting apperas.

[theo-armour]

@bwelle

Somebody changed column W from 'total transmitted solar kwh' to 'PV Generation

[bwelle]

@theo-armour Yes I did that intentionally. Here's the sequence of events....Your original spreadsheet has been renamed "Old Spreadsheet" as a tab in the Google Docs. If you put that back as "Sheet 1" everything shows up as before. However, the old spreadsheet doesn't work for us, since we will always run into the situation where the user has different inputs and a bunch of outputs (I cut out total transmitted solar since no ones uses that as a metric for anything). As I mentioned, you needed that field because you had it fixed to the Z axis. But we don't this fixed to the Z axis. We want EUI, Annual Cooling Load, and Num of Floors as the three parameters on the axes. So as long as you can find those headings in the spreadsheet, you have the ranges and values to plot. Right now in Sheet 1, which has PV Generation rather than Transmitted solar load, all those runs represent new runs we did. We varied different input parameters, trying not to get outputs that were so similar from the last time we examined them.

[theo-armour]

@bwelle got it. will fix tomorrow

[theo-armour]

@bwelle

unclosed. ;-)