Abigail-Herbas

[aherbas3]

Weekly Notebook Entry — Week 2

Overview

• Week Span: 1/22 to 1/27

Tasks for This Week

List your specific objectives for this week, including learning targets and project milestones.

• [x] Task 1: Establish lines of Communication
• Ensure that we have access to the Dropbox, Microsoft Teams, and Github. Also create a groupchat with team members.
• [x] Task 2: Initial Research
• Look into papers and scholars Dr. Kastner linked to our portion of the slideshow to gain familiarity with the subject. Next week we must make a short presentation on our findings.

Tasks

Task: [Initial Research] - (ongoing)

• Current Status: After having set up a groupchat, we began dividing the work. I will be in charge of the Problem Definition section. I've begun reading through the articles.

Challenges & Solutions

• Reading through the articles is quite difficult due to my lack of familiarity with the subject. I'm having trouble understanding what I'm reading, but in the meantime I've been googling a lot of vocabulary and trying to keep up.

General Takeaway, Skills & Knowledge Acquired

• Monday was our first meeting. We found our sub-teams and got a general overview of our respective topics. What's most important is we set up all our communication resources so that this semester can go smoothly. Also, we need to begin delving into our research topic.

Personal Reflection

• I'm excited to get to know everyone and to learn more about energy usage in buildings!

Plans for Next Week

• [ ] Next Week's Task 1: Present our Initial Research

[aherbas3]

Weekly Notebook Entry — Week [3]

Overview

• Week Span: [1/29] to [2/2]

Tasks for This Week

Continue literature review so we can build a strong foundational background knowledge to work from.

• [ ] Task 1: Present Initial Research
• Having read through the article on the introductory meeting slide and skimmed through the scholar's abstracts, I need to present my slide on the problem definition.
• [ ] Task 2: Meet with Dr. Kastner
• We need help narrowing down our research. A meeting with Dr. Kastner can help guide us in the right direction considering his expertise in the subject.

Tasks

Task: [Present Initial Research] - (completed)

• Description: Presented my slide using what I've learned. Throughout the research process, I kept a document of excerpts I found particularly succinct and helpful.
• Link to Work: https://www.dropbox.com/preview/VIP_SMUR_ClassFolder/Sp24-EnergyInBuildings/Weekly%20Slides/1_29.pptx?context=content_suggestions&role=personal
• Outcome & Reflection: Honed my presentation skills. Gained a greater understanding on the purpose behind our project, which all boils down to making sustainable decisions easier to implement.

Task: [Narrow Down Research] - (ongoing)

• Current Status: Met with Kastner to discuss project goals in the hopes of finding direction.
• Expected Completion: Right now we're in the discovery phase, I suspect by the next week or so we'll be more solidified in our standing.

Challenges & Solutions

• Finding our path. The field is large and we are all new to it. Having research gaps laid out for us makes it easier, but we still need a concrete goal in sight.

General Takeaway, Skills & Knowledge Acquired

• After the meeting we laid out our project parameters and got insight into what data sets we could use. The team plans to each pick a paper and make a deep dive into it.

Personal Reflection

• I got some pointers from Dr. Kastner on how to efficiently read through research papers that I plan to use.

Plans for Next Week

• [ ] Next Week's Task 1: Present Deep Dive

[aherbas3]

Week 4 - 2/5 to 2/10 Looked into the paper https://www.nature.com/articles/s41560-023-01341-5.pdf and gained a clear understanding of the process. Presented my findings to the class https://www.dropbox.com/preview/VIP_SMUR_ClassFolder/Sp24-EnergyInBuildings/Weekly%20Slides/2_5.pptx?context=content_suggestions&role=personal.

Week 5 - 2/11 to 2/16 Attended the introduction to Grasshopper and Urbano presentation from Silvia. Decided to split running the two models. I'll try running mine and Aarit his. Received input from Dr. Kastner saying we should prioritize ease of use over anything, and if running the simulations proves too hard we may need to look in another direction.

Week 6 - 2/18 to 2/24 We found our hourly and atlanta data sets, as outlined in our team notes. While the models we'll be using is still unclear, we know we have the data to begin running beginner simulations. Update: We got the build environment set up for one of our models. We're having issues with AWS configuration which we will be looking into.

[aherbas3]

Weekly Notebook Entry — Week [7]

Overview

• Week Span: [2/25] to [3/2]

Tasks for This Week

List your specific objectives for this week, including learning targets and project milestones.

• [ ] Task 1: Complete Demand Ninja Test Run
• [ ] Task 2: Gather Demand Ninja Dataset

Tasks

Task: [Complete Demand Ninja Test Run] - (complete)

• Description: Create a new script with the help of Demand Ninja's documentation to get the model working. I'm starting with 3 starter data points.
• Outcome & Reflection: While I was able to pass input data and receive output, I ran into the issue of NaN values being returned. Overall, this process increased my familiarity with the python language. However, a new task arose: debugging.

Task: [Debugging Demand Ninja Test Run] - (ongoing)

• Current Status: I've been looking into python language documentation to gain a sense of where the issue might lie. While still confused on why the output is returning NaN values, I'm slowly getting familiar with the code behind the calculations, core.py.
• Expected Completion: Get the test run working by next week. If not, seek help.

Task: [Gather Demand Ninja Dataset] - (complete)

• Description: We looked into Demand Ninja's website to gather its dataset in hopes of using it as well.
• Outcome & Reflection: Dead end. While the About page linked its sources, we were left with complex processes and calculations instead of the actual dataset itself.

Challenges & Solutions

Understanding other developer's code can be difficult and take a lot of time, especially when you have to debug a long series of calculations. Python documentation is really helpful in establishing some foundational knowledge. Also, it's disappointing when datasets are inaccessible.

General Takeaway, Skills & Knowledge Acquired

I'm slowly getting the hang of the python language itself and also the logic behind Demand Ninja. Excited to gain a deeper understanding on the inner workings.

Team Meetings

• Meeting Highlights: Canceled with Dr. Kastner but reached a consensus on holding regular, long team meetings. Half of us are working on running models and the other half is working on gathering data.
• Peer Feedback: We are all running into technical issues but are dedicated to grow our skillsets together to tackle them.

Personal Reflection

• slowly but surely we're delving deeper into our subject, which is exciting.

Plans for Next Week- [ ] Next Week's Task 1: Description and strategy.

• [ ] Debugging Demand Ninja Test Run: get rid of NaN values
• [ ] Find Alternate Datasets: Discuss with Dr. Kastner, use suggestions

[aherbas3]

Weekly Notebook Entry — Week [8]

Overview

• Week Span: [3/3] to [3/9]

Tasks for This Week

• [ ] Task 1: Debugging Demand Ninja Test Run
• [ ] Task 2: Read Demand Ninja Criticism

Tasks

Task: [Read Demand Ninja Criticism] - (completed)

• Description: During meeting with Dr. Kastner, he brought an article to our attention that criticizes the model. The article points out a flaw with the model's data; it relies on large amounts of retrofitted building data which is often kept private.
• Demand Ninja Criticism: https://www.nature.com/articles/s41560-023-01407-4
• Outcome & Reflection: Given the fact that we're already having difficulty gathering data, we're reconsidering how viable Demand Ninja is as a model for our purposes.

Task: [Debugging Demand Ninja Test Run] - (ongoing)

• Current Status: Tweaking the range of input values, I was able to get more output data to return real numbers. However, bait values are still NaN and therefore causing a domino effect, leading the final total demand value to also be NaN.
• Expected Completion: Have it done by next week.

Challenges & Solutions

While the paper's finding's raised alarms, we're still going to attempt to get the model running because we need backup models at this point and are optimistic in finding the necessary data when the time comes.

General Takeaway, Skills & Knowledge Acquired

Blackbox testing can only take you so far, so I'll begin digging through the core.py code line by line to discern what's going on.

Team Meetings

• Meeting Highlights: Must gain a solid understanding of how Demand Ninja is functioning. Rest of the team is working on running Rank 1 model and data crawling.
• Peer Feedback: We're making progress, optimistic about getting a fully running model soon.

Personal Reflection

• I learned how time-consuming testing can be, especially when documentation is limited. Looking at the code line by line is really helping me understand the working parts of the code and how they interact together.

Additional Notes & Resources (optional)

• Weekly Slides

---

Plans for Next Week

• [ ] Debugging Demand Ninja Test Run: white box testing

[aherbas3]

Weekly Notebook Entry — Week [9]

Overview

• Week Span: [3/10] to [3/16]

Tasks for This Week

• [ ] Task 2: Debugging Demand Ninja Test Run
• [ ] Task 3: Reconsider Models

Tasks

Task: [Reconsider Models] - (completed)

• Description: After progress meeting with Dr. Kastner we determined that Rank 1 took too much computing time in comparison to similar models. Decided to pivot to Rank 5 which provides similar results considerably faster. Also, we must gain a stronger understanding of what its dataset values represent.
• Outcome & Reflection: Switched efforts from Rank 5 to Rank 1, will begin exploring that path more.

Task: [Debugging Demand Ninja Test Run] - (ongoing)

• Current Status: Began isolating code segments and printing output at intermediary locations to gain an idea of where the issue lies. Have reduced it down to the bait function, but am struggling to determine at what point exactly. Currently believe the problem lies in how the data is spread into different timeframes.
• Expected Completion: Hopefully next week

Challenges & Solutions

It's difficult understanding what output I'm supposed to be seeing at each step of the code. What helps is referencing the research paper's flowchart showing the data transformations taking place as well as stackoverflow.

General Takeaway, Skills & Knowledge Acquired

Greater understanding on how data is smoothed over hours.

Team Meetings

• Meeting Highlights: We discussed the change from Rank 1 to Rank 5 in this meeting as outlined above.
• Peer Feedback: We're starting to concentrate efforts on efficiency even more now.

Personal Reflection

Debugging process has been long and difficult. Might have to enlist help of teammates.

Plans for Next Week

Spring Break

[aherbas3]

Weekly Notebook Entry — Week [10]

Overview

• Week Span: [3/17] to [3/23]

SPRING BREAK, RESUME WORK NEXT WEEK

[aherbas3]

Weekly Notebook Entry — Week [11]

Overview

• Week Span: [3/24] to [3/30]

Tasks for This Week

List your specific objectives for this week, including learning targets and project milestones.

• [ ] Task 1: Get Help With Debugging Demand Ninja
• [ ] Task 2: Get Building Data

Tasks

Task: [Get Building Data] - (completed)

• Description: Looked into Fulton Assessor's website to retrieve building data that'll be used with our models.
• Link to Website: https://qpublic.schneidercorp.com/Application.aspx?AppID=936&LayerID=18251&PageTypeID=2&PageID=8154
• Outcome & Reflection: New dataset for our project. We must get models working.

Task: [Get Help With Debugging Demand Ninja] - (ongoing)

• Current Status: Met with Aarit to explain my issues and provide an overview of core.py. Described my testing progress so far and where I've isolated the issue to. We've both been looking into it since.
• Expected Completion: Unknown

Challenges & Solutions

Having reached a roadblock in debugging Demand Ninja alone, decided to ask a teammate for help. Realized the importance of reaching out for help and once again how important code documentation is.

General Takeaway, Skills & Knowledge Acquired

Learned how to accurately and concisely describe code and bugs.

Team Meetings

• Meeting Highlights: Anubha gave an update on data collection, Shivam on Rank 5, and I on Demand Ninja. Decided Aarit would help me out.
• Peer Feedback: As data collection progresses, the pressure is building to get a model running.

Personal Reflection

Hopefully with two of us working on debugging we'll fix the issue. Maybe with two sets of eyes we'll catch something I've been overlooking all along. I have more optimism that it'll work out.

---

Plans for Next Week

• [ ] Continue Debugging: this time with extra help from teammate

[aherbas3]

Weekly Notebook Entry — Week [11]

Overview

• Week Span: [3/30] to [4/6]

Tasks for This Week

• [ ] Task 1: Debugging with Teammate

Tasks

Task: [Debugging with Teammate] - (ongoing)

• Current Status: Not much progress. We can't seem to get it to work, even together. I've added comments to my code describing all my test cases and communicated them with another teammate as well to see if they have any input, but will have to wait for feedback due to busy schedules.
• Expected Completion: Unknown

Challenges & Solutions

Despite having 2 work on it, Demand Ninja remains a mystery. I'm extremely confused by where the problem is and have resorted to asking ChatGPT to explain what certain lines are doing.

General Takeaway, Skills & Knowledge Acquired

I'm still trying to find where the problem lies. Considering the immense amount of time spent trying to debug it, I'm wondering if it'd be more efficient to split up and instead focus on helping out our other teammates on web scraping and Rank 5.

Team Meetings

• Meeting Highlights: Running into issues with Recaptcha while running web scraper. Dr. Kastner introduced us to Kepler GeoJSON tool we could use further along.
• Peer Feedback: As finals approach, we're eager to get models running.

Personal Reflection

Despite having fully gone through the code, I don't understand how I could fix the issue. I'm feeling the pressure to get this working. Hopefully next week we'll have another teammate look into it and possibly gain some clarity.

Plans for Next Week

• [ ] Debug with Teammate 3: Fresh set of eyes might help.

[aherbas3]

Weekly Notebook Entry — Week [12]

Overview

• Week Span: [4/7] to [4/12]

Tasks for This Week

• [ ] Task 1: Evaluate Team Strategy

Tasks

Task: [Evaluate Team Strategy] - (completed)

• Description: Web scraper is using new methods to mimic human behavior and get past restrictions. Rank 5 is facing permission issues. It's becoming clear we might have to rely on Demand Ninja afterall.
• Outcome & Reflection: Now that we know where we stand, we're shifting focus to Demand Ninja.

Challenges & Solutions

Deadlines coming up, but so are exams. We're trying to remain determined and focus on getting some final product out by the end of the semester.

General Takeaway, Skills & Knowledge Acquired

Patience is a virtue.

Team Meetings

• Meeting Highlights: Discovered we're most likely going to have data, just not models to run them.
• Peer Feedback: Remain in contact and help eachother wherever possible.

Personal Reflection

I'm glad building data will be available for next semester. However, I worry about our issues running models. Despite having two backup models, we ran into technical issues that made both difficult to complete in time.

---

Plans for Next Week

• [ ] Debug Demand Ninja: consult third member

[aherbas3]

Weekly Notebook Entry — Week [13]

Overview

• Week Span: [4/13] to [4/22

Tasks for This Week

• [ ] Task 1: Debug Demand Ninja
• [ ] Task 2: Final Presentation

Tasks

Task: [Debug Demand Ninja] - (completed)

• Description: With the help of third teammate we were able to fix the issue in the bait function. It was caused by wrong inputs.
• Outcome & Reflection: Can now obtain processed output data

Task: [Final Presentation] - (ongoing)

• Current Status: We compiled a slideshow of our work this semester. After having gotten processed data from Demand Ninja using addresses obtained from the NREL API, it was turned into GeoJSON format and Kepler.gl helped visualize results.
• Expected Completion: While data was returned, we discovered that data points were too close in nature because Demand was meant to be used on larger geographic scales. Presentation was completed, but we finished with an open end.

Challenges & Solutions

Significant challenges throughout this semester were data collection and model debugging. Solutions were teamwork and being creative by finding new workarounds.

General Takeaway, Skills & Knowledge Acquired

Learned to reach out for help when encountering issues. The more you reach out, the higher your chances of receiving helpful input. Also learned the importance of having backups to the backups.

Personal Reflection

This semester taught me the importance of independent learning but also how to balance it with resourcefulness. It's okay to ask for help. Next semester we plan to look into the discrepancies with Demand Ninja data points.

Additional Notes & Resources (optional)

• Final Presentation

---