TechLabs-Berlin / project_proposals

2 stars 0 forks source link

Reuse Record by Jannik Oslender #12

Open valiantone opened 5 months ago

valiantone commented 5 months ago

What's the problem you are trying to solve?

Baukreisel is an interdisciplinary collective and non profit working towards enabling reuse in the construction industry at scale. Material category after material category we are developing rationalized reuse processes for cataloging/recording, extracting, transporting, refurbishing and reinserting building material and elements.Due to their high carbon impact and the potential to be reused at scale our current research projects are focused on concrete primary structures and windows elements. The research initiative is called reuse.matters.In the reuse process the recording and surveying of the elements is a very time consuming and tedious task.The current methods (Excel or Airtable) do not offer any level of automation and require you to manually input information for similar elements again and again. Besides that, the databases cannot store the different sub elements, that e.g. a window consists of, in a machine readable and understandable way.This knowledge about the interconnection of elements and subelements is especially important when refurbishing or transforming. For example when dealing with windows the glass and the frame might be reused separately and in a different way. Thus information about the subelement is important and should be derived as implicit knowledge.In short the project should solve 3 problems:-

  1. Create a way to easily record and survey building elements and material in soon to be demolished buildings
  2. Make that process a lot faster (e.g. through a convenient interface, smart templates and live recommendations/autofills as well as computer vision and object recognition.
  3. Save the information and relations in a semantic and machine readable way so that they are available in the reuse process

What's your idea for a solution to the given "problem"?

A mobile app to record and survey building elements on site and a graph database to store the information so that later on explicit and implicit knowledge can be retrieved.In order to being a feasible project, that can be developed in the scope and duration of the project phase the app shall be focus on the surveying of window elements.To speed up the recording process there are three key elements:- An easy to use interface that quickly guides you through the recording process- Smart autofills and suggestions that reduce the need to input the same information several times. An example could be that the building location and the current storey you are in is prefilled with information from the last entry. Similarly element types/categories could be suggested while typing etc.- A catalogue of possible shapes and window configurations (amount of windows sashes, fanlight, door etc.) and object recognition that suggest, based upon the catalogue, the type of configuration to the user after taking a photo. Retrieving information about the frame colour or material would be great but maybe more difficult to retrieve.(- A way to quickly get rough measurements using the phones camera and AR features would be great but very likely beyond the scope of the project phase so just as a note)Regarding storing the data, the building industry has developed several semantic ontologies to structure the data that could be used as the foundation for the layout of the graph database. In contrast to relational databases, graph databases allow you to very easily store relationships (I know, somewhat unintuitive). This is espacially important when working with buildings and materials. There are geometrical relationships (room is part of a storey is part of a building) and type relationships (oak is a type of hardwood is a type of wood) and subelement relationships (glass and frame are a subelement of a casement is a sub element of a window is a sub element of the facade).This can be modeled in a graph like neo4j and not only explicit but also implicit knowledge can be retrieved (amount of glass elements inside a storey) (all wooden window frames in a building etc.).Even though graph databases offer a lot of advantages in this scenario their implementation might be beyond the scope of the project phase and as an MVP the app could also work with a relational database.

Which tracks do you think could be involved?

DL/DS WD UX

How do you imagine Deep Learning and/or Data Science could contribute to this project?

Through setting up and training a neural network that allows the app to recognize the shape and configuration of the photographed windows elements.

How do you imagine User Experience Design could contribute to this project?

Though setting up an easy to use interface that lets the user quickly cycle through the recording steps.

How do you imagine Web Development could contribute to this project?

Through setting up a front end that autofills and suggests inputs to make the recording and surveying process faster.Through setting up a backend that allows the information and relationships to be stored in a graph database.

What are the most important challenges & bottlenecks of your project?

What are the most important success factors of your project?

Teamwork:Only if different people from the different tracks work well together the project can realize its full potentialPractical insight:Baukreisel is going to supply us as good as possible with all the real world information about the recording process.

Share your resources here:

Links:

valiantone commented 5 months ago

Ok, so the scope of this project may be too large to achieve in 10 weeks, let's make a first start towards a MVP that should be usable and solve the core need, this would require:

UX

The task would be straight-forward to implement however the app seems to be for expert users in a specific domain. Can you guarantee the UX techies access to test users (3-5 users) who would be willing to test and give feedback? This is mandatory for the UX process.

WD

Also a straight-forward task, once you lockdown a few features for development the FE and BE techies can divide the required tasks between themselves. Maybe already take some time to think of features that would be required (eg search over graph database, dashboard view, history view, photo upload etc) for the app. Final decisions may differ, but it's good to list some prospective ideas to start.

Please also remember only a webapp would be within the scope of project phase, native application development has not bee covered in our curriculum.

DS/DL

Please differentiate between tasks that would be performed by DL techies (building image classification to detect and identify window elements) vs DS techies (auto-suggestion and completion for analytical and ml-based inference of parameters in templates). Here again, final features are subject to change but would be good to have a clear initial proposal on this from project requestor.

Simply, respond to the comments @Jannik0s and we can close this conversation and move your project to Accepted so you are all clear for team formation and project phase! 🚀

Jannik0s commented 5 months ago

I totally agree! The topic is super vast but concentrating on windows as the element to record and prioritizing features might make it feasible.

The web app quite likely consist of 3 parts

1 Input: This is where the recording and surveying happens. I think it is the most important part since this is currently the biggest pain point.

2 Catalogue: Here you can see the contents of the Graph Database. Important to have but maybe a bit less challenging to develop since there are a lot of examples for it

3 Dashboard: Get analytics about the stock of materials/building elements that have been recorded. This would add a lot of value but would not be necessary for the app to work.

UX

The collective has a core team of 6 including me and everybody is eager to support us with knowledge and testing. However, since there are buildings - and windows for that matter - all around us, everybody else can give it a test at home too.

WD

1 Input: Necessary:

Important:

2 Catalogue: Necessary:

Important:

Great to have:

3 Dashboard: Necessary:

Important:

DS

1 Input: Necessary:

Important:

2 Catalogue: Necessary:

Important:

3 Dashboard: Necessary:

Important:

DL

1 Input: Necessary:

valiantone commented 5 months ago

All right @Jannik0s, quite some detailed answers, looks like we've thought of most of the requirements and have some starting points for all discussions. Please feel free to start looking for team mates! I would suggest a standard team setup for your need:

Team Formation Requirements

Jannik0s commented 5 months ago

This is great news!

@martajasinska already expressed her interest and is joining as UX Techie! So good to have you in the Team, Marta  🥳🚀

I am going to take part as Backend WD so still positions of every track vacant.

Really looking forward to you guys reaching out!

Jannik0s commented 5 months ago

On this Miro we have started collecting relevant information for the project:

https://miro.com/app/board/uXjVN2Hi_FY=/?share_link_id=121195818869

The Password is: TechLabs

Jannik0s commented 5 months ago

Happy to announce that @pr4t1ma joined the team as Front End WD. Looking forward to making this project a reality with you!

Soma, Christoph and Arpad expressed their interest to become part of the project as DL Techies. It is so great that you would like to push it from a Deep Learning perspective!

Since there are currently only 1-2 seats assigned for DL maybe the Track Leads @valiantone can give some guidance on how to proceed.

Jannik0s commented 5 months ago

Reuse Record 🌀 Team

Jannik0s commented 5 months ago

DL

During the first discussions with DL Techie Christoph, some questions about the dataset to train the image recognition model have come up. Unfortunately there is no dataset that we are aware of that labels the images in the way we need it.

Since there is still some time before the project starts and since through Baukreisel we have a couple of people at hand, is it possible to build and label that dataset on our own?

I think it would be quite doable to have a couple of hundred labeled pictures ready before the project starts. Especially considering it is possible to include photos of the same window from different angles.

Bildschirmfoto 2024-01-24 um 17 28 27
martajasinska commented 5 months ago

I would add the opening direction DIN left/ DIN right/ bottom hung window (Kippfenster)/ fix-glazed window (Festverglasung) to the window parameters - this should be easily determinable based on the position of the handle

Jannik0s commented 5 months ago

That would be nice to have indeed but classifying is getting complicated when the windows consist of several sub elements like the first one.

Would then be something like (1l + 1r + 1r) x (1l + 1r +1l)

Probably we have to focus on simpler windows first anyways. Would still be nice to know how to set this up so that it can be extended.

mahanfa commented 5 months ago

Hello, nice project! I've studied Architecture as well. I would like to join as a UX designer if possible, thanks!

Jannik0s commented 5 months ago

That's great! Welcome to the team!

mahanfa commented 5 months ago

Hey guys, I'm very sorry, just in the last minute I understood that we can only be in one project. I had my own project and I thought I have to also take part in someone else's project. So I have to leave this one. :( Just for the record if it is for any good, my very good friend Mana is looking for projects so I will mention her here. @Mananosrati

Mananosrati commented 5 months ago

Hi there, it's an amazing project and idea. if it is possible I would like to join you as a UX member.