colouring-cities / colouring-britain

Developed out of the Colouring London prototype. Collecting data on Britain's buildings and testing new core features
https://colouringbritain.org/
GNU General Public License v3.0
10 stars 2 forks source link

Data: Energy: EPC data aggregated at property level #58

Closed fallipalaiologou closed 3 months ago

fallipalaiologou commented 7 months ago

@matkoniecz @polly64 @taimaz22 @polly64 comment: Provide EPC data for MKon aggregated at property level (i.e. for many flats in one block?) and streetblock level (i.e. for all buildings in a block so these will match with OS Open Map Local spatial units)."

@polly64 to confirm: 1) Aggregation at property level: this means aggregate by building polygon (or same building ID number in the EPC dataset)? 2) Aggregation at block level:

3) Aggregation at street level: Is this by street name? Also, does this mean each side of the street aggregated separately or are we aggregating the blockfronts of both sides together?

4) When we aggregate by block or street, do we need to take into account of the building size? I.e. weight the aggregation according to building size (namely, bigger buildings' rating will weight more compared to smaller buildings)? In general, do we need to consider weighting or do we plan to simply average the ratings?

polly64 commented 7 months ago

@fallipalaiologou @matkoniecz @polly64 @MKIM1008

@polly64 comment: Provide EPC data for MKon aggregated at property level (i.e. for many flats in one block?) and streetblock level (i.e. for all buildings in a block so these will match with OS Open Map Local spatial units)."

@polly64 to confirm: 1) Aggregation at property level: this means aggregate by building polygon (or same building ID number in the EPC dataset)? I think we need to experiment here suggest @MKIM1008 does tests to see what makes sense and allows us a few options

2) Aggregation at block level:

3) Aggregation at street level: Is this by street name? Also, does this mean each side of the street aggregated separately or are we aggregating the blockfronts of both sides together? Good point Again can you experiment and make recommendations no fixed answer-am going to introduce you to the Birmingham energy team now (see email)

4) When we aggregate by block or street, do we need to take into account of the building size? I.e. weight the aggregation according to building size (namely, bigger buildings' rating will weight more compared to smaller buildings)? In general, do we need to consider weighting or do we plan to simply average the ratings? As above

MKIM1008 commented 6 months ago

From #57 Colouring Loughborough stage on check list


M_Kon: Change to Colouring Loughborough and alter landing text to read: _Welcome to Colouring Loughborough!

Colouring Loughborough is a free knowledge exchange platform designed to provide over 100 types of open data on Loughborough's buildings to help improve their quality, efficiency, sustainability and resilience and to meet net-zero goals.

Colouring Loughborough is also the Colouring Britain regional demonstration site, and is part of the international Colouring Cities Research Programme/CCRP. Teh CCRP is run by the Alan Turing Institute and provides open-source code for networked open data platforms providing open spatial data on the composition, performance and dynamic behaviour of the building stock. This is designed as a sustainable resource with new features and data added and improved on all the time.

At the moment data on buildings is very fragmented and also often heavily restricted. The Colouring Loughborough platform allows you to help the University and local authority build a free digital platform for the town that can help achieve sustainability goals,

All our data and code are free to download, use and share under our open licence terms. Our open manual also provides non-technical information on the CCRP for anyone interested in our research.!

Colouring London is a free knowledge exchange platform designed to provide over fifty types of open data on buildings in the city, to help make the city more sustainable.

Colouring London is part Colouring Cities. The Colouring Cities Research Programme (CCRP), led by the Alan Turing Institute, is an international consortium of academic institutions involved in building research. Our aim is to maximise accessibility to building-level data across countries, to help improve stock quality, efficiency, sustainability and resilience and meet net-zero goals.

New data and features are added all the time. We are keen to engage as many people as possible in platform development. Whether you are a resident, or a stakeholder in academia, government, industry or the third sector, any help you can give colouring in our Colouring Cities maps, and enriching and verifying our open databases with your knowledge, is greatly appreciated.

All our data and code are free to download, use and share under our open licence terms. Our open manual also provides non-technical information on the CCRP for anyone interested in our research._

M_Kon: Demo of progress with Colouring Loughborough interface

M_Kon: Discuss progress re use of OSMM to map Loughborough on staging - can we see? Need to keep these behind the firewall once use approved.

M_Kon/PH: Helpful to track amount of Azure credits used for Loughborough site - current plan to work with a university as part of Colouring Britain academic network who will will buy a server and manage the database so we are no longer on the Azure system and can cut these costs in the long run.

M_Kon: Discuss progress with testing of INSPIRE to split Open Map Local Upload for Charnborough /or E. Midlands Region - can we see sample?

M_Kon: Demo any EPC data provided by MKim if time

M_Kon/TL: Consider a short screen video showing insertion of these features as a demo to encourage engagement from stakeholders, and to produce quick demo screen videos as new features are built which you can blog about

Data: Energy: EPC data aggregated at property level #58 streetblock level (i.e. for all buildings in a block so these will match with OS Open Map Local spatial units).

M_Kim: Show any open SME or land use open Datasets for Loughborough available ideally mapped using GIS

M_Kim & TL: Are we mapping land use using MSE data, or land use and SME information? If both need to create new SME data class. How will this be updated?

M_Kim and TL, Does the current Colouring Loughborough 'mixed use' feature under 'Land Use' also require an option for floor level for each different land use to be put in?

M_Kim and TL: Do we want to track change of land use? If so we need to be able to add date

For subsequent meeting

MK: How could moderated bulk upload potentially work?

All: Should we create email verification for sign in?

PH: Showcase/case study section - working with international teams


M_Kim: Show any open SME or land use open Datasets for Loughborough available ideally mapped using GIS. M_Kim & TL: Are we mapping land use using MSE data, or land use and SME information? If both need to create new SME data class. How will this be updated? I've discovered an SMEs dataset on Gov.uk and will bring it along this Wednesday for our review. This includes address information which enables us to map using GIS. * We can designate certain business types as land use categories. PH: We could easily create an additional box which says tick if SME but should also have the date next to it and have options to add new

https://www.gov.uk/government/publications/companies-house-foi-responses-january-to-june-2022/a-list-of-small-and-medium-sized-enterprises-sme-in-the-uk http://download.companieshouse.gov.uk/en_output.html

Regarding land use, are we able to use the data from DEC and EPC? If so, we should discuss how to effectively translate the land use codes to align with the current Colouring London's land use legend. image Or can we create a new DEC / EPC (land use) legend? This opinion is from the meeting with Falli.

PH: Yes definitely can extract land use and age data and excellent to do, Grant and Joe at Birmingham have already worked in age I know

*There's not enough land use data in OSM and I will review it more.

PH: OSM data can be supplemented and corrected using the crowdsourcing feature - we need to make sure the date is added as this is the fastest changing of all our datasets. This will also enable land use history and land use change to be tracked from now which is very useful.

Once we have age and land use data we can use these to infer other characterisatics such as materials and look at probability of accuracy. We can talk to experts in our international group in terms of possibly testing AI and ML approaches on the data as they may well be interested.

M_Kim and TL, Does the current Colouring Loughborough 'mixed use' feature under 'Land Use' also require an option for floor level for each different land use to be put in? I've noticed that in the DEC and EPC data, land use is categorised as 'mixed-use' for entire buildings, without specifying the land use for each individual floor. I'll look into whether there's more detailed information available on this matter.

PH: Thanks - I think providing evidence that retail nearly always will fall on ground floor for example is iactually mportant especially in inference work - ie what are the rules that we can use - this will help with rapid generation of data for Charnwood as a whole to give the regional context for Charnwood and also promote work on the regional prototype. I am going to write this up in the manual too

MKIM1008 commented 6 months ago

From #41 Data: Energy - Issues related to EPC data release


https://www.owenboswarva.com/blog/post-hou2.htm records Owen Boswarva's detailed efforts to gain clarification on permitted reuse of EPC data, from Dept for levelling up, housing and communities (DLUHC), and on the classification of EPC data as personal data subject to GDPR.

Though the outcome was that DLUHC revised view of EPC data as 'personal', in Colouring Cities it is considered an area where sensitivity is required as how much energy each household uses can be used to infer lifestyle, income, activities within tpersonal space. Because of this, though DLUHC EPC ratings will be recorded for each property in Colouring Britain (as they are published by government) they are not currently being visualised by building but aggregated at block level.

Aggregation is also necessary as a buildings may contain mutliple flats each with different EPCs ratings. It is important note that EPC assessments are not carried out by trained surveyors.


‘CURRENT_ENERGY_RATING’ can be added/visualised in the ‘Energy Performance’ data category, under ‘Energy Rating’, to populate the ‘Residential Building Energy Rating’ data

‘BUILT_FORM’ can populate data in Typology, for the Attachment/Adjacency layer . But for this data, we need to add two more data entries/colours on the map (‘Enclosed End-Terrace’ and ‘Enclosed Mid-Terrace’) in addition to the existing ones (‘Detached’, ‘Semi-Detached’, ‘End-Terrace’, ‘Mid-Terrace’).

EPC building age data to 'Architectural Style 'section ( ask Loughborough to prepare) but only able to be done for certain age bands-(EPC age bands are: Pre1900; 1900-1929; 1930-1949; 1950-1966; 1967-1975; 1976-1982; 1983-1990; 1991-1995; 1996-2002; 2003-2006; 2007-2011; 2012 onwards). The ones that can be absorbed into the 'architectural style' section are as follows:

For 'Victorian'- Add 'Before 1900' band - (limited no. of Georgian buildings included in this can be extracted later using Listed building data) For '1946-79 (Post war)' add 1950-1966 and 1967-1975 bands For '1980-1999 (Late 20thc)' add 1983-1990 and 1991-1995 bands For '2000-2025 (Early 21stc)' add 2003-2006; 2007-2011 & '2012 onwards' bands Others bands cannot be integrated as these cross over style dates but quite a lot of data will be able to be visualised using the above.

matkoniecz commented 3 months ago

This seems to be done? Unless we have more datasets?

(feel free to reopen!)

MKIM1008 commented 1 month ago

@polly64

VOA Property attributes including age https://webarchive.nationalarchives.gov.uk/ukgwa/20141002134540/http://www.voa.gov.uk/corporate/Publications/PropertyDetailsGuide/overview.html

polly64 commented 1 month ago

@MKIM1008 I believe these are the domestic ones which VOA won't release at property level only aggregated - but incredibly rich