Closed agrignard closed 3 years ago
RELNO
commented
4 years ago Reiterating from the email chain:
To allow for CityScope to be generic and reusable as possible, it's analysis modules must expect always the same values/data. Therefore mapping should occur as part of the onboarding process for new CS (like we're currently facing in Grasbrook, Aalto, and soon in Mexico). This is roughly how I'd imagine this process taking place:
LAAP
commented
4 years ago Hi, Can we should have Markus and Alex in this conversation. Maybe we can have a face to face meeting.
doorleyr
commented
4 years ago Here's my proposal (which is similar to Ariel's idea above):
Assuming we agree on a list of standard land uses, we also need a number of mapping files. I think we can solve the whole issue with the following text files (examples can be found for Detroit here: https://github.com/CityScope/CS_Mobility_Service/tree/master/scripts/cities/Detroit/mappings):
lu_types.json a list of LU types we agree on. Ideally this is the same for every project.
lu_inputs.json Since different projects will always require different land use options on from-end, we need a to define this list for each project. This is equivalent to the ../header/mapping/type field which we currently store on cityI/O
lu_input_to_standard_lu.json This is a mapping from the land use inputs types to the standard land use types. If a new project has a new set of land use inputs, this mapping must be updated (by us or the partner team).
base_lu_to_lu.json The data we have about land uses at baseline (from GIS files etc.) will never conform to our standard (eg. in Detroit we have data on 28 land uses which seem to be unique to Detroit). This file maps from the land uses types in the data to our standard land use types. If a new project has land use data with different types, this mapping must be updated.
activities_to_lu.json This file defines which activities can take place at which land uses. The activities should match those in the motif example file.
The linked files are based on a “standard” list of land uses which I just made up. But the important point is that if we capture all of these relationships using simple mappings in json files, then it is very easy to update any of this information in the future without changing any line of code (eg. if and when we finally agree on a land use standard) so there is no reason to stall development while we wait for agreement to be reached.
yasushisakai
commented
4 years ago to me there are two things to build consensus.
One approach will be having a centralized hub that contains all of the block definition. The structure of that database is another discussion, but having a unique id(hash) for each definition may be handy for modules to lookup in O(1). Human users will also be able to search for block definitions that are close to their needs. Human Planners can make another pretty portal to curate the types they are specifically interested, more like the early Yahoo approach. The city science standard might by one of them. This is similar to separating the mapping part of today's table and have a dedicated store for that. You can combine or link them if you want.
I would version the format. The table can show which version they are exposing. This version will be better to be serial, but it can be different versions like a branch. The modules can declare which version it wants to see.
As for the format itself, there will be major changes and minor changes. Minor changes are guaranteed to be compatible with the older version, but new modules consuming that data will have interpreted it differently. Adding fields to the JSON file will be a minor change. A Major change is a breaking change, where the modules need to update their parsing mechanism. Deleting a field is a major change(or a hardfork).
LAAP
commented
4 years ago OK, here is my migration from gmail:
@Ronan, the new profiles are aligned with the goals of the Urban Indicators (RADAR), I think that they emerged nicely from the set of data. Even when, probably a different place/country/city will have a different segmentation and behaviour, I think that this is a fantastic baseline.
In my side I have some issues:
1) I am starting to think how can I migrate the impacts per person (in energy, pollution, wellbeing, etc.) that where based in traditional census groups (age, gender, income...) into this new profiles.
2) Unfortunately, today we have no accurate access to all the real data needed for correlating this emerging behaviours with the real impacts in order to have the urban impacts per person emerging from the data: real geolocated emissions of CO2 per hour, or consumption of energy, or "happyness" per area of the city, etc.
3) With that in mind, ABM could help us a lot to simulate this impacts. As soon as we can map the behaviour to land use, mobility mode choice, etc. Then we will have material for understanding the real impact of each of these profiles. With that in mind, I think that we need to start finding out what are the average behavior of each profile, not only in time, but in how they are using the city, the land use, the amenities, the mobility, etc. For this, I will need your help, for sure.
I think that the land use and the mapping are the first step for building this new impacts per person. I think that it is looking good and that will help us to correlate some impacts with the behaviour of people.
RELNO
commented
4 years ago to me there are two things to build consensus. Mapping of the block definition The data format of each table
@yasushisakai can you please provide an minimal example of each? I'm not 100% sure I get the difference
yasushisakai
commented
4 years ago block definition is what kind of information that unit holds. ex:
Small Residence in U.S.: a unit that has X or fewer people living and mostly
consists of lower-income house holds (<30K household income)Small Residence in Japan: a unit that has lower than 20 sqm per head.
The income varies. Site area ratio is restricted to 80%, exceptions may occur.this can be anything and the JSON format is IMO very hard to standardize. Thus, hash it
"898daf":{"name": "Small Residence", "area": "US/detroit", "population": "20",
"population_unit": "sqm", "description":"foo", "income": "30000"},
"ac671b":{"name": "Small Residence", "area":"JP/tokyo",
"name_jp": "第一種住宅専用地域", "sar": "0.8", "income": null, "description":"bar"},
"980ff2":{"rotation": "north", "description": "a unit facing north"},
"8212e":{"co2":"a lot", "energy": "not so much"},
"123aa7":{"ronan_factor": "3rd_place", "kids":"yes", "bike":"0.9", "apples": 1000},
"111111":{"description": "weird land use from hamburg, only hamburg", "people":"are very happy"}this might be one endpoint, for the modules to refer:
https://cityio.media.mit.edu/api/block/898dafthis might be for someone looking for similar block, giving you two entries
https://cityio.media.mit.edu/api/block?search="Small Residence"this might be blessed block definitions that are curated by us.
https://cityio.media.mit.edu/api/blocks/city_scienceIt's not solving the mapping issue, but it gives the infrastructure to communicate and remix to make new definitions that are compatible with modules. A module maker can "inject" info to the block def. to have a table compatible with their module. You can, if needed link them in many ways. The server can provide statistics which block def is popular, and naturally converge.
the Data Format is how it is presented in each table, the same information can be expressed in many different ways. one might be
"header":{"format_ver":"0.5"}
"grid":{"x":1, "y":1, "interactive": true, "landuse": "residence", "rot": 90}Note that it doesn't have to relate to the above block definition, where the table self explains what it is. That's why I said these are independent issues.
this one is close to what we have now
"header":{"format_ver":"1.8"},
"blocks":{"a":"898daf", "b":"980ff2", "c":"765abd"}, // assumes it has the block definitions
"grid":["a", "a", "b", ..., "c", "a"],or an extreme case might be
"header":{"format_ver":"3000.0"},
"grid":[["898daf", "ac671b"], ["898daf"], ..., ["ac671b"]]
but as long as we have the version, the makers of the modules can refer to it.
This is something the cityio currently dictates on how to get the data, I guess it's saner considering the above mess.
yasushisakai
commented
4 years ago It's kinda close (not same) to package managers such as npm but inverted. You have a list of packages, that all have versions. which itself is versioned
MarkusElKatsha
commented
4 years ago Hello: To add to Ariel's entry above, I'm writing about INPUT DATA ONLY; Its important we standardize input data so that modules can be used across tables and so that we can collaborate with other labs developing analysis tools or modules - like the CRE or the CityForm Lab at Harvard.
Another conversation will could develop around CityScience Standards, which aggregate information into types we are looking to identify (ie, walkers, bikers, green buildings, and so on).
Thus far we identified standards for:
Land-Use and Activity - based on the LBCS Schema (see the following link for the directory): https://planning-org-uploaded-media.s3.amazonaws.com/document/LBCS.pdf Schema Description: X 0 0 0 - First level classification - General XX 0 0 - 2nd level - Type XXX 0 - 3rd level - Activity XXXX - 4th level - Specific Activity
Economic Activity - based on the NAICS classifications (see the following link for the definitions activity list): https://www.bls.gov/sae/additional-resources/what-is-naics.htm Note that the NAICS standard is what is used by the USA Census Buro - There is a published crosswalk between the NAICS and the SIC (Standard Industrial Classification)
Schema Description: X 0 0 0 0 - First level classification - Industry Sector (20 broad sectors up from 10 SIC) XX 0 0 0 - 2nd level - Industry Sub sector XXX 0 0 - 3rd level - Industry Group XXXX 0 - 4th level - Industry XXXXX - 5th level - U.S., Canadian or Mexican National specific
The following metics need further research to come up with a standard:
Mobility - ? NHTS (for household mobility): https://nhts.ornl.gov GTFS (for public transit mobility): https://developers.google.com/transit/gtfs
Energy Consumption - ?
Waste Production - ? Carbon-dioxide Production: Tons/Person Solid Waste (Refuse) Production: Tons/Person
What do you think?
agrignard
commented
4 years ago Land-Use and Activity - based on the LBCS Schema (see the following link for the directory): https://planning-org-uploaded-media.s3.amazonaws.com/document/LBCS.pdf Schema Description: X 0 0 0 - First level classification - General XX 0 0 - 2nd level - Type XXX 0 - 3rd level - Activity XXXX - 4th level - Specific Activity
@MarkusElKatsha I don't understand what you mean by "General, Type, Activity, Specific Activity". I would say that the number of digit is related to the granularity of the description or subcategory.
Also by looking more in detail in this standard, it seems important to understand that LBCS is made of five dimensions: activity, function, structure, site, ownership. Do we want/need to keep this 5 dimensions? If we want to be compatible with the LBCS I would say yes. That means that any building should expose 5 attributes related to each dimension. It means that a potential module might be interested by a specific dimension (CRE might be more interested in the ownership dimension) whereas another module could deal with another dimension.
RELNO
commented
4 years ago Also by looking more in detail in this standard, it seems important to understand that LBCS is made of five dimensions
Yes, we'll probably have minimal 20 characters (5*4 of type int xxxx) describing only this entity. This taps into what @yasushisakai described in the concept of hashing the unique types in a searchable dictionary.
yasushisakai
commented
4 years ago I've dug a bit about Germany Land Use for interoperability of the proposed LBCS Schema. https://vtechworks.lib.vt.edu/bitstream/handle/10919/48185/hirt_to_zone_or_not_to_zone.pdf @RELNO, @MarkusElKatsha correct me if this is not the case.
I can personally relate to the German way of thinking since it is (not completely but) similar to planning in Japan. (I've been told they adopted circa WWI.) And I think the proposed LBCS schema can be abused to some extent to fit the German way of planning.
turns out in Germany "Residence" can be translated to α1 0 0 0 + β2 0 0 0 + γ4 0 0 0 + δ5 0 0 0
will the new standard let this happen or put this in consideration?
agrignard
commented
4 years ago Results of the 03/30/20 meetings something like this (with slight modification) can be used even if there is still some discussion especially for the physical attribute
"Basic_CS_type": {
"LBCS": {[1, "0041"] },
"NAICS": [1, "4534"]),
//dynamic attr
"physical": {
"height": [8],
"color": "rgb(255,0,0)",
"density": 100
}
}"Composed_CS_type": {
"LBCS": {[0.2, "0041"],[0.8, "0052"] },
"NAICS": [1, "4534"]),
//dynamic attr
"physical": {
"height": [8],
"color": "rgb(255,0,0)",
"density": 100
}
}Example 1, CS Type - Mixed Use: Ground Level Retail (store and bank), Upper Levels Residential
"CS_Mixed Use": {
"FAR": {[3:1]}
"Ht Limit": {100}
"Level": {[1, 0.25]}
"LBCS": {["2140"]},
"color": "rgb(255,0,0)",
"Density": 60
"Mean Income": Null
"NAICS": ["4534"],
"Employees": 34
"Energy Consumption kWh/Person/Year": 12984
"Water Consumption Ltr/Person/Year": 140240
"Waste Generation Kg/Person/Year": 916
"Level": {[1, 0.75]]}
"LBCS": {["2210"]},
"color": "rgb(255,0,0)",
"Density": 22
"Mean Income": Null
"NAICS": ["5221"],
"Employees": 34
"Energy Consumption kWh/Person/Year": 12984
"Water Consumption Ltr/Person/Year": 140240
"Waste Generation Kg/Person/Year": 916
"Level": {[2-6, 1.00]}
"LBCS": {["1120"]},
"color": "rgb(255,255,0)",
"Density": 42
"Mean Income": 92000
"NAICS": Null,
"Employees": Null,
"Energy Consumption kWh/Person/Year": 12984
"Water Consumption Ltr/Person/Year": 140240
"Waste Generation Kg/Person/Year": 916
}
}Example 2: CS Type - Singe Use: Housing
"CS_Single Use": {
"FAR": {[3:1]}
"Ht Limit": {25}
"Level": {[1-4, 1.00]}
"LBCS": {["1120"]},
"color": "rgb(255,0,0)",
"Density": 54
"Mean Income": 82000
"NAICS": ["Null"],
"Employees": 34
"Energy Consumption kWh/Person/Year": 12984
"Water Consumption Ltr/Person/Year": 140240
"Waste Generation Kg/Person/Year": 916
}
}I agree with differentiating between levels but we can't be explicit about floor numbers (i.e. levels 1-4, 2-6 in the example above) because we don't know how many floors there will be. This is why we use percentages. I think the exception is defining ground floor vs upper floor uses. Below is my proposed edit to the example above where ground floor uses and upper floor uses are defined separately.
{"CS_Mixed Use": { "FAR": 0.75, "Ground": { "color": "rgb(255,0,0)", "proportions":[1], "Density": [60], "LBCS": ["2140"], "NAICS": ["4534"], "Energy Consumption kWh/Person/Year": [12984], "Water Consumption Ltr/Person/Year": [140240], "Waste Generation Kg/Person/Year": [916] }, "Upper": { "color": "rgb(255,0,0)", "proportions": [0.75, 0.25], "Density": [22, 42], "LBCS": ["2210", "1120"], "NAICS": ["5221", "NULL"], "Energy Consumption kWh/Person/Year": [12984, 12984], "Water Consumption Ltr/Person/Year": [140240, 140240], "Waste Generation Kg/Person/Year": [916, 916] } } }
This way, if I place this type somewhere on the table and set it to 13 floors using the UI, we will end up with a ground floor of LBCS 2140, 9 floors of LBCS 2210 and 3 floors of LBCS 1120. If we put this type in another location and set it to 45 floors, we would have a ground floor of LBCS 2140, 33 floors of LBCS 2210 and 11 floors of LBCS 1120.
MarkusElKatsha
commented
4 years ago Hi - I see two issues, how we define or each grid and how the definition gets updated through user interaction. I also hope that the schema we pick is flexible enough to allow for course or more granular definition of each grid.
In my mind, all attributes could be dynamic, depending on the interaction design. For more complex interactions, users may have the opportunity to define where upper level public spaces - things like rooftop parks/gardens or sky bridges and sky malls (as found in NYC, Tokyo, Singapore and Barcelona.) That being said, if complexity is not needed, the grid tile can be defined as single use (or two uses, Ground Level and Upper Levels as shown above:
Example 2: CS Type - Singe Use: Housing
"FAR": 3:1 "Ht Limit": 25 "Level": {[1.00, 1-4]} "LBCS": {["1120"]}, "color": "rgb(255,0,0)", "Density": 54 "Mean Income": 82000 "NAICS": ["Null"], "Employees": 34 "Energy Consumption kWh/Person/Year": 12984 "Water Consumption Ltr/Person/Year": 140240 "Waste Generation Kg/Person/Year": 916
}}
The second issue will be how we incorporate dynamic vs fixed attributes. Each table/setup/interactive... will have an "interaction area" that need dynamic attributes. Depending on what the users can change, we would build in var attributes.
M
LAAP
commented
4 years ago After our last meeting and several conversations with most of you I am convinced that in terms of types, this is how we could operate (At least as a first iteration):
1) Type = Person ID: The data in the string needs to be the minimum information for the modules to understand "who is the type". In USA person has a "Name" and "Last name". In our case the types will have a "LBCS" (Land use) and "NAICS" (Activity). Similar like in the old good times: OL = "Office" (Land use) + "Large/Headquarters/High-income-people" (Activity)
2) Dynamic attributes = Exploration (what if scenarios): The "User" will play with "sliders" to explore the different scenarios.
2.1 - Basic types (mono use): At this point, I think that playing with "Density" and "color" in the basic types will be enough. 2.2 - Mix use types: In "crazy special types", maybe we need to play with 4 "Sliders": "Density, % of "LBCS" and % of "NAICS" and color (To differentiate from other types) 2.4 - New uses: PEV parking etc. = to Mix uses type (at least for now) 2.3 - Special types: Roads, Parks, public spaces, etc. Not sure yet
This apply either for the meta-grid (Static part of the city) if any, as well as in the Dynamic grid (Users "playground) if any.
3) The modules decide: Each module will read this ID, and will say, "OK, with this "name" and "Last name", and this is the "Dynamic attributes", then this are my "Math" (or my "magic" and my "secret or open source library" or whatever thing that the module will do with that information), and this is the output of my module = Number of floors, number of people, diversity of people, Energy consumption, OD Matrix, Water consumption, Pollution, Economic impacts, ESGs metrics, Radar urban indicators, mobility patterns, Wellbeing, heat maps, etc."
Then they will talk with the front end, and the front end will visualize it.
Keeping that in mind, this are the schema that I think can work for the 2 types:
Basic Type:
"Basic_CS_type[out of 20]": { "LBCS": {[1, "0041"] }, "NAICS": [1, "4534"]), //dynamic attr "physical": { "color": "rgb(255,0,0)", "density": 100 } }
Mix use / Crazy Type:
"crazy_type": { "LBCS": {[0.2, "0041", 0.8, "0002", 0.1, "0041",...,”10”, 0.9] }, "NAICS": [0.2, "4534", 0.8, "34435",...,percentile,”type”]), //dynamic attr "physical": { "(%)of"LBCS" ": 75, "(%)of"NAICS" ": 25, "color": "rgb(253,250,0)", "density": 100 } }
Methodology for building the types?:
A - As a starting point: I suggest to have some 10 to 15 CS "pre-cooked types", both, basic and mix uses as a starting point for the open source tool, testing, and basic interaction.
B - New CS Projects: In the kick off of the new projects, the first "workshop" will be focused on defining the "specific types" for that specific place an research question Exactly as we tried to do during the first workshops in Hamburg, in Shanghai, Madrid, Andorra, etc. Where the team spent several nights "building" in Grasshopper and other programs, new types from the students and stakeholders - Now will be muuuch easier!
C - Try catch: All the above doesn't mean that we shouldn't think also about having a "try catch" simple CSV file, with some synthetic data, just in case everything crashes.
MarkusElKatsha
commented
4 years ago I have a few questions/comments after conversations with Arnaud and Luis.
The fundamental questions are: 1 - How is "global information" unique to each site stored? 2 - What is the [minimum] information required for each cell of a grid?
Initially, I had assumed that each cell, whether fixed or dynamic, would include all information about that cell (or tile). But after conversations, my understanding is that we need to decide what is the minimum information needed per cell and that all other data would be part of global file for the context.
If the above is correct, what data is tile specific and what data is global?
BASIC TYPE:
Example 1: CS Type - Singe Use: Housing
"CS_ SINGLE USE": {
"FAR": 3:1
"Ht Limit": 25
"Level": {[1.00, 1-4]}
"LBCS": {["1120"]},
"color": "rgb(255,0,0)",
"NAICS": ["Null"],
"Employees": 34
}
}Example 2, CS Type - Mixed Use: Ground Level Retail (store and bank), Upper Levels Residential
"CS_MIXED USE": {
"FAR": 3:1
"Ht Limit": 100
"Level": {[0.25, 1]}
"LBCS": {["2140"]},
"color": "rgb(255,0,0)",
"NAICS": ["4534"],
"Employees": 34
"Level": {[0.75], 1]}
"LBCS": {["2210"]},
"color": "rgb(255,0,0)",
"NAICS": ["5221"],
"Employees": 34
"level": {[1.00], 2-6]}
"LBCS": {["1120"]},
"color": "rgb(255,255,0)",
"Mean Income": 92000
"NAICS": Null,
}
}
In the above scenarios the Social and Environmental attributes would be part fo the global data file. As Luis mentioned above, we could build a set of "pre cooked" types, then add as needed.
M
doorleyr
commented
4 years ago A few comments/updates I would make to Markus' last suggestion:
As for what is the minimum information, I would say NAICS, LBCS, color and people_per_floor or people_per_sq. After that, additional attributes could be added depending on what modules are being used and the input data they need. This will be case-by-case and require communication within the project team.
As far as I can see, the only significant sticking point is about how we treat different uses on different floors/levels and the answer to this depends mainly on the front-end design. I don't think it's realistic (or desirable) to have a front-end that can edit multiple LU percentages on every floor of a cell. I would say that the FE should only be adjusting the number of street level floors and the number of total floors (the way the FE is now) and the mix of LUs on each level should be encoded in the type. See my last post for type example. @RELNO do you disagree with this?
RELNO
commented
4 years ago FAR 3:1 should be just FAR 3.0 that way it's a float, not a string.
FAR is a tricky piece of data which cannot easily be adopted to many of our case studies. I suggest having FAR as optional indicator (like noise or water).
We don't need to specify a different color for every use. There's no way the front-end is going to have a single floor in 2 colors. I'd say one color per type or at most one color per floor. @RELNO thoughts on this?
There are ways (though complex) of multi coloring elements. However, for brevity we should keep the color per main usage. So if the building/parcel is mostly housing, it will follow housing color. A click on this object will show the breakdown of the different usages.
doorleyr
commented
4 years ago I chatted to Markus a bit more about this and I think we're all on the same page now. I think we can move on to making a list of sample types and trying a first pass of implementation. Do we use the README or wiki for sample types and post the actual types for each project to its table on cityIO?
RELNO
commented
4 years ago OK great. I've been building a dot md renderer so that cityscope.media.mit.edu could serve the docs (same as CSjs). It's ready, however I need @yasushisakai help to deploy it using our url. So in the meantime you can clone the branch #37 cleanup and edit the mds there. We'll deploy in the next few days
MarkusElKatsha
commented
4 years ago FAR can be expressed as two variables:
My sense is that both variables could be dynamic. Please excuse Syntax Errors. Following is an example of ground level Bank and Store, upper levels Residential:
Example 200409, CS Type - Mixed Use: Ground Level Retail (store and bank), Upper Levels Residential
"CS_Mixed Use": {
"FAR": {[foot print = 500, BUA = 1500]}
"Ht Limit": 100
"Ground Level": {[0.25, 1]}
"LBCS": {["2140"]},
"color": "rgb(255,0,0)",
"NAICS": ["4534"],
"Employees": 34
"Ground Level": {[0.75], 1]}
"LBCS": {["2210"]},
"color": "rgb(255,0,0)",
"NAICS": ["5221"],
"Employees": 34
"Upper Levels": {[1.00], 2-6]}
"LBCS": {["1120"]},
"color": "rgb(255,255,0)",
"Mean Income": 92000
"NAICS": Null,
}
}
@MarkusElKatsha , I think this is great, however, I think that this is the task for a backend module.
The minimun information needed is the "%" of NAICS and LBCS in each "cell". The "Urban-Module" in the back will "Mix it" and "produce" a type, based on the local legislation, the interaction from the table (slider giving the number of floors), etc.
agrignard
commented
4 years ago so potentially (new) current one:
"Basic_CS_type": {
"LBCS": {[0.5, "0043"],[0.4, "0043"],[[0.5,0.5], ["0041","0042"]] },
"NAICS": [1, "4534"]),
//dynamic attr
"physical": {
"height": [8],
"color": "rgb(255,0,0)",
"density": 100
}
}"Composed_CS_type": {
"LBCS": {[0.2, "0041"],[0.8, "0052"] },
"NAICS": [1, "4534"]),
//dynamic attr
"physical": {
"height": [8],
"color": "rgb(255,0,0)",
"density": 100
}
}Proposal for compounded types (per @agrignard ):
{
"Composed_CS_type": {
"LBCS": {
"0.2": {
"0.8": "0041",
"0.2": "5554"
},
"0.8": {
"0.8": "0341",
"0.2": "2554"
}
},
"NAICS": { "1": "5455" }
}
}
density in LBCS
@MarkusElKatsha , this book is very important! Please, read it. We should add this reference to our "official text"
doorleyr
commented
4 years ago I created a draft of the CS Types documentation here: https://github.com/CityScope/cityscope.github.io/blob/ronan/docs/md/Land-Use-Types.md
This is based on the multi-level nesting that we agreed on at the end of the last meeting. The format is very similar to @agrignard's example with one important difference. In order to preserve ordering (for floor levels) each attribute must be a list of objects rather then an object of objects.
Having written this out for some pretty simple types, I feel that this is actually too complicated. It will be very difficult for most people to create or even edit types in this format, especially non-CS people. Also, the data transfers with every table update will be very large. Therefore, I think we should just use a single level of nesting. The only thing we lose is the ability to specify multiple types of usage for a single floor which we don't really need.
LAAP
commented
4 years ago Hi @doorleyr ,
I think this is great!
My plan is to build the basic/historical CS types that I am drafting in the architectural way here: https://docs.google.com/spreadsheets/d/17qAILto20ncNKMljwksYA-wsys3b8xHDxWflsA_LjjE/edit?usp=sharing
But that I will define in a "coding way", ASAP. The list today is the following:
| CS Amenities
1 | Hotels
2 | Restaurants
3 | Night live
4 | Leisure and Wellness
5 | Culture
6 | Fashion Shops
7 | Luxury shops
8 | Shopping Centers
9 | Technology shops
10 | Super Markets
11 | Banks
12 | Educational
13 | Post offices
14 | Working places
15 | Parks
16 | Public Transport
17 | Intersection Density
18 | Police and security
19 | Number of Healthy food shops
20 | Household activities (Residential activities)
21 | Transient living (Residential activities)
22 | Institutional living (Residential activities)
23 | Office activities with high turnover of people
24 | Office activities with high turnover of automobiles
25 | Industry
26 | Infrastructure
27 | Transportation
28 | Events
29 | Sports
30 | Natural resources
31 | Nature
In terms of complexity and people building their own type, I think that that is ok, and that is the reason why we are building the City Science CROSSWALK MAPPING as a guide for them to try to build their own decisions when building their type when following our "standards":
Also, part of our job is to help and guide building the types, in a collaborative way with the local stakeholders in any new collaboration
doorleyr
commented
4 years ago In response to Markus: https://github.com/CityScope/cityscope.github.io/commit/0fa176a103a41217e81d70d2fcba043bbb852c79#commitcomment-38725441
I think some measure or utilisation or capacity is important but since we haven't been able to agree on what this should be (people/area, units/area, income level) or if it should even be part of the type, I think we should allow this to be project-specific. So NAICS and LBCS would be the only mandatory attributes.
Regarding the organisation of NAICS and LBCS codes, I would be fine with the way you suggest. I mainly just organised in this way based on the consensus of the last few meetings. This way also gives more flexibility because in some cases, we may include additional attributes which vary independently of NAICS and LBCS.
'P' refers to the proportion allocated to each NAICS/LBCS.
LAAP
commented
4 years ago I agree with @doorleyr
doorleyr
commented
4 years ago I think we're agreed on the structure at least so I'll create draft type lists for Corktown/GB and post them to cityIO so we can start working on implementation. We will still be able to easily iterate on content after this as long as the structure stays the same.
MarkusElKatsha
commented
4 years ago What are the land uses needed for Corktown? Do you have a list? Is it the list Luis published above?
LAAP
commented
4 years ago Hi Markus,
I am already working on this. I have made a first correlation in the Google doc, and I am working into a first-draft-of-a-translation into "code" for this week. I will share ASAP.
The Corktown list is:
"1": "Residential" "2": "Office" "3": "Retail" "4": "Lab_Light_industrial_Makerspace" "5": "Parks _Open_Space" "6": "Parking" "7": "Academic_Institutional_Art_NGO"
doorleyr
commented
4 years ago @LAAP I'm creating a draft of the type list for Corktown today, in the json format, using your spreadsheet as a reference. So don't worry about creating the json structure.
LAAP
commented
4 years ago @doorleyr , Ok, awesome. Thank you! I am making a Json format for the general CityScope types. So after we can also review, share and comment both lines of work
doorleyr
commented
4 years ago I created the examples for Corktown and Grasbrook so we can now work on the implementation: https://cityio.media.mit.edu/api/table/corktown/GEOGRID/properties/types https://cityio.media.mit.edu/api/table/grasbrook/GEOGRID/properties/types
The contents of the types will be subject to change (especially GB as I just made very basic definitions) but we can still start updating the code and testing the modules etc. If the contents of the types change later, everything should still work without any code updates.
LAAP
commented
4 years ago Let me share the first draft of the City Science Types. They are builded by following the following Json structure, agreed in previous talks and issues:
…Here are the proposal for the City Science Types
1 | Hotels
2 | Restaurants
3 | Night live
4 | Leisure and Wellness
4.1 – Pharmacies
4.2 –Dentist
4.3 –Ophthalmologists
4.4 –gyms
4.5 –Spas
4.6 –Clinics
4.7 –Daycare
4.8 –Hospital
4.9 –Nursery
5 | Culture
5.1 –Libraries
5.2 –Museums
5.3 –Cinemas
5.4 –Art-galleries
6 | Fashion Shops
7 | Luxury shops
7.1 –Jewelries
7.2 – Perfumeries
7.3 – Tobacco
8 | Shopping Centers
8.1 – Mall
8.2 – Shopping Centers
9 | Technology shops
9.1 – Vehicle/car sales
9.2 – Electronics
9.3 –Hardware
9.4 – Computer
10 | Super Markets
10.1 – Super Market
10.2 – Convenience
10.3 – Food /Markets
11 | Banks
12 | Educational
12.1 – Universities
12.2 – Schools
12.3 –Profesional training
13 | Post offices
14 | Working places Accelerators, Co-working, Fab lab… TBD???
15 | Parks
16 | Public Transport
16.1 – Bus
16.2 – Taxi
16.3 – Air (Airports “+”)
16.4 – Rail
16.5 – Boat
17 | Police and security
18 | Healthy food shops
19 | Household activities (Residential activities)
20 | Transient living (Residential activities)
21 | Institutional living (Residential activities)
22 | Office activities
22.1 – Public Administration
22.2 – Finance
22.3 – Real Estate
22.4 – Scientific and Technical
22.5 – Management
22.6 –Health Care
23 | Industry
23.1 – Mining
23.2 –Manufacturing
23.3 –Wholesale
23.4 – Accommodation and Food
24 | Nature
This can be the basic City Science Library. It is big, however, covers a big amount of the urban uses. Please, let me know your thoughts
RELNO
commented
4 years ago @LAAP thanks, two notes:
{"menu": {
"id": "file",
"value": "File",
"popup": {
"menuitem": [
{"value": "New", "onclick": "CreateNewDoc()"},
{"value": "Open", "onclick": "OpenDoc()"},
{"value": "Close", "onclick": "CloseDoc()"}
]
}
}}Good job @LAAP
I also have some question/comment:
so yes to save you and us tons of time instead of creating a png than cannot be copy/paste you can use the " ` " to do this
Text that can be copy/paste appears in blue and is not a pngAnd I am actually surprised and a bit worried by the complexity of some (not to say many of them) of types that seems intiaily some simple for instance: manufacturing?food market, clinics etc.What really surprised me it that we chose together the LBCS standrd that was supposed to be easy and almost exhaustif and most importantly something hierarchiacal (the 1, 2 3 digits concept) now end up with a very complex structure made of LBCS just to describe a restaurant for instance.
At the end all this long discussion and structure is finally just to integrate LBSC and NAICS standard right?
Finally, as we seem to converge I am also questioning what will be the process to edit a real city. If we imagine for example the volpe case it's currently made of something like 40x40 cells, we don't want to edit this by hand. @RELNO do you think that the JS grassbrook drag and drop feature can be used to edit any cities? Otherwise, maybe a python script in QGIS can also be explored to edit a .shp or .geojson file
Tiny little thing also I am afraid about the size in terms of memory with this type, image a volpe table full of factory for instance?
doorleyr
commented
4 years ago Thanks @LAAP, this is great. A few comments on the draft and response to @RELNO and @agrignard 's comments:
LAAP
commented
4 years ago Thank to @All,
I totally agree. this was a first iteration trying to cover all the granularity of the NAICS, and maybe useful for @cristian. Maybe his module can take the basic type and reed the granularity in his side.
So, my next steps are:
I can make a simplification of the types.
I can also reduce the number of types to have a very basic short list
I will keep the 2 levels of nesting (As simple as possible)
- I will use the "text" instead of "png",
I have everything in this word document, so @agrignard, not work is waisted:
[City Science Types.docx](https://github.com/CityScope/cityscope.github.io/files/4558778/City.Science.Types.docx)
Please, let me know your thoughts
LAAP
commented
4 years ago | CS Classification Amenities
Dear @all,
Here I share a second iteration. I have reduce the number and complexity of the list. However, we still have a lot of "symple" Classes (29):
3 Residential 11 Amenities (it can be reduce into 7 if needed) 1 Park 1 Transportation 1 Safety 1 Wellbeing 1 Health care 5 Office (it can be reduce into 3 if needed) 4 Industry (it can be reduce into 1 if needed) 1 Nature
29 Total
It can be reduced into 20 Classes
After talking to @MarkusElKatsha we have agreed on the following structure for the Class:
A) Static attributes: "LBCS", "color", "NAICS" B) Dynamic attributes: "Economic", "Height" and "Density"
Example of the structure:
{
" CS_Name ": {
"LBCS": {"LBCS-Code":"1"}
},
"color": {"rgb(rgb-Code)}"
},
"NAICS": {"NAICS-Code":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}IMPORTANT NOTE: A type will include one or multiple CS Classifications depending on the context and project needs
RESIDENTIAL
1 | Household activities
{
" CS_Household_activities ": {
"LBCS": {"1100":"1"}
},
"color": {"rgb(255,255,0)}"
},
"NAICS": {"000000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}2 | Transient living
{
" CS_Transient_living ": {
"LBCS": {"1200":"1"}
},
"color": {"rgb(255,255,117)}"
},
"NAICS": {"000000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}3 | Institutional living
{
" CS_Institutional_living ": {
"LBCS": {"1300":"1"}
},
"color": {"rgb(123,123,0)}"
},
"NAICS": {"000000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}AMENITIES
4 | Hotels
{
" CS_Hotel ": {
"LBCS": {"1300":"1"}
},
"color": {"rgb(255,255,100))"
},
"NAICS": {"721110":"1"},
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}5 | Restaurants
{
" CS_Restaurant ": {
"LBCS": {"2500":"1"}
},
"color": {"rgb(255,0,100))"
},
"NAICS": {"720000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}6 | Night live
{
" CS_Night_live ": {
"LBCS": {"2540":"1"}
},
"color": {"rgb(255,100,100))"
},
"NAICS": {"620000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}7 | Leisure and Wellness
{
" CS_ Leisure_and_Wellness ": {
"LBCS": {"6500":"1"}
},
"color": {"rgb(47,79,79))"
},
"NAICS": {"722500":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}8 | Culture
{
" CS_Culture": {
"LBCS": {"5000":"1"}
},
"color": {"rgb(190,190,190))"
},
"NAICS": {"710000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}9 | Fashion Shops
{
" CS_Fashion_Shop ": {
"LBCS": {"2100": "1"}
},
"color": {"rgb(255,145,145))"
},
"NAICS": {"541490": "1”}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}10 | Luxury shops
{
" CS_Luxury-shops ": {
"LBCS": {"2100": "1"}
},
"color": {"rgb(190,47,47))"
},
"NAICS": {"423940":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}11 | Shopping Centers
{
" CS_Shopping_Centers ": {
"LBCS": {{"2100":"0.75 ", "2500": "0.25"}:"1"
},
"color": {"rgb(156,0,0))"
},
"NAICS": {{"440000":"0.50", "450000": "0.50"}:"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}12 | Super Markets
{
" CS_Super_Market ": {
"LBCS": {"2100":"1"
},
"color": {"rgb(150,61,61)}"
},
"NAICS": {"445110":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}13 | Banks
{
" CS_Bank ": {
"LBCS": {"2200":"1"}
},
"color": {"rgb(122,2,2)}"
},
"NAICS": {"450000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}14 | Educational
{
" CS_Educational ": {
"LBCS": {"6100":"1"}
},
"color": {"rgb(3,107,107)}"
},
"NAICS": {"610000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}PARK
15 | Parks
{
" CS_Park ": {
"LBCS": {"0000":"1"}
},
"color": {"rgb(2,139,2)}"
},
"NAICS": {"712190":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}TRANSPORTATION
16 | Transportation
{
" CS_Transportation": {
"LBCS": {"4000":"1"}
},
"color": {"rgb(0,0,255)}"
},
"NAICS": {"480000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}SAFETY AND SECURITY
17 | Public Safety
{
" CS_Public_Safety ": {
"LBCS": {"6400":"1"}
},
"color": {"rgb(75,75,255)}"
},
"NAICS": {"480000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}WELLBEING
18 | Healthy food shops
{
" CS_Healthy_food ": {
"LBCS": {"2500":"1"}
},
"color": {"rgb(198,176,32)}"
},
"NAICS": {"445200":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}19 | Health Care
{
" CS_Health_Care ": {
"LBCS": {{"6500"}:"1"
},
"color": {"rgb(124,206,206)}"
},
"NAICS": {"620000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}OFFICE
20 | Public Administration
{
" CS_Public_Administration_office ": {
"LBCS": {{"6200":"0.50", "6300": "0.50"}:"1"
},
"color": {"rgb(109,160,160)}"
},
"NAICS": {"920000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}21 | Finance
{
" CS_Finance ": {
"LBCS": {{"2200"}:"1"
},
"color": {"rgb(182,11,79)}"
},
"NAICS": {"520000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}22 | Real Estate
{
" CS_Real_Estate_office ": {
"LBCS": {{"2300"}:"1"
},
"color": {"rgb(182,24,87)}"
},
"NAICS": {"530000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}23 | Scientific and Technical
{
" CS_Scientific_and_Technical ": {
"LBCS": {{"2400"}:"1"
},
"color": {"rgb(145,49,81)}"
},
"NAICS": {"540000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}24 | Management
{
" CS_Management ": {
"LBCS": {{"2400"}:"1"
},
"color": {"rgb(145,49,81)}"
},
"NAICS": {"550000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}INDUSTRY
25 | Mining
{
" CS_Mining ": {
"LBCS": {{"8000"}:"1"
},
"color": {"rgb(34,139,34)}"
},
"NAICS": {"210000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}26 | Manufacturing
{
" CS_Manufacturing ": {
"LBCS": {{"3000"}:"1"
},
"color": {"rgb(160,32,240)}"
},
"NAICS": {{"320000":"0.50", "330000": "0.50"}:"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}27 | Wholesale
{
" CS_Wholesale ": {
"LBCS": {{"3500"}:"1"
},
"color": {"rgb(109,0,177)}"
},
"NAICS": {"420000":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}28 | Accommodation and Food
{
" CS_Accommodation_and_Food ": {
"LBCS": {{"1300":"0.50", "3100": "0.50"}:"1"}
},
"color": {"rgb(105,71,126)}"
},
"NAICS": {"722310":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}NATURE
29 | Nature
{
" CS_ Nature ": {
"LBCS": {{"9000"}:"1"
},
"color": {"rgb(255,255,255)}"
},
"NAICS": {"712190":"1"}
//dynamic attr
"Economic": [?],
"Height": [?],
"Density": [?]
}
}Also, here is the document in Word
200501_City Science Types V01.docx
Please, let me know your thoughts and ideas
agrignard
commented
4 years ago @LAAP great job!
I would be in favor if it possible to reduce it to 20 (I think we can already do a lot) and it will be already challenging to integrate all of them and also easier to maintain with fewer
LAAP
commented
4 years ago Thank you @agrignard ,
Please, @RELNO, @doorleyr , @MarkusElKatsha , let me know if I should reduce the classes into 20 as @agrignard suggested or Keep it as it is now in 29
RELNO
commented
4 years ago @LAAP I'm agnostic to how many, the goal is to have the minimal number of types to start a simple table, and a baseline to extend it later. I do have a few comments on the format:
"color": {"rgb(0,0,255)}"
I'd prefer using hex color scheme (https://www.color-hex.com/) which is thinner in size, reduces errors due to wrong ',' location etc. Also, we don't need to explicitly mention {"rgb(.
//dynamic attr is not part of a json, it was a leftover from past discussion
I might missed something, but what does "Economic": [?], mean?
LAAP
commented
4 years ago @agrignard and @RELNO ,
Thanks for the comments. Ok, this is my plan:
1) Reduce the types in this iteration to the "minimum - minimum - minimum" 2) Replace the rgb to ch
Question to @RELNO : What action is required to answer this: "//dynamic attr is not part of a json, it was a leftover from past discussion"? Remove the text?
Answer to @RELNO : Talking to Markus, we realize that economic is a missing parameter: High income, low income, etc.... The same LBSC+NICS + same m2 element is different if it is in a high income community or in a lower one.... Where are we going to get this info/data? No idea yet
RELNO
commented
4 years ago Question to @RELNO : What action is required to answer this: "//dynamic attr is not part of a json, it was a leftover from past discussion"? Remove the text?
yes
Answer to @RELNO : Talking to Markus, we realize that economic is a missing parameter: High income, low income, etc.... The same LBSC+NICS + same m2 element is different if it is in a high income community or in a lower one.... Where are we going to get this info/data? No idea yet
I think we should be a bit more thorough here: adding economic indicator is something I'd want a module to do, not engrave it in the type itself. Also, If we go with economy, why not energy, or any other indicator?
LAAP
commented
4 years ago @RELNO ,
Ok. Thanks. I will Change that too.
Regarding economics. Yes, a module can do the job. Same thing could happen with "density" of people and "height", however we have decided to put it in the dynamic attributes, isn't? I think that we can change the name of "economic" for other less "institutionalized" maybe "Worth" (?)
This is an attempt to have a common place to discuss on the unification task. Nothing is set in stone but at least let's try to converge on a common place to discuss.
What we have so far (not pretending to have an exhaustive view so feel free to complete/comment)
So here the current entry point that should be used to unify everything and that should be included in the wiki if everyone agree:
wiki https://github.com/CityScope/cityscope.github.io/wiki/CityScope-Data-Format-version-2.0 (it has been moved from the cityIO repo to here). It is pretty weak for now but this is what we have (this is more or less the RS, RM, RL OS,OM,OL, etc) and it will hopefully quickly evolve as a synthesise of this issue in the following days
Other non github reference (for non github user), is the paper an overleaf initiated by @LAAP that should be also used as a reference https://www.overleaf.com/project/5dc19bd7180565000164bd25
Concerning cityIO https://github.com/CityScope/CS_CityIO/wiki/cityIO-Data-Structure
Concerning ABM Standard: https://github.com/CityScope/CS_Simulation_GAMA/wiki/GIS-Structure here the documentation of the input that a volpe like model is requiring to run https://github.com/CityScope/CS_Simulation_GAMA/wiki/Game-IT for what we call the Lyon Model
Concerning Macro Mobility https://github.com/CityScope/CS_Mobility_Service there is some standard here used to drive the ABM micro mobility model (both macro and micro) we can think on how to migrate it to this repo or at least make a link
Current github issues related to this topic: https://github.com/CityScope/CS_Simulation_GAMA/issues/66 https://github.com/CityScope/CS_cityscopeJS/issues/5
I am sure there is other entry point so please feel free to complete it.