Define the land use types using the CityScope Types system

doorleyr commented 3 years ago

In order for CS modules to understand the land use changes in each scenario, each of these land uses (eg. Educational, Commercial, Public Facility) must be defined using the Types System. At minimum, each 'Type' should have a defined mix of NAICS codes and LBCS codes. These should be in a json file. For an example, please see the Corktown types definitions.

dangbuingochan commented 3 years ago

Hi Ronan,

I have uploaded land use type code here. Please support us matching these land use types to NAICS and LBCS as Marcus mentioned before

LAAP commented 3 years ago

Hi @doorleyr and @dangbuingochan , I think that @Markus and I will make a first pass of the correlation between NAICS and LBCS and the proposed types. We will share it ASAP

LAAP commented 3 years ago

Dear @doorleyr and @dangbuingochan ,

Please let me share the 1st pass through the Types document that we have made. We recommend to have a-meeting and re-visit the list together:

List of landuse type name (landuse-1) R-M_L.xlsx

In the document, you will find: -In Orange, some land-uses that looks like very similar between them to us, so we have tried to find a correlation with the NAICS and LBCS based in our educated guest -In red, Land uses totally repeated or "time related" (New, existing"): Review it together -In dark blue: types that needs to be made from scratch -In light blue: Mix uses -Green: Land use only existing in LBCS and not NAICS

All this types are being correlated by using the codes in the following link:

https://planning-org-uploaded-media.s3.amazonaws.com/legacy_resources/lbcs/background/QLBCSConvFunction2NAICS.TXT

Here is the table:

LIST OF LANDUSE TYPE NAME (LANDUSE-2)
ID	Type Code	Landuse type name	FunctionCod	FunctionDescription	LBCSGlobalID	NAICSCode	NAICSDescription	NOTES	More definition needed	ARC Answer
1	CCDVO	Administrative	2421	Office and administrative services	-1350379339	561110	Administrative management services	Same?	Yes
2	CCDVO	Administrative - office	2421	Office and administrative services	-1350379339	551114	Centralized administrative offices
3	CCDVO	Administrative facility	2422	Facilities support services	-856508974	561210	Facilities
4	CCDVO	Adninistrative	2421	Office and administrative services	-1350379339	551111	Bank holding companies	Same?	Yes
5	CXCL	Buffer greenery	5230	Zoos, botanical gardens, arboreta, etc.	-100071090	712130	Arboreta
6	HTKT	Bus (station)	4134	Interurban, charter bus, and other similar	1007755198	485510	Bus charter services
7	CXCL	Canal - side greenery	4332	Irrigation and industrial water supply	1088837106	221310	Canal, irrigation	Same?	Yes
8	CXCL	Cau Phao canal side greenery	5370	Fitness, recreational sports, gym, or	-1713573402	713990	River rafting, recreational	Same?	Yes
9	CCDT	City public facility	6200	Public administration	1095099480	923130		?	Yes
10	CCDVO	Commercial	2321	Commercial property-related	-669437911	531120
11	CCDVO	Commercial/ service	7130	Industrial, commercial and institutional	53936015	233320		?	Yes
12	HH	Complex	1320	Rooming and boarding	575361369	721310	Clubs, residential	?	Yes
13	CCDVO	Complex - hospitality	6140	Technical, trade, and other specialty schools	197903784	611519	Hospitality management schools
14	NNO	Complex - primarily residential	2320	Property management services	631894472	531311	Condominium managers' offices, residential
15	NNO	Complex - residential	2322	Rental housing-related	-1695396554	531110	Building, residential, rental or leasing
16	NNO	Complex apartment building land	7110	Residential construction	-1779403079	233220	Apartments
17	CCDT	Complex land - cultural/ recreational	5370	Fitness, recreational sports, gym, or	-1713573402	713990	recreational	SAME Than 8?	Yes
18	CCDT	Complex land - hotel/ cultural/ recreational	1330	Hotel, motel, or tourist court	1797457336	721110
19	CCDVO	Cultural	5110	Theater, dance, or music establishment	247511209	711120			Yes
20	CCDVO	Cultural - club	6830	Civic, social, and fraternal organizations	-1346381961	813410	Alumni clubs
21	GD1	Education (elementary school, junior school, high school)	6121	Elementary	1778376337	611110	Elementary schools	SAME
22	NCDT	Education (university)	6130	Colleges and Universities	960459683	611310	Academies, college or university
23	NCDT	Education and research centre	2416	Research and development services	-876225255	541720
24	GD1	Educational	5210	Museum	1652864544	712110	Community museums	?	Yes
25	GD1	Educational (kindergarten)	6121	Elementary	1778376337	611110	Kindergartens	SAME
26	TGDT	Existing religious	6600	Religious institutions	-2133182482	813110
27	NNO	Existing and new residential						Combination of 14 and 15 ?	Yes, why "EXISTING"?
28	CCDVO	Existing and renovated public facility						SAME Than 9	Yes, why "EXISTING"?
29	NNO	Existing and renovated residential						SAME Than 14?	Yes, why "EXISTING"?
30	GD1	Existing educational						SAME Than 24?	Yes, why "EXISTING"?
31	KTBB	existing industrial land	7130	Industrial, commercial and institutional	53936015	233320		Same than 13?	Yes, why "EXISTING"?
32	CCDVO	Existing market	2151	Grocery store, supermarket, or bakery	-2086917798	445210			Yes, why "EXISTING"?
33	CCDVO	Existing medical	6513	Medical and diagnostic laboratories	2133988457	621511			Yes, why "EXISTING"?
34	CCDVO	Existing public facility						SAME Than 9	Yes, why "EXISTING"?
35	NNO	Existing residential						Combination of 14 and 15 ?	Yes, why "EXISTING"?
36	CCDVO	Existing service facility						SAME Than 3?	Yes, why "EXISTING"?
37	NNO	Existing, renovated and new residential						SAME Than 14?	Yes, why "EXISTING"?
38	GD1	Expected school	6125	Alternate education services	728998317	611110	Schools for the handicapped, elementary or secondary	SAME Than 21?	Yes, why "EXPECTED"?
39	HTKT	Ferry parking lot	4151	Marine passenger transportation	1197173560	483114	Ferry passenger transportation, Great Lakes
40	CXCD	Ferry square						SAME Than 40?	Yes
41	HTKT	Ferry wharf						SAME Than 40?	Yes
42	CXDT	Greenary - park land	5500	Natural and other recreational parks	1112968241	712190
43	CXDVO	Greenary land in residential unit						TBD	Yes
44	CXCL	Greenary park land - landscape						SAME Than 42	Yes
45	CCDVO	Hospital - medical facility	6530	Hospital	2002239380	622210
46	CQCT	Landmark office tall building						SAME Than 2	Yes
47	CXCL	Landscape greenary						SAME Than 42	Yes
48	CXDVO	Landscape greenary - park						SAME Than 42	Yes
49	NNO	Low - BCR residential						SAME Than 27	Yes
50	NNO	Low - rise apartment						SAME Than 27	Yes
51	CCDVO	Medical	6511	Clinics	-968270205	621112			Yes
52	CCDVO	Medical (expected)						SAME Than 45 or 51 ?	Yes, why "EXPECTED"?
53	ANQP	Militaries	6310	Military and national security	1738929483	928110	Army
54	HH	Mixed - use						MIX OF Types	Yes
55	NNO	Mixed - use residential						MIX OF Types	Yes
56	HH	Mixed - use residential until 2020						MIX OF Types	Yes
57	NNO	Multi-functional residential						MIX OF Types	Yes
58	CCDVO	Multi-purpose sports stadium	5140	Promoter of performing arts, sports, and	560476372	711310	Stadium operators
59	CCDVO	Multi-function commercial	2321	Commercial property-related	-669437911	531120
60	ANQP	National defense - security						SAME Than 53	Yes
61	CCDVO	New city service facility						SAME Than 9	Yes, why "NEW"?
62	NNO	New high - rise residential						SAME Than 14?	Yes, why "NEW"?
63	NNO	New residential						SAME Than 14?	Yes, why "NEW"?
64	NCDT	New school	6120	Grade schools	-517307696	611110	School boards, elementary and secondary	SAME Than 24?	Yes, why "NEW"?
65	CQCT	Office						SAME Than 2	Yes
66	CQCT	Office - production and trading	2230	Investment banking, securities, and	-1819676217	523130	Trading companies, commodity contracts
67	CXCL	Open space alongside pavement						TBD	Yes
68	CXDVO	Park greenery						SAME Than 42	Yes
69	CXDVO	Parkland - sport	6147	Sports and recreation education	1883269907	611620			Yes
70	CXCD	Pedestrian road	4130	Road, ground passenger, and transit	-313322862	488490
71	KTBB	Production land - warehouse	3600	Warehouse and storage services	-943416425	488991
72	CCDVO	Public facility						SAME Than 9	Yes
73	CCDVO	Public facility in residential unit						MIX OF Types	Yes
74	CXDVO	Public greenery	7110	Neighborhood or local park				Only LBCS	Yes
75	CXDVO	Public park	7120	Community park				Only LBCS	Yes
76	TGDT	Religious	6600	Religious institutions	-2133182482	813110
77	TGDT	Religious - relics	3510	Durable goods	1519736895	421210	Religious furniture wholesaling
78	NNO	Renovated residential						SAME	Yes
79	NNO	Residential						SAME	Yes
80	CCDT	Resort/ outdoor entertainment						SAME	Yes
81	NNO	Rise residential						SAME	Yes
82	GD	School	6123	Senior	1144064807	611110	Boarding schools, secondary
83	CCDVO	Service - commercial	2424	Business support services	-1802921904	561421
84	GD1	Short-term planned elementary school						TBD	Yes
85	NNO	Short-term planned residential						TBD	Yes
86	CXDVO	Short-term planned urban greenary land						TBD	Yes
87	CXDVO	Sport - greenery						SAME	Yes
88	CXDT	Sport - greenery park						SAME	Yes
89	CXCD	Square						TBD	Yes
90	CXCD	Square ground						TBD	Yes
91	HTKT	Technical infrastructure	7440	Power lines, communication and transmission	-1822416901	234920
92	CXCD	Thematic greenery	5500	Natural and other recreational parks	1112968241	713990	Picnic grounds
93	CXCD	Thematic park	5310	Amusement or theme park establishment	840075308	713110	Parks (e.g., theme, water), amusement
94	CXCD	Thematic park (amusement park)	5310	Amusement or theme park establishment	840075308	713990	Amusement ride concession operators
95	DGT	Transportation	4120	Rail transportation	-1889786889	482112			Yes
96	CXCL	Vegetative buffer						TBD	Yes
97	NNO	Villass						?¿
98	MN	Water surface	7242	National lakeshore				Only LBCS	Yes
99	CXCL	Waterside vegetative buffer						TBD	Yes
100	DK	Wetland						TBD	Yes

Leon-Carto commented 3 years ago

Hi @LAAP , thank you for your supporting. Data team will review the table quickly and set up a meeting.

LAAP commented 3 years ago

OK. Perfect. Thank you @nqlong-vlab

agrignard commented 3 years ago

Just to be clear on this at some point those land use will have to be updated in the corresponding GIS file that we already agreed together here https://github.com/CityScope/CSL_HCMC/issues/1

doorleyr commented 3 years ago

This issue is now the main bottleneck towards having working CityScope model. We usually only have ~10 types for a CityScope table because any more than that makes the user experience too complicated. Many of the types in the list above are very similar and can be consolidated into one type. Also some of the types are referring to future plans (eg. Expected School) and this is a problem. These areas should simply be coded as what exists there in the scenario (this can be None if the area has no current function). I suggest we add a 'CityScope Type' column to this table and assign every row to one of ~10 types. Then we can have another table of ~10 rows which assigns the NAICS, LBCS etc. to the CityScope types.

Leon-Carto commented 3 years ago

I agree, so I will add the column as CityScope type.

LAAP commented 3 years ago

@Markus and I are working on this.

Tonight we will have have a very simplified version with the 14 types that ARC team has selected and correlated just with the LBCS

LAAP commented 3 years ago

From @Markus:

_Hello All - @LAAP Luis and I went through the Land Use and added a consolidated list to the right of the sheet (Columns U-V). Note that Mixed Use is a combination of land uses in one cell... IE if a building includes residential and retail space, that cell would include 1000, and 2000 for the uses.

Also note that we had included a very good reference document on the project page under the references. Please see the following link: https://planning-org-uploaded-media.s3.amazonaws.com/legacy_resources/lbcs/background/pdf/rslucm2sic2naicsnotext.pdf_

The final 14 basic land uses can be:

Hai-Hoang-88 commented 3 years ago

I agree, so I will add the column as CityScope type.

Great, @nqlong-vlab please refer to the link while you create additional "City Scope Type" column for: https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=1953133771.

LAAP commented 3 years ago

Dear @Hai-Hoang-88 ,

After talking with Ronan, we need also to add:

The Naics code(s) [Luis and Markus] The density (sqm/person) where applicable [ARC] A color ( I think this should roughly follow lbcs [Luis and Markus]

We will do it As soon as possible

LAAP commented 3 years ago

Dear @Hai-Hoang-88 ,

Please find the update in the types, we have added to the table the The Naics code(s)and the color, as @doorleyr has suggested. The only thing missing is the The density (sqm/person) where applicable:

You can find it in the link:

https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit?usp=sharing

Columns: U, V, W, X, and Y, rows: 2 to 16th

Hai-Hoang-88 commented 3 years ago

The excel file has been updated, please take a look @doorleyr

LAAP commented 3 years ago

Awesome @Hai-Hoang-88 , Thanks for adding the average sqm. So, The final numbers are:

Hai-Hoang-88 commented 3 years ago

A few updates from data team is adding that 14 cityscope landtypes in shapefile for all scenarios.

LAAP commented 3 years ago

Awesome!

TuCTruong commented 3 years ago

Thanks, @Hai-Hoang-88 for your help in updating the information.

Dear @LAAP , @doorleyr and All, after working with the Data team and Hai, I d like to revise the number and explain to you a little bit about the table above.

As I mentioned on Whatsapp, the value sqm/person we have provided above is for land area. (Since we don't have a specific sqm/person for floor area ready in our planning documents. When working on urban planning, this value will be considered on a case-by-case basis, based on regulations, standards, planning on different scales, etc.).

However, as far as I understand, we need both sqm/person for (1) land use and for (2) the floor area.

Seems like the density for land uses for each scenario can be useful for us in informing our storytelling and the urban planning analysis.
At the moment, we also need firstly the density in the building, to calculate occupancy, for what we are still missing (jobs mostly).

For that, I have updated the table with information as follow:

Please refer to column C for the sqm/person in the floor area for the building. (Some functions won't have this info. specifically. This value will be consistent. Indeed, it is the minimum value, any building can apply a higher value --> provide more spacious standard).
Column D for the sqm/person in land use regulation. (This value in each scenario may vary according to the planning implementation process)
Please refer to Reference from statistics from the Data team in Column F-N, Row 1-17 for detailed numbers in land use and the population for each scenario. I also added the according density value.

You may find this is in the same link as the chat above, Sheet 2: https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=1172435229

Hope that it explained.

doorleyr commented 3 years ago

@TuCTruong thanks. The figure we're looking for is sqm per person per floor, assuming that the floor occupies the entire area of the grid cell. If the building footprints will occupy less than the full area, this should be factored into the density value.

For example, if we have a grid cell of 50m x 50m which is assigned to 10 floors of Residential, using the figure above of 8sqm/person, we would calculate that (50x50x10/8) = 3125 people can live in this grid cell.

Can you confirm that your figures are correct for this calculation or revise them if necessary?

doorleyr commented 3 years ago

Apart from the density values, here still a few other issues with the CSL types which need to be addressed:

the CSL type names in the land use shape files include a 'Water' type whereas the type definitions include a River(Water) type. The type names must be exactly the same in the type definitions and in the LU files.
The types include a 'Mix-used' type and it's attributes are still undefined. This type needs to be properly specified. If this is, for example, a combination of Commercial and Retail, then the NAICS and LBCS codes should reflect this. It is possible for a type to have X% of one NAICS/LBCS code and Y% of another code.
The different attributes of the types are currently specified in a few different places: as comments in this issue and in external google sheets. The HCMC team should provide a single file inside the repo which specifies the attributes of the types. Ideally this should in json format like this. However, if this is difficult, you can provide it as a csv like this

TuCTruong commented 3 years ago

@TuCTruong thanks. The figure we're looking for is sqm per person per floor, assuming that the floor occupies the entire area of the grid cell. If the building footprints will occupy less than the full area, this should be factored into the density value.

For example, if we have a grid cell of 50m x 50m which is assigned to 10 floors of Residential, using the figure above of 8sqm/person, we would calculate that (50x50x10/8) = 3125 people can live in this grid cell.

Can you confirm that your figures are correct for this calculation or revise them if necessary?

Hi @doorleyr ,

Yes then the figure in Column C will do it. I have updated the reference for the number and assumption.

You may find this in the same link: https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=1172435229

LAAP commented 3 years ago

Dear @doorleyr and @TuCTruong,

Please, let me answer @doorleyr questions:

1) The CSL type names: @Markus and I have adapten the definitions of our CS types to the ARC definitions, so now "Water" is the correct name for this land use. We encourage @TuCTruong and the ARC team to do the same in the geo-referenced files that are being produced for the 3 Scenarios by the ARC team

2) 'Mix-used': @Markus and I have made a mixed use type with 70% of "Residential" and 30% of o "Commercial, service, office". Please, let us know if that works for you

3) Centralize attributes of the types in a CSV file: @Markus and I have centralized all the attributes in this CSV file (cs_types_2.0.csv):

Please, let us know if anything else is needed

Un abrazo

Luis

LAAP commented 3 years ago

Dear @TuCTruong,

We have take the missing density of office and healthcare (in red) from the following link. It is not the most scientific link, but sometimes helps in order to have an OK place holder and move ahead.

Regarding your Excel file, I have some suggestions, now that we have no scenario 1, That it was messing up everything. My recommendation is not to have different densities in each scenario: if we change the density in each scenario, then we have 14 types x by 3 scenarios = 42types. Keeping this in mind. And since we don't have any more scenario 1; I recommend that If we want to play with the densities, we should stick with the 14 types and just change the numbers of floors:

TuCTruong commented 3 years ago

Dear @TuCTruong,

We have take the missing density of office and healthcare (in red) from the following link. It is not the most scientific link, but sometimes helps in order to have an OK place holder and move ahead.

Regarding your Excel file, I have some suggestions, now that we have no scenario 1, That it was messing up everything. My recommendation is not to have different densities in each scenario: if we change the density in each scenario, then we have 14 types x by 3 scenarios = 42types. Keeping this in mind. And since we don't have any more scenario 1; I recommend that If we want to play with the densities, we should stick with the 14 types and just change the numbers of floors:

Hi, thanks Luis,

Yes, I agree with the scenario and density. It was for reference only. So, I'll hide scenario 1 in the file. Also thanks for the ref. on room area. It simplifies all these numbers.

TuCTruong commented 3 years ago

Dear @doorleyr and @TuCTruong,

Please, let me answer @doorleyr questions:

1) The CSL type names: @markus and I have adapten the definitions of our CS types to the ARC definitions, so now "Water" is the correct name for this land use. We encourage @TuCTruong and the ARC team to do the same in the geo-referenced files that are being produced for the 3 Scenarios by the ARC team

2) 'Mix-used': @markus and I have made a mixed use type with 70% of "Residential" and 30% of o "Commercial, service, office". Please, let us know if that works for you

3) Centralize attributes of the types in a CSV file: @markus and I have centralized all the attributes in this CSV file (cs_types_2.0.csv):

Please, let us know if anything else is needed

Un abrazo

Luis

Dear @doorleyr , @LAAP ,

I have checked with @dangbuingochan , our data team has changed the land use to Water in the 3 scenarios.
For the mix-use. There is actually not very clear guidance for us, the 70-30 sounds logical to me. I have put a little ref. on the same file to compare with the land use and seems like it looks perfect. I think we are okay to stick with this.

I have questions here:

In the 14 land-use types, first, shall we remove the number before the name. I've put it there as a reference code for our data team to proceed with it only, it doesn't have any specific meaning for us.
Second, we have the type of "industrial" and "wetland", which are 0.0 in all of our scenarios. Shall we keep them or shall we remove them?

Many thanks, Tu

doorleyr commented 3 years ago

@TuCTruong Can you please remove the numbers before the names and ensure that the types names are always the exact same in terms of whitespace, punctuation etc.

If there are any types which never appear in the scenarios then I think they should be removed.

LAAP commented 3 years ago

Dear @TuCTruong and @dangbuingochan,

I have a question about the art of the excel where you show the 3 scenarios. I can see that the "m2" doesn't increase in scenario 2 and 3 (It si always 4178668.5 m2), even when, at this scenarios we are incrementing the density of buildings (Towers and high buildings are planned). So I suppose that you are just sowing the m2 of land, and not the "m2 constructed" (or m2 of floors in buildings. Keeping that in mind, I wonder if you could provide to me the constructed m2 per land use. That will help on the storytelling:

Thank you very much in advance

Un abrazo

Luis

TuCTruong commented 3 years ago

Hi @LAAP ,

I and @dangbuingochan have updated this table with values from Data team. You may find the constructed area in columns J, O, R. Made it by (footprint x number of floor). Please check the same link.

LAAP commented 3 years ago

Dear @TuCTruong , Thank you very much. This is very useful!

LAAP commented 3 years ago

Dear all, Since it looks like we have solved the types I am closing this issue.

Please, feel free to open it again if needed

doorleyr commented 3 years ago

After running some analysis of scenarios and taking a closer look at the types, there appear to be a few issues to be solved:

Density: the densities of the land uses are too high (i.e. sqm_per_person is too low) and this is causing the residential and working population estimates to be much too high in all scenarios. eg. in the baseline scenario, the site population works out as 5 million and the working population of the site is 83 million. Note that for any type which has an associated NAICS code, the density will be used to calculate the number of employees, not the number of visitors. Here is an example reference for the US which may be helpful: https://www.engineeringtoolbox.com/number-persons-buildings-d_118.html
Missing functions: none of the types have any food & beverage or accommodation (72xxxx) or retail (44xxxx and 45xxxx) codes. This affects the mobility model because agents are attracted to food, beverage and retail locations. It also means we cannot show heatmaps or indicators related to accessibility to shopping, groceries, restaurants, nightlife etc. This can be solved by (i) adding new types for these functions, (ii) editing existing types so that some percentage is allocated to these NAICS codes, or by a combination of (i) and (ii).

TuCTruong commented 3 years ago

After running some analysis of scenarios and taking a closer look at the types, there appear to be a few issues to be solved:

Density: the densities of the land uses are too high (i.e. sqm_per_person is too low) and this is causing the residential and working population estimates to be much too high in all scenarios. eg. in the baseline scenario, the site population works out as 5 million and the working population of the site is 83 million. Note that for any type which has an associated NAICS code, the density will be used to calculate the number of employees, not the number of visitors. Here is an example reference for the US which may be helpful: https://www.engineeringtoolbox.com/number-persons-buildings-d_118.html

Missing functions: none of the types have any food & beverage or accommodation (72xxxx) or retail (44xxxx and 45xxxx) codes. This affects the mobility model because agents are attracted to food, beverage and retail locations. It also means we cannot show heatmaps or indicators related to accessibility to shopping, groceries, restaurants, nightlife etc. This can be solved by (i) adding new types for these functions, (ii) editing existing types so that some percentage is allocated to these NAICS codes, or by a combination of (i) and (ii).

Thanks Ronan,

You might want to try Column C, Sheet 'Updated land area - population'. I didn't change the value, just adding reference. Let me know if you have any doubt in the result. If we want to be close to the local standard, then these values might be useful. If we can be flexible, then I guess anything between these values and other standard will be okay. https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=2052915292
Types you mentioned are for spatial function. In urban management here, we don't have specific land use for F&B, accom., retail, etc.

@Hai-Hoang-88 and team Data @nqlong-vlab might want to discuss this. From my viewpoint, since we are working with mostly the current built which might or might not be the same as planning, so most of our data are from site visits to recording the function of the building, Google, statistic report, etc. Then, if we want to give percentage for types, then team Data might need to find ref. If we add new types, team data might need to proceed the shapefile again.

LAAP commented 3 years ago

Dear @TuCTruong , @dangbuingochan, @Hai-Hoang-88, and @nqlong-vlab ,

I think that matching the CS types with the exact land use it will be challenging (as @TuCTruong is commenting) so we will need to be "creative", so keeping that in mind, Please, let me try to help here:

Please, use the link as a reference, so, if it is unclear the real density for Vietnam, we can use the density from this table.
For the land use, I have try to make a match (even id it is not perfect, maybe can be a good starting point) by using the land uses from your Table at the link, in Sheet1: https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=2052915292

food & beverage or accommodation (72xxxx) = 32 | CCDVO | Existing market or/and 18 | CCDT | Complex land - hotel/ cultural/ recreational (NOTE: Maybe it is not a perfect fit, however, if you do not have other land use type, I think Markets can make the job
retail (44xxxx and 45xxxx) :
- 44xxxx. = 11 | CCDVO | Commercial/ service
- 45xxxx = 59 | CCDVO | Multi-function commercial

Please, let me know your thoughts

namkyodai commented 3 years ago

After running some analysis of scenarios and taking a closer look at the types, there appear to be a few issues to be solved:

Density: the densities of the land uses are too high (i.e. sqm_per_person is too low) and this is causing the residential and working population estimates to be much too high in all scenarios. eg. in the baseline scenario, the site population works out as 5 million and the working population of the site is 83 million. Note that for any type which has an associated NAICS code, the density will be used to calculate the number of employees, not the number of visitors. Here is an example reference for the US which may be helpful: https://www.engineeringtoolbox.com/number-persons-buildings-d_118.html

Missing functions: none of the types have any food & beverage or accommodation (72xxxx) or retail (44xxxx and 45xxxx) codes. This affects the mobility model because agents are attracted to food, beverage and retail locations. It also means we cannot show heatmaps or indicators related to accessibility to shopping, groceries, restaurants, nightlife etc. This can be solved by (i) adding new types for these functions, (ii) editing existing types so that some percentage is allocated to these NAICS codes, or by a combination of (i) and (ii).

Hi Ronan,

Would you kindly advise on the steps in order to address the item 2 as raised? using an example of existing data file.

From programing perspective, I wonder this assumption on percentage can be done in an automate fashion rather than manually editing the shape file. For example, within commercial or mixed-use land use categories (or types), if we want to include sub-types (food, retail), we might be able to assume randomly percentage in a range [e.g. min =1% to max of 5% for food], and [eg. min =3% to max = 10% for retail]. This is similar to performing sensitivity analysis as no one can know exact % of sub-types within main types.

Mathematically, we might use the following simple model to redistribute sub-types

Y(i) = SUM[X1(i)+X2(i)+...+Xn(i)] where Y(i) is the type value of row i in the database X1(i), X2(i),.....Xn(i) are the sub-types values

Since value of Y(i) is already given, the subtypes value can be further expressed as Xn(i)=pn(i)*Y(i) where p1, p2, pn is the percentage of subtype for row i in the database.

The constraint is

p1(i)+p2(i)+....+pn(i) = 1

so first we need to know total number of subtype n. and then we randomly generate value of p1, p2, ...., to p[n-1]. pn = 1 - sum[p1+p2+...+p[n-1]]

we can control the random of p1 to p[n-1] based on min and max value. This is subject to expert opinion. So instead of asking experts on exact value of p, we ask the min and max, the the rest will be automated with your program.

Please note that if using this assumption, each row of database will yield different distribution of sub-types. However, for simplifying the process, we can assume that a same percentage might be assigned to rows of the same types.

Pls let me know your though on this process as definitely we need you to guide us from your side when it comes to assigning the percentage to sub-type. Best that you can show on the screen in the next call the database structure and what need to be filled in [percentage] so we can further provide inputs for you to properly running the agent base simulations.

Cheers,

Nam - ISCM

doorleyr commented 3 years ago

@namkyodai the data input we need to adapt is just a simple csv which describes each Type. https://github.com/CityScope/CSL_HCMC/blob/main/Data/Table/cs_types_2.0.csv

We don't have a concept of sub-types- instead we describe each Type by the typical composition of land uses (LBCS) and economic activity (NAICS).

The idea of stochastically assigning percentages based on a range makes sense but would require rewriting of modules and would be incompatible with other Cityscope projects. The modules expect each type to be associated with fixed proportions of LBCS and NAICS.

In general the use of these types is a simplification and will not capture the real activities perfectly but it's a necessary abstraction in order to make the model understandable to non experts.

namkyodai commented 3 years ago

@doorleyr, my understanding of subtypes are similar to the composition you mentioned.

Agree with your point below "In general the use of these types is a simplification and will not capture the real activities perfectly but it's a necessary abstraction in order to make the model understandable to non experts."

btw, in order for you to complete that task (running the model on provided dataset), i suggest a quick call with you (assuming @TuCTruong will coordinate on this) so you can just open the excel file and explain clearly which columns and how many items in a composition under each type need to be filled in by VN team.

LAAP commented 3 years ago

Dear @TuCTruong , @dangbuingochan, @Hai-Hoang-88, and @nqlong-vlab ,

Do we have a first pass of the new update to the 14-18 types? As soon as you share it with me, I will translate it to NAICS and LBCS

Thank you very much in advance

Luis

TuCTruong commented 3 years ago

Dear @LAAP , @doorleyr Regarding the land types, the team is working on that and we will have it by mid-week. There won't be much change in the groups but there will be more details regarding the functions included inside. That will also be easier for us to fill in the density :D. Best, Tu

LAAP commented 3 years ago

Dear @TuCTruong , @dangbuingochan, @Hai-Hoang-88, and @nqlong-vlab , I have made a 1st pass to the excel that you are sharing to translate it to NAICS and LBCS: https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=225945613 This is a screenshoot:

LAAP commented 3 years ago

TuCTruong commented 3 years ago

Thanks @LAAP ,

@vietlq@ueh.edu.vn and I have put these into the Sheet Updated_Landtype in the same doc. https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=523130602

For the land types in Column B, we have walked through it in VN group. We think that we can work with it at this stage, in long term, it might need further adjustment. Land types here are ... complicated.
Column E, we explained a little bit on the VNmese activities happening in these land types.
Column C, D, F, G, we used the code and proportion you proposed, only change where we added more functions.

The screenshot here

TuCTruong commented 3 years ago

Dear @doorleyr ,

Since we weren't able to give enough density info. for all functions last time, that might cause you some misleading, hehe, we also added few more functions and revised them all now. So, @nngocquang.arc@gmail.com and I have made a 1st pass of the density for the Updated-Land type. We used the toolbox you provided and VN standard where applicable.

Since we are modifying the land type and function, so our Data team will need to work a little bit more on the constructed area. Once we have that information, we will let you know. Meanwhile, please have a look at the link for the updated density, sheet draft_landtype. Thankss! https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=225945613

LAAP commented 3 years ago

Hi @doorleyr, and @TuCTruong , @dangbuingochan, @Hai-Hoang-88, and @nqlong-vlab ,

I just want to double check if the types and format are ready and being used. The last update from Vietnam team is a bit confusing for me. Can we translate it to a CSV file like this?:

https://github.com/CityScope/CSL_HCMC/blob/main/Data/Table/cs_types.csv

doorleyr commented 3 years ago

@LAAP the spreadsheet you linked to above is not the right format. The right format is https://github.com/CityScope/CSL_HCMC/blob/main/Data/Table/cs_types_2.0.csv

We need the new types in this format before we can proceed.

TuCTruong commented 3 years ago

Dear @LAAP , @doorleyr ,

The final land type will be as in Sheet Final_landtype_density in this spreadsheet.

About the NAICS and LBCS code, since we are adding social housing, commercial housing and tenement in the Residential types, we could not find the code for this, can you help us in this matter.
For other land types apart from Residential (Type 1, 2), there won't be any change in total land area.

I guess the csv. file will be uploaded tomorrow with value of land area and constructed area.

https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit#gid=378855163

LAAP commented 3 years ago

Dear @TuCTruong ,

I recommend using for housing types that matches with the existing LBCS types for housing:

1 | Household activities (Residential activities)

LBCS = 1100 NAICS = null

2 | Transient living (Residential activities)

LBCS = 1200 NAICS = null

3 | Institutional living (Residential activities)

LBCS = 1300 NAICS = null

LeViet3910 commented 3 years ago

Dear @LAAP,

According to your previous comments, @TuCTruong and I proposed some changes listed below

Residential – lowrise will be divided into 4 LBCS codes which are: • 1100 - household for living use only; • 1110 – tenement; • 1210 – household that uses a part of its space for F&B (transient living, if we understand your suggestions correctly). For now, we will predefine that each house of this type spare 20% of its space for F&B. • 1220 - household that uses a part of ít space for retail. For now, we will predefine that each house of this type spare 20% of its space for Retail. The reason is that in Vietnam, it is very common if a house has direct access to main roads (or even smaller ones), then there is a high probability that they will use part of their house, usually the ground floor (or more if possible) to open a small restaurant or a shop. Please note that we want to highlight those 4 housing types which are quite close to reality in the Vietnamese urban context, in tables, and in storytelling.

In Residential – highrise, there will be 2 LBCS codes which are: • 1120 – Commercial housing (apartments for middle to high-income households) • 1130 – Social housing (apartments for low-income households)

And there are some minor changes to the Commerce, Services, and Mixed-Use codes to clarify a bit more and keep things consistent.

Is this more suitable with cityscope's workflow than the one proposed earlier?

https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit?pli=1#gid=378855163

TuCTruong commented 3 years ago

Dear @LAAP , @doorleyr ,

@LeViet3910 , @nngocquang.arc@gmail.com and I have updated the land type and area per person. We may work on this value for now. If there is any value that is needed to be changed, we will let you know.

Please check this link here https://github.com/CityScope/CSL_HCMC/blob/main/Data/GIS/landuse_code/cslhcmc_landtype_v2.csv Also in Sheet Final_landtype_density, docs file at https://docs.google.com/spreadsheets/d/19D1czN65F05iamg5kvdG9rJ46cZPvfrojBQ89XqUGrI/edit?pli=1#gid=225945613

Best, Tu

Hai-Hoang-88 commented 3 years ago

Please refer density column in #36 for Mix-use row: NAICS : (null, 445110, 551114) and NAICS_proportions :(0.7, 0.15, 0.15). I am not sure if we can run null * 0.7

CityScope / CSL_HCMC

Define the land use types using the CityScope Types system #9

we can control the random of p1 to p[n-1] based on min and max value. This is subject to expert opinion. So instead of asking experts on exact value of p, we ask the min and max, the the rest will be automated with your program.