Citizens virtualization for large cities

[dymanoid]

In the vanilla game, the citizens get spawned randomly, regardless of time of the day. Furthermore, there are only a few movement reasons that cause a certain spawn: like e.g. illness (find hospital) or need for goods (go shopping). Entertainment and going to work are reasons that are throttled by the game depending on current number of spawned citizens and vehicles.

Since Real Time changes the going to work behavior, this movement reason gets a very high priority. The only throttled reason left is entertainment. However, the throttling only occurs while a citizen is at home. This is because whenever a citizen is not at home, there are time restrictions to be applied (e.g. everyone must leave parks until night time, the workers must leave workplaces according to their work shifts etc). The only unrestricted place to be is "at home".

For large cities (>65k), this introduces a new kind of problems. The spawned citizens throttling relies on the current number of spawned citizens. So, e.g. at 3 p.m., when almost all first shift workers are still at work (already despawned, because inside of buildings), the game allows to spawn lots of unemployed citizens (going e.g. to entertainment). At 6 p.m., when the first work shift ends, all the workers need to be spawned to get back home (or go shopping etc, according to their schedules). Since there is no throttling for citizens being at work, the number of spawned citizens quickly jumps to the maximum 65k. For very large cities, this even causes problems with citizens trying to leave their workplaces, because they cannot be spawned. So they have to wait at work for a "free slot".

This problem needs to be addressed somehow, but currently I have no ideas in which way.

[DaEgi01]

i guess this is where teleporting comes in in the original game behavior. the only options i see is a) increase the limit - have not checked if its technically possible or just too much work (best solution ^^ though performance will drop even further) b) distribute the leaving of working places to a bigger time range - no solution since it only delays the problem (sucks but works for certain city sizes 65k < x < ?) c) let them wait until a "citizen vessel" is available -> as is the case already (kinda sucks since it is basically a forced distribution with an open end, who knows, so cims will maybe never leave work ^^) d) let them teleport (as vanilla i guess, kinda sucks, but it is what it is) e) use another datastructure as an extension where the invisible cims are put into and where their position etc. is also calculated although they are not shown on the map. develop a mechanism to move cims from and to the extension depending on likelyhood of visibility to the player. -> (okeyisch, have not checked but unlimited trees mod does something like that i guess, only that trees are much easier than cims since their position is fixed and does not need to be calculated on update)

[Sipke82]

Perhaps reduce the allowance of unemployed workers to be spawned, so it's actually less busy during certain times of the day and especially at night.... In the middle of the night (3-4 AM), my 120k city still has around 55k cims spawned..

[dymanoid]

@DaEgi01, I see no difference between d) and e). Since the citizens are invisible, they don't walk, don't use public transport, don't drive. So there's no point in simulating their movement at all - just plain CPU load without any benefits.

@Sipke82, this doesn't help unfortunately. If there are 100k first shift workers that want to get home at 6 p.m., even completely disabed spawning of unemployed ones doesn't solve the problem.

The high number of citizen instances you observe at night can be explained: the dummy traffic, the shopping at night citizens, and the extended first shift workers are active at that time.

[dymanoid]

Possible solution - smart teleporting:

• calculate the teleporting probability according to current city population
• the starting point is always "at home"
• for every activity (work, shopping, entertainment), citizens either get spawned and move in their normal way, or teleport directly to the target
• a teleporting flag is applied to the citizens that were teleported
• when moving further according to their schedules, citizens act according to the teleporting flag (spawned ones can spawn further, teleporting ones can only be teleported)
• this continues until the citizen is back home; then, the teleporting flag is reset and will be recalculated according to the current situation

Advantages:

• apparently, the problem gets solved
• no ditched cars because of partly moving as spawned and partly getting teleported
• no skew towards workers (distributing the teleporting equally among employed/unemployed)

Disadvantages:

• more info needs to be saved in the save game (but maybe I can squeeze it in the current data structure)
• less realism
• probably less visible citizens e.g. in parks or hanging around by the buildings

[DaEgi01]

nah .. e) is much more complex in my mind than that. they could become visible if you change the camera position for example. covering distance still takes time, they still produce traffic and congestion. they are just phased out visually while still contributing to the simulation. its like in MMOs where players get "phased out" to other servers if it is too crowded. its not easy to implement properly though, i would not want to do it, but look for another solution that is for sure ^^

[dymanoid]

Okay, now I get it. This is impossible with the current game's implementation, since you'd need to run all the instances simulation using the whole citizens buffer. Movement only is not enough because of public transport, collisions, etc etc. Even if that'd be technically possible, you'd need a monster overclocked machine to be able to run a, say, 400k city - the simulation is single-threaded (okay, with exception of path finding).

[DaEgi01]

i wonder ... isn't smart teleporting what the game already does in a similar way? ok ... lets call game's default behaviour dump teleporting ;)

[dymanoid]

No, the vanilla game only teleports citizens in exceptional cases (e.g. when they are "confused", evacuating while instance limits reached, or in some invalid state like "at work" with no work building assigned - this occurs sometimes due to bugs in the game). Otherwise, the citizens just stay in the buildings until there's a next possibility to move.

[dymanoid]

Another idea: track the number of citizens that are not at home. (This number is not equal to the number of spawned citizens.) When the number of not-at-home citizens reaches the 65k threshold, no further citizens can leave their homes (regardless of their schedules).

Advantages:

• No teleporting
• Technically simple

Disadvantages:

• Large cities will function (mostly) as if they had only 65k citizens (visits count, income from shopping etc)
• Additional data still needs to be saved (but just one number)

[benzoll37]

This last solution, if it is simple to do, can be tested. I had observed this from time to time: unemployed climbed when there were jobs to spare, and I did not understand. Anyway, it is very timely and scarce.

I guess it also produces jams that are too long in time from 6pm

[dymanoid]

@benzoll37, both unemployment and jobs assignment have nothing to do with this issue. It's only about spawned citizens.

[Sipke82]

i like option a:) from egi the most.... but i guess it will be impossible to increase the cim/vehicle limit.

So basically, the game does not get more challenging past 65k cities, from a simulation game perspective.. in fact i think it will become less challenging, because the traffic (cims+vehicles) is will be more spread out over a larger area, and thus less congestion....

[DaEgi01]

that is exactly the case @Sipke82 i call it the empty streets syndrome ;D its funny how ppl show cities >300k and are proud that they have no traffic issues ;(

[liiir1985]

I just took a look at the current implementation of CitizenManager. It is possible to tweek the limit of 65536 maximal citizen instances, but quiet hacky.( It used a type named Array16 to store all possible instances, which could only take 16bit indices) Is there any other problem of setting the limit to a higher value, despite the CPU and GPU cost? Or, is it a solution for this issue if we could make it possible for the players to set this limit value through setting menu?

ps. Besides the limit of spawned citizen instances, there's also a limit for citizens living in the city, which is 524k. Don't think this would make trouble though

[Sipke82]

Perhaps equally important the vehicle limit and the parked cars limit.

[liiir1985]

@Sipke82 Technically, it's the same as Citizen instance limit, and can be modified. Just... loads of methods need to be patched.

It'll be easy if we just modify the dll file, but it's bad for realeasing it as mod in the workshop

PS. Performance can be improved if we move the simulationStep function into multiple threads. Current implementation already uses a dedicated thread for this job, basic thread safty is already been made, so it might be not that difficult to move it onto multiple threads

[dymanoid]

@liiir1985, I don't think it's even possible to increase the instances limits. Currently, there are many methods that do use hard-coded upper bounds while processing the arrays. How do you want to solve this? Please don't suggest the Harmony's "transpiler magic". Messing around with the IL code during run-time is the worst thing one could imagine. I know, some modders do use thins technique. But they maybe don't realize the danger of that approach.

Citizens limit is 1m by the way.

One more word about the multi-threading: the game doesn't support this, sorry. The current implementation has no built-in parallelism support, with a single exception for the path-finding thread.

[liiir1985]

@dymanoid It's actually a constant value which got compiled into IL code. yes, the only way to do it is to messing around the il code, but in a more clever manner. We could use code analysis library like Mono.Cecil(I had a lot of experience with such stuff, also at work. see my github repository- ILRuntime, which is a IL interpreter execution engine) to generate the patch automatically, instead of making it mannually, this would make it more reliable and consistent during future game updates.

What makes the whole thing problematic is the definition of m_instances as an Array16, and all related indices used ushort, this may make the whole thing impossible through runtime patching

The game doesn't support multi-threading, yes. But the calculation of the AI and all the simulation stuff(Logic is seprated from rendering) is already on another thread, rather than main thread. And synchronization lock also exists(See SimulationManager, which will start a new Thread for all the simulation stuff), So basically we can modify the SimulationStep function of SimulationManager, to make the for loop multithreaded, or at least, to make the citzen/viehcle simulation multi threaded. The current implementation is very friendly to the new ECS system introduced in Unity 2018, really hope the developer could make use of it, so simulating millions of Citizens would become reasonable

[liiir1985]

OK, I took a deeper look at the current implementation, 16bit instanceID is everywhere, it's impossible to patch all of these methods including parameter definition/ field definition. We can basically forget about increasing the instance limits :(

[dymanoid]

@liiir1985, that is exactly what I was talking about.

[liiir1985]

@dymanoid How about share riding? Will it solve the problem in some degrees? If the Citizen cannot be spawned, then call another citizen which has a viehcle to pick him up, then they move together. Maybe it's somehow more realistic than just teleporting the citizen directly

[dymanoid]

This would need a comprehensive search mechanism which will drastically drop the simulation performance. The citizens will still need to be spawned to get into car and later to move from the car to the building. If we don't spawn them and let them ride "virtually", then why bothering at all? It's still like teleporting them directly to the target.

[liiir1985]

Let them ride virtually will still produce traffic to the area where this citizen lives, so maybe it will make the streets look more busy?

[dymanoid]

Traffic flow is calculated "by-car", not "by-citizen".

[liiir1985]

I know it's calculated by car. What I mean is, maybe it won't be any car driving by otherwise, and now some one needs to pick him up, so it looks like a "spawned" car is driving by.

[dymanoid]

I don't think it's a good solution in general. It won't solve the issue actually. For very large cities, every passenger car would need to pick up 3 passengers each time. Just imagine the traffic nightmare and the simulation performance drop in that case.

The feature itself is actually not bad. It adds some realism. But I see it rather in the "mass transit" domain - how does a particular citizen want to travel: by car, on foot, by public transport, or share a ride. So this is out of scope for Real Time.

[willbattel]

@dymanoid I like your smart teleporting idea. It's unfortunate that C:S has these limitations but I think that's probably the best way to go about it- at least for the time being. Maybe someday CO will improve on this given the improvements of the hardware of the average user since the game's launch- or if not, then perhaps in their successor to C:S they've mentioned before that'd eventually solve many of the technical limitations.

[jguy1987]

What if you take a "time zone" approach, especially for larger cities....

Imagine the city tile layout: A B C A B C A B C

Everyone can still go to work at ~6am for first shift but citizens in row A go to work at 5am, citizens in row B go to work at 6am and citizens in row C go to work at 7am. Then, they get off 8 hours later.

Example: At my job we all work "first shift" in the typical sense, but half of us come in at 6 and leave at 3 and half of us come in at 8 and leave at 5.

Any possible way to assign a "shift" randomly to a citizen? I know many jobs which require odd working hours...for example my dad goes to work at 2am for a 3am start and gets off at 1-2pm and gets home at 2-3pm. Even if you just randomly assign a work start time, specifically for office/factory jobs. Some factories are 24h/7d operations. assign a start time to a citizen, say 5am. They leave at 4:30am for their job and get off of work 8-9 hours later, +/- overtime chances.

[originalfoo]

For the time zone idea, would that be based on where cim lives, or the destination? Doesn't existing variance already do that sort of thing anyway (config: on-time ratio, and max overtime hours)?

On similar theme to time zone, maybe as city grows the distribution of shifts could become more equalised?

tl;dr: As city grows, increasingly disregard shift ratios and some other config settings in order to more evenly spread cims spawn times.

The config set by player would define how smaller cities work, but beyond a certain point RT could start equalising the ratios between shifts (weighted to so that evening shift grows faster than night).

They'd never become fully equalised, as you still want less people in evening, and even less at night, to create distinct periods of the day; but it would give player general sense that the city is getting more busy as they'd notice more people on streets in evening/night. It gives a way to make the city feel like it's still growing once you're regularly over 65k limit during day time.

Does dummy traffic count towards the 65k limit? If not maybe that could be increased to make the roads look busier?

EDIT: Maybe the on-time ratio and overtime hours config settings could also be less adhered to as city grows, in order to increasingly spread the load as the 65k limit is reached?

• Small city: Does what you tell it to
• Big city: You're still the Mayor, but the lunatics have taken over the asylum

[originalfoo]

Maybe as population grows, increase +/- time randomisation factor of schedules, applied to all cims.

[TMaekler]

Quite some time ago that I programmed... so if my idea is complete bollux, just ignore. If it would be possible to start a separate thread next to the official simuation thread which takes over the calculations for the citizens, maybe it would be possible to come around the number limitations of current C:S... .

[dymanoid]

"Time zone" approach won't do, because it is not deterministic. We need a deterministic way to limit the number of "instantiated" citizens at a time.

@TMaekler, it's not about threading, the limitations are hard-coded in the game and cannot be overridden by user mods.

[cptalpdeniz]

One of the problem that needs to be addressed is which type of "cims" you want to get prioritized for instance slot. Basically, If you want to prioritize the tourist or other cims going outside for the day, you have to prioritize them for the slot and when its time to leave the work, workers have to wait. Or if you want to prioritize the workers, you can limit the instance slots during daytime or closer to workers leaving work to make room for them. However, this is just KISS approach to the problem. Which is not wanted.

Since we cannot alter how the game works and its limits, we can only maximize how we are working with those limits. For instance, in 100K workers example given by @dymanoid, even if you have all 65K instance slots available, that is still not enough slots and you would end up with 35K workers whom has to wait till there's slot available.

If you want to implement something far deeper, which is what is wanted, I'd like to add few things to suggested ideas. They can be drastic changes, but I'm just trying to provide an idea to discuss it further and/or open up for more discussion.

Change work/worker behavior As of now, with how the mod works is to make workers go to work and leave the work at a specific time (either default value or user selected). Even though this steps up the game in terms of realism from vanilla version, it's unrealistic. Not all the jobs are, for example, from 8 AM to 5 PM. By changing work behavior, you eliminate a chunk of workers from that rush hour time frames. This is kind of what the mod does, but in a different way. Few examples for this are;

• Add randomness to the shift behavior. I understand that shift behavior is to make the game more realistic, however, not every job has to start exactly at that time. This decreases amount of people going to (or at least trying to) work at the same time. For instance, instead of shift starting at "work start time", it starts and finishes 1 hour later.
• Making the leisure areas, jobs, have shifts, more on weekends, later hours and at night instead of daytime. For example, there's no need for most of the leisure buildings to be open at 8AM, or Monday morning. You can add probability selector, what percentage of them open at work start time (first shift), just like in shift behavior, to make it more realistic by adding some randomness.
• Improve building's that are open 24/7, shift logic. Decrease the first shift percentage
• Change Garbage processing to after 8PM to early morning to match real time.
• Remove lunch break completely

Smart Teleporting Smart teleporting can be implemented to the game to help with cims stuck where they are because there is no free slot left. However, it is important to find a good ratio between how many cims are being teleported and how many of them are actually moving.

• Since there is a instance limit, this also creates a traffic limit. Technically, every city created in Cities Skylines has a traffic limit. If you hit instance limit or close to it during those rush hour times, you are also at traffic limit, which means the traffic cannot get worse since a cim with a car can't get spawned due to instance limit.
• Instead of adding teleport flag to the cim, teleportation can be implemented in a dynamic fashion. For instance, it can depend on time of the day, how many cims are in line to spawn, what is the distribution of instances (how many are going to work, how many are out of shopping, how many are kids going to/leaving school, etc.) are and how game is treating these instances. In this way, you eliminate the probability of less people in parks and keeping that "rush hour" feeling.

Adding Weather/Environment to the Simulation This is kinda fishy but small things like these help with both realism and performance. This does not have as big impact as other suggestions. Depending on the theme, weather (if its raining, cloudy, foggy, etc.), the movement of cims will change. Basically this would decrease number of cims going outside or having movements other than mandatory ones. If the weather is not nice,

• Cims will stay at home during weekend
• Cims won't go to lunch breaks
• Only small amount of tourists will actually do touristic activities during daytime.

All of the suggestions above should also differ in execution depending on the population of the city. As city grows, RT should increasingly disregard shift ratios and some other config settings. As @aubergine10 said it before, the bigger the city gets, the more Real Time should take control and start taking these measures to make it so that the performance doesn't get affected as much as it is right now.

[originalfoo]

I'm not sure what that last comment is about, it's sort of just reiterating what's already been implemented or discussed elsewhere. For example, there are already options to add variance to shift times, variance to the start ('on time') and end ('overtime') of shifts; people head indoors when it rains, and so on.

[dymanoid]

I just wanted to write the same comment as @aubergine10. Thanks @cptalpdeniz for your effort, but in fact you don't propose any solution. The whole probability-related stuff is non-deterministic and won't solve the problem. The teleporting cannot be dynamic because there will be ditched cars everywhere. And so on. I didn't find any suggestion in your post that would help solve the issue, unfortunately.

[TMaekler]

Do I see it correct that the main issue with reaching the limit are people who leave school/work to go home and vice versa? If so, maybe it is possible to create a new type of (virtual?) citizen for only those people who do that and leave all others (like go shopping, do tourism, etc.) to the normal simulation?

[dymanoid]

This would eliminate the commuter traffic completely and make the game totally boring. But the problem is not necessarily related to those citizens. In large cities, there is always a probability to hit the citizen instances limit because of citizens' schedules that Real Time implements. Please read the first post. The vanilla game can effectively throttle the number of spawned citizens because there are only few reasons for a citizen to be necessarily spawned. Real Time's schedules change that so there are many reasons for necessary spawning.

[TMaekler]

I understand. My line of reasoning was to create a new type of citizen (which basically acts like the normal citizen), but can additionally be spawned alongside normal citizens (they should for example fill up public transport slots, drive around in cars etc.). Thereby circumventing the ingame limits of 65k spawned citizens.

[originalfoo]

Add a scaling factor to the cims and vehicles? Once limit is reached, they start getting physically bigger...

[dostillevi]

As I see it, there are two main problems caused by the citizen and vehicle limits:

1. Citizens get stuck and can't get to where they need to go while waiting for a slot to be open.
2. The games becomes unrealistic when most solutions to #1 are applied, namely teleporting.

The only solution I see to #1 involves teleporting. The cap on citizens has already been discussed and no solution is available. However for appearances sake, could groups of citizens be made that are treated by the game as one citizen but hold stats values for several, or even are simply visually more than one citizen?

Alternately, not all citizens the player sees need to be following the simulation. As long as what they're doing seems realistic, they can just be fancy props. The key is to have enough citizens within the player view doing simulation things that those standing around, walking around the block forever, etc aren't so obvious. This might require writing some simulation code outside the main simulation or creating a new class of object, and I'm not sure how feasible that would be.

So that leaves #2, if citizens are teleporting, how can the traffic simulation be preserved? As I understand it, the traffic simulation is done on a "per vehicle" basis. What if vehicles could be grouped together with multiple vehicles temporarily being treated as one when they're traveling together in groups in the same lane? Two vehicle groups could act like an articulated bus as long as they're both headed in the same direction.

[timapelov]

Support of the above idea - I suggest several cims, moving from one common point to another common point, to unite in groups and simulate these groups. Then have us will 65K groups, in which can be 2-3-4-5 people. And for statistics (transport, entertainment, job) to consider them separately, it's easy, because the place and transport for the group is one. When the groups get to the point, you can combine them into a new group or calculate as separate cims. In real life-many colleagues go somewhere together after work, family members often move in groups (joint purchases in the store, joint vacation). Indeed, in C:S all the cims alone and the car alone for one cims. Groups can be visualized as multiple people walking side by side, there will be one model for one group and one travel simulation . What do you think @dymanoid ?

Additional question. Do a lot of computer resources spent on the calculation of traffic without cim (trucks, transport, municipal services). If so, is it possible to disable the simulation of urban services (let's say garbage) manually? In order to have more computer resources for cims.

• sorry for the translator*

[dymanoid]

I will be joining a Modders Meetup at Colossal Order in June 2019. I'll discuss this topic with the game developers directly. Maybe they will have an idea about how to solve this.

[RenaKunisaki]

The solution is for them to change all those 16-bit indices to 32-bit. I hope you'll bring that up. Maybe it was a good way to reduce memory/CPU usage, but 4 years later our computers are a little more powerful.

As for temporary workarounds, it seems like one solution would be to impose limits per type of citizen. Eg a maximum of 20000 unemployed can be spawned at once, 20000 employed, 20000 tourists. Then, those limits can gradually change with the time of day; eg from 3am to 8am the limit for Employed gradually rises, and the limit for others gradually drops, until at 8am the Employed limit is maybe 50000. The key is the limits change gradually.

At all times the limits should add up to around 60000, which leaves you with a buffer of 5535, that allows citizens to get to places when they're over the limit. (eg if 20000 unemployed are currently spawned, and that limit starts to drop, then some unemployed who are near home or near their destination will hurry up - maybe even start physically moving faster - to try to bring the number back under limit quickly.)

The limits could also be adjusted by region/district; eg if a district has a lot of employed people, then it will generally have higher limits for number of employed people. (If a district is at its limit, people won't decide to go there, but people who are already en route might still go, prompting someone else to leave; or, they might switch to a different destination.)

Eventually, if you run up against the hard 65535 limit, you could fall back to despawning/teleporting both the citizen and their car, or forcing them to wait. It's unrealistic, but shouldn't be too noticeable. If you look closely you already see other unrealistic behaviour at that scale (cars driving through eachother, people pulling bicycles out of their pockets...); at some point you have to accept that a game won't be 100% realistic and you're going to see such quirks occasionally, especially when you're pushing the limits of its design.

Another thought: do people count toward the spawn limit while in vehicles? Maybe you can despawn people and just let empty vehicles drive around? Draw some models inside to look like people but don't actually have a person inside?

Anyway, the numbers are just off the top of my head and can be tweaked. Also by "employed" I mean "currently going to/from work"; ie not referring to whether they actually have a job, just whether they're commuting for it.

[originalfoo]

Maybe it was a good way to reduce memory/CPU usage, but 4 years later our computers are a little more powerful.

There are still vast numbers of people on potato computers, particularly laptop users.

[willbattel]

Maybe it was a good way to reduce memory/CPU usage, but 4 years later our computers are a little more powerful.

There are still vast numbers of people on potato computers, particularly laptop users.

This is an argument as old as time itself. There will always be personal computers considered "potato", but the standards for which they qualify does increase over time. The average computer today is certainly more powerful than the average 4 years ago- however, as you said, there are still a significant enough number of users on insufficiently powerful devices that hold back more compute/memory-intensive game development.

Additionally, increasing memory address size isn't a trivial change. At that point, CO might be more inclined to just make C:S 2- but that isn't likely to happen until their DLC development starts seeing diminishing returns.

Some day.

[RenaKunisaki]

I got the impression from previous comments that it would be a pretty simple change, but one that has to be done at source-code level, not something that a mod can do. Is that not the case?

[willbattel]

I got the impression from previous comments that it would be a pretty simple change, but one that has to be done at source-code level, not something that a mod can do. Is that not the case?

Maybe; It's impossible to say without seeing their code base. Usually it would be a major overhaul but that's not always the case. They may have developed it such that it'd be an easy change.

[originalfoo]

Even if it is a trivial change, like updating a single constant and then recompiling, it doesn't necessarily mean its a feasible change - there will be so much stuff linked to that value that, should it be increased, you're effectively multiplying the computations taking place across large swathes of the codebase.

Case in point, in a recent pull request review on the TMPE mod, this cropped up:

Assuming every source lane has 2 target lanes on average: n * 2 extra computations needed (max. 262144 * 2 = 524288) which gives us a total maximum of 2*n*n + n*2 = 137,439,477,760 required calculations.

That's the outcome of an increase from 1 to 2. Simple change, big ramifications.

[originalfoo]

Doesn't solve the core issue, but Quboid's Cargo Hold Fix mod has option to turn off dummy traffic, which might free up some vehicle slots on mid-sized cities.

[supermerill]

Do you know the number of car & pedestrian instance in an average (good) city? Maybe teleporting pedestrians to their next hop (with a timer, a bit like when they are at work, but instead they are "waiting at the station") can free up some agent slots. But it may be a big change because now public transports have to check the waiting agents & the new "ghost queue".

Anyway, it's pretty useless if pedestrians count < car count.