Read Population from Game

[suhrmann]

Did you find a way to read the population details from the game - number of farmers, workers, ... and number of their residences? I was playing with your API and did only find the global total number of population:

def total_population(modules):
    return modules['TextSources'].TextSourceRoots.AreaPopulation

I try to integrate your project in our Anno 1800 Cacluator 👍 Thank you for this awesome project! :)

[Nick11]

I don't know, but I can give my best guess 😁

1. I think the GetEconomy and GetEconomyStatistic contains something like "workforce". There might be data about total number of farmers, missing farmers, unemployed farmers,... and so on.
2. The scenes module containes structures is alike to controllers for the different scenes/screens. You might find the overlay that displays the current area's (island) stats (top edge of the screen). Maybe IslandBar? These controllers are loaded with new data by user actions. So it only contains the data of the last visited island. This is a big limitation of the Python API, we still haven't really figured out how to deal with it. (Since we obviously want the data for ALL areas, and not just the one that is currently on screen)
3. You can always try the statistics screen modules['scenes'].StatisticScreen. But again, same issue as in 2., it loads its data, when you click on something in-game (e.g. select an island in the list). Maybe you could navigate it in-code, but that would popup the statistics menu and actually perform the navigation.

Last, please add your insight to the cheat sheet, so others can benefit from your work ☺️

PS: You might also want to try the in-game console; makes debugging faster in some cases (see updated README)

[suhrmann]

Thank you for your detailed answer. I queried the Python API with depth=1 recursion and dumped all to log to search through for certain keywords. The log and my adjusted code is here (feel free to include it in your repo if you want): https://gist.github.com/suhrmann/0535cff12417be515718132dbb54350c

When I found the needle in this Anno haystack I would create a pull-request with all my information.

[Nick11]

As far as I understand from your project, you're trying to read out population and production, right? My goal was to create statistics for the trade and charter routes, so the problems are somewhat similar.

My best idea so far, was to fetch the data, as you play the game: You visit island A, then the python script loads all the information for island A and keeps it in memory. As you move on to island B, it gets all data for island B and so on. That won't give you real-time data, but for the strategic planing, it might be good enough (I need X farms and Y factories for product Z, given a population of P). Three problems arise:

1. When to fetch the data from the game? So far you need to press a key to execute a script. However, the python API seems to have event-listeners, such as screenDidLoad, and you can add your own Items (buildings, products,...) to the assets.xml that trigger scripts. Or maybe you could run a timer to fetch at fixed intervals? The python tests scripts in the Data0.rda file (/script/benchmark/test1.py) does:

   someVar = Anno6.GameEvents.onSessionLoaded
   someVar.append(self.OnLevelLoaded)

   Or Spice it up Mod for examples of mods

2. The triggered python script needs to pass the data to your calculator.
3. You can't read the island names (and some other strings) in the python script. They use a string format, that is not readable through python (probably native to the graphics engine?)

So if you find solutions to these problems, that would be a huge step forward 😁

[suhrmann]

Update

The PR #2 found a way to read population from statistics and describes it quite well - but this is up to 4 minutes old. So far there is no current population found yet.

[Mathew3000]

Hey, was there any progress? I'm at this myself, but for now i can only read remaining workforce. I scanned the API with a depth of 6 so there is a huuge list of stuff to check :) Do you have any starting points?

[suhrmann]

Unfortunately: No. I was hoping that the stats-screen provides access to the values - but I could not find it.

Though from time to time I analysed the API further, there is no progress since last year.

[Mathew3000]

The "GetPopulationStatsitic" function returns an object that is 54 bytes long, but unfortunately I have no idea what to do with that. Also I cant get the historic data but I'll continue with testing. I found a calculator that works when you open the statistics panel. I try to check how that works, maybe I can quickly flash the infoscreen. That would not be ideal but better than nothing.

[Mathew3000]

I scanned the API with depth of 6 but nothing so far worked. At least for populatiob data :D i can kill selected objects but something as simple as Population seems undoable...

[Nick11]

As far as I understand, the Python API provides the data exactly as the GUI needs it. And that the data depends on the context (e.g. selected island) This makes it tricky for us to use it for mods...

My best guess would be to dig through the statistics screens and their corresponding API functions. Sometimes they only return data after you have actually opened the corresponding screen (as if there was some kind of "load me now with" API call that gets executed when a screen is selected/opened). I never quite figured out how the statistic screen's data is served by the API. It seemed to me, as if even the currently selected submenu was part of the APIs state (not only the currently shown/selected island/resource/...)

As mentioned before, we can use the API to open and select screens and submenus, but this is a rather nasty hack.

[suhrmann]

My understanding so far:

1. Anno does not contain the full Python 3.5.2 API. Some modules are (maybe on purpose) not available, like threading though subprocess is still in (I used this to run binary curl.exe [packaged in my mod] and communicate with my REST API)
2. The Python API interacts with the core-game over an C/Python interface. I dont think it was Cython pybind11 [Edit], but something simliar (but I cant remember its name)
3. From 2 follows: Many retrieved data are binary C packages that I could not figure out how to parse and get any usefull values from From my humble understanding, figuring out how to process these binary packages allows access to all internal values.

---

Two interesting projects I can mention:

• https://github.com/JulesRenz/Anno1800MemoryReader Abandoned, but "in the old days" working to read the values directly from memory - though needs lots of work for each Anno update
• https://github.com/egomeh/anno1800-scripting-api I had only a glimpse, but it looks promising

---

P.S. I invited you in my private "Playground-Repo" (Anno-1800-Python-API). Feel free to clone it or copy useful parts over into your repo.

[Mathew3000]

To 2: the Interface is pybind11 To 3: i guess that could Work. And i guess that could beva way to decode the population datatype