Ship Shortages

[Xylephony]

I completed the test we previously discussed about supplies needed for naval construction and what could be the bottlenecks on shipbuilding. These are my observations:

1. Coal within reasonable range until ~1880 when it starts to go into extreme shortage (3k+), presumably in part due to demand from monitors. Concrete and machine parts both went into an approximate 1k unit shortage around the same time, so shortage likely coincides with naval yard, rail and fort construction coalescing around the Scramble for Africa (though I honestly don't know where the machine part shortage is coming from). Interestingly enough, Iron becomes overproduced at around the same time and then jumps back to in-demand in the 1890s (and in my test eventually hit a 2k shortage by 1910). Concrete and machine part shortages resolve themselves within about 15 years, but coal remains short 3k+ units until at least 1910. I ended my test in 1915 when China became civilized and didn't see levels after that.

2. Russia in my test game did not have either High-Pressure Steam Engine or Interchangeable Parts by 1905. Huge impact on world wood demand; wood is in perrenial shortage, usually not by a large amount (~500 to 1000 units) but I suspect that there is hidden further demand. A good half of all naval bases in 1905 are demanding lumber inputs despite lumber being ostensibly only 200 units short; even the UK doesn't have sufficient supply. I suspect that wood demand would actually be much higher if there was a sufficient amount of wood on the market to actually keep open more lumber factories, and that more lumber factories would be sustained by more naval yard construction if the bottleneck for lumber didn't exist.

3. Telephones and fuel have a major spike in demand in the 1890s but settle into stable values by the 1900s.

4. Electric gear shortages are almost fatally short. Demand 200-1000 in excess of supply from around 1885 to 1910.

Out of all of this, I think the coal and especially wood shortages are the most important. Coal shortage at the level it's at impacts the construction of a variety of facilities and general economic stability, while the wood shortages lead to naval bases not upgrading and thus nations being stuck with few facilities that are equipped to construct battleships and cruisers. More critically, it also limits colonial range and naval points overall, so nations can support fewer ships in total. In several of my recent games I've seen Britain release parts of Canada, as well as France release parts of Africa after the scramble because they lacked colonial points, presumably in part from naval yards. Wood probably does not need a huge tweak, but it does need at least a little bit of help I think to ensure it's not a naval bottleneck.

In the long-term though I think that electric gears would eventually become the bigger problem. So many states are going to have full and profitable industry by the time electric gears are unlocked that it can take decades for them to finally get going, and in that time rubber can go out of demand and RGOs can shift off of rubber, which can then lead to bottlenecks on electric gear production. I don't know if it's actually possible to create a factory with a decision, but a decision to "Subsidize Electronics Facilities" which creates a new electric gear factory to try to forcibly start production might not be a bad idea, since they're such a critical input good. Maybe usable once per nation and only by GPs or SPs, and also only possible in nations with State Capitalism or Planned Economy.

[Xylephony]

Saw the changes and I will try to test soon, though for what it's worth just anecdotally I think the changes to square-set timbering probably aren't needed. There were no major shortages until the Scramble and even a little bit of overproduction for brief periods, so I think all the coal production boosts should be 1870 or later.

Also I am doing a big project right now at work and have basically a full month of busy period ahead of me so it might be a while before I can actually test comprehensively.

[ujtcelvn]

possible to create a factory with a decision

This be possible, there is a command but it can only build a factory in the capital state.

The new local steamers modifier should help increase industrial rgo production (coal, iron, wood) while reducing overproduction of basic farm goods (due to local_reapers_und_soil_enrichment being disabled)

[ujtcelvn]

This SHOULD fix most of the issues with the economy. Even better, the economy doesn't seem to implode past 1945 anymore. Electric gears are still often in shortage, but there's not much we can do about that.

[Xylephony]

Yeah so I am constantly sorry whenever I need to hit you with this, but that broke things almost harder than I've ever seen them break before, and I've done two tests so I know it's not a one-off. Not a single nation in the primary test has a million pounds reserve, not a single one, and nobody's even built canals so it's not like they had the cash but the money was just invested. The US only has 15,000 pounds total, and they're among the best. Almost every nation is in fatal debt, rebels are everywhere and POPs are mutinous. France lost Algeria to rebels and lost Senegal, no shit, to Wolof beating them in a war. Industrialization seems fatally crippled.

I will say, demand looks like electric sex and has this entire test. Demand has virtually always ridden maybe about 100 units above supply for everything but grain/livestock/fish/fruit, which is I think the ideal. But that doesn't help much if the economy is blowing up.

I don't even know what was changed that could make something like this possible. The test right before these economic changes were completely fine, so I have to assume it was the attempt to curb early-game production.

[ujtcelvn]

I was too excited by the supply and demand numbers... Yeah after checking the saves for my last test turns out everyone WAS broke before the 1880s. Should I just give up and make opium print money again?

[Xylephony]

I don't know, the test immediately before the supply/demand changes you made three days ago seemed fine, as with the couple that were before that, aside from the later coal demand spike and sort of constant low-level wood issues. I know your concern is probably opium demand more than cash availability, but I worry that making opium a money printer good will inject so much more cash into the world economy that it will wildly change supply and demand again.

[ujtcelvn]

I wonder if it might be the 1% factory throughput the steamers modifier is giving that throws off the economy, because it doesn't make sense that tweaking rgo efficiency would cause everyone's budget to just die like that...

[Xylephony]

I don't get it either, but I don't see how 1% throughput could wreck the whole economy either.

[ujtcelvn]

In my last test, by 1860: UK, France, Prussia were in very deep debt with $0 in the treasury. The US and the Netherlands had $10k, and Austria and Russia only had a few thousand. That's a bad sign, right?

but I worry that making opium a money printer good will inject so much more cash into the world economy that it will wildly change supply and demand again.

Well, it seemed to work in the development branch of HPM, so there shouldn't be TOO big of a problem... Maybe the real reason why everyone is broke is simply from a shortage in money supply.

[Xylephony]

Yeah, I have another test right now where everybody's deeply in debt and the economy is crashing because of it as well. I just don't understand why. Just a few tests before this everything was fine. What, 1% throughput does all this?

The only thing I can think of is that it's maybe related to the way that the new local steamers modifier spreads, and how common it is for nations to qualify for it. As far as I remember, the last stable test I did was immediately before that commit, and then everything after had these economy woes.

We can try turning opium into a money printer and seeing what it does to the world economy, but I think it's not going to solve the debt problem for nations that don't have opium RGOs.

[ujtcelvn]

It can't be because of the steamers modifier, I reverted that change 5 days ago...

[Xylephony]

Did you? I thought I saw it ingame, but I might have misread it, it was just an event that added a production modifier that I quickly clicked off of. Well hell, I don't know what it could possibly be then. I can't see anything in the leftover changes which would explain why nations which had millions in the bank a week ago are broke now. I guess let's try the opium change and see what it does? I can't see it working, but I'll test it just to see. If it doesn't, it might be best to revert every economy change made to the commit immediately prior to when I opened this ticket.

[ujtcelvn]

Yeah there's a local_reapers_und_soil_enrichment modifier early on, which was replaced by the steamers modifier. The opium changes make the UK with a million in the treasury, Netherlands with $200k, USA, Prussia and Austria with like $20k, and only Russia and France in debt. I guess it makes it not as bad as before, but is it acceptable?

it might be best to revert every economy change made to the commit immediately prior to when I opened this ticket.

PAIN Maybe HPM really is balanced around a massive coal shortage...

[Xylephony]

I guess it makes it not as bad as before, but is it acceptable?

I wouldn't say so. Like you said, it's not as bad, but it's not nearly enough reserve for most nations to function normally. Like, before nations all had like a million or so in treasury minimum, and that let them actually afford protracted wars. If they only have a couple thousand, or even just a couple hundred thousand, that's not going to keep the AI constantly underfunding itself. And then you also have to consider that, at least in my tests, POPs were also seeming to have needs issues and were getting extremely militant over it. Has to be something to do with demand, but I don't know what specifically. Still, it made nations way more unstable than they should be.

[ujtcelvn]

Wait a minute, all the production increases I added now are only unlocked by Drilling and Blasting, a 1870 tech, so it's impossible for that to be the cause of nations being bankrupt in 1860.

[Xylephony]

Were there any changes at all prior to this ticket opening that altered the economy? I might've missed a commit or two between the stable test that I had before this and then the issues we're having now. If not, I have no idea where the problem crept in, but with the volume of tests I've done it's clear we had economic stability and then lost it somewhere.

[ujtcelvn]

Nope, nothing between 1/10 and now would have affected the economy prior to 1870 that wasn't quickly reverted.

[Xylephony]

So based on the commits on 1/16, that's the version I tested and the one that had a stable economy. The only change since then is the 1870 coal, everything else was reverted and there's no POP demand changes or anything that snuck back in that could be causing issues?

[ujtcelvn]

1/16: revert pop demand to HPM levels drought and productive year now exclusive --- things worked above this line --- 1/29: increase coal production, timber-heavy countries research more power techs 1/31: removed pre-1870 coal increase from 1/29 update removed local_reapers_und_soil_enrichment, add local_steamers add electric gear factory decision Revolution of the Reforms 2/2: tweak post 1930 inventions 2/4: Fix Qing infamy bug 2/5-6: remove local_steamers, add back local_reapers_und_soil_enrichment 2/9: HPM P&EP 4.0 2/10: Opium prints money

I don't SEE anything that would touch pop demand or treasuries ... the only pre-1870 change to make timber-heavy countries more likely to research more power techs.

[Xylephony]

Maybe that's causing overproduction of other basic goods? A lot of nations have timber after all, it could be destabilizing basic goods demand, and that would explain the POP militancy as well if a lot of basic farm RGOs were losing profitability. Let's try to revert that and see how it looks.

[ujtcelvn]

I reverted everything I could think of and the results still look similar to the last test.

[Xylephony]

What the fuck. I'm not as familiar with github as I could be, is there a way to download the version from 1/16 exactly as it was and test that?

[ujtcelvn]

I made a Jan-16-2024 branch for you, you can download that.

[Xylephony]

Thanks, testing now.

[Xylephony]

Everybody's broke. I don't get it. I legitimately don't understand, I feel like I'm going schizo. I went back and double-checked my comments in #118, and they confirm that the most recent test I did where everything was fixed was on the commit from 1/16. I tested it more than once at the time to make sure. Now in this test everybody is back to being a completely indebted shithole. I mean, in the 1/16 tests I did I even said they might've been TOO wealthy, and they were! Each GP or GP-adjacent nation had millions of pounds banked and almost no debt.

Looking further back in #118, the very first test where debt finally got resolved was Jan 7th, and the most stable test prior to 1/16 was from commits on 1/10. Can we ditch the 1/16 branch and replace it with the 1/10 branch, and I'll test that?

[ujtcelvn]

New branch created.

[Xylephony]

...1/10 doesn't work either. Same issues. The only thing I can think of is that maybe, maybe there's a delayed recovery, and that's what we noticed before. Like nations research tax techs and get better industrial techs and maybe by the 1870s or 1880s they pay off their debts and start to receive huge incomes, and I didn't notice that other nations were starving before that point and only saw them after they fixed their incomes. But that doesn't seem possible either, because we'd even seen before in prior tests that nations with huge monetary reserves seemingly won't pay off their loans, and in the tests I did with stable economies no nations had any debts.

I'm not missing something obvious, right? There's nothing that would be saved in the EEM documents folder from earlier mod versions which might be interfering? I was under the impression only map changes could potentially cause issues.

[ujtcelvn]

F It might just be that, because when I checked in 1900 previously everything seems fine and dandy. I don't think anything in the documents folder could interfere with anything... yeah as you said I thought only map (and flag) caches could problems.

[Xylephony]

Well now I just did a very interesting knock-on test. I booted up my local HPM copy and tested, and what do you know? 1850 and everybody's broke just like in EEM.

Based off of that, I think I can guess what's happening. Nations want to build infrastructure - forts, naval bases, railroads. In the early game they have a tremendous amount of provinces in which to do this but they lack lategame financial resources, so nations spend a huge amount of their income queuing these constructions. They go into debt from wars and temporary economic downturns, but my guess is that they probably aren't actually in the red in treasury terms, at least not constantly. They are making income, but spending a large amount of it on infrastructure, and with the debt that they gain from other costs it makes them look like they're economically nonviable when they aren't. I'd bet that if one continued through to 1880/90 with the current EEM it will go back to looking like nations are wealthy again, because they'd have less cost outflows and way higher incomes. I will test just that when I get back later tonight.

[Xylephony]

So I tested both the most recent HPM dev branch and the current EEM branch. I realize the current EEM branch rolled back almost all the economic changes so it doesn't help us identify whether any of those caused real issues, but I can at least confirm they're virtually identical: both mods have nations that barely ride the line on solvency until their incomes catch up to their expenditure rate & they no longer need to invest in infrastructure. The "turning point" in EEM's current version appears to begin in about 1870 and end in about 1890, the latter point being when almost all GPs & GP-adjacents have at least 500k in reserve. In HPM it seems to be slightly tighter, but that could easily be down to the test itself. Functionally they seem about identical.

So on the one hand this feels like a giant waste of time I've put us both through by not catching this earlier, but on the other I think this was helpful in establishing a baseline to compare to. What we need to keep an eye on, seemingly, is goods supply and industrialization vis-a-vis the treasury recovery rate. The ideal is a setup where the supply of goods is just short of demand while industrialization is stable and not top-heavy on the GPs, while also having a global economy which allows for GPs to begin stockpiling reserve resources no later than 1880. So we know for future tests we need to go at least that long before we draw any conclusions about economic stability.

However, finding all this out also concerns me. I wish HPM were still around to bounce some questions off of, but realizing how little reserves the AI keeps until literally 40 years in makes it seem like early-game nations are almost always teetering on the brink of insolvency. They aren't literally, but without asset reserves the moment they get themselves into a war they take on major debt and that major debt can easily lead them to a debt spiral if they're not a top-level producer. Moreover, it seems likely to me that the overwhelming majority of nations will never have enough assets to fully invest in infrastructure as the AI wants to, and that most nations on the map will rarely have any cash surplus lying around for contingency even into the endgame; only the wealthiest tags will. Hell, even GPs run some risks here. For instance, most of my EEM tests this past month and a half have had Italy become a complete, and I mean complete shithole which is in debt for the entire game and never gets out of it. I'm not saying that any of this is an EEM problem because HPM has all this too, what I'm saying is I think this is a Victoria 2 problem in general, and a big one. Nations need reserves to be able to fight wars effectively, but also for army and naval expansion, which to my mind helps explain why the midgame naval switch over to ironclads/monitors is so anemic: the funds aren't there yet.

Now I kind of see why Naselus did what he did with PDM. It wasn't the right solution, but looking at this I can see why thinking multiplying the money supply by 10 and washing your hands of it might have looked like a simple solution to a complex problem. I don't think there is a solution, really, because either you trivialize the economy like PDM did or you have this issue of limited reserves, because you can't teach the AI to hold more funds back. But bottom-line I think we at least need to be absolutely positive that, whatever economic changes are made from here on out, nations continue to be able to industrialize normally and reserves start appearing around that 1880 mark.

[ujtcelvn]

Oh thank goodness. I was about to ask if maybe we hadn't played HPM for so long we forgot how broke they were in HPM.

The ideal is a setup where the supply of goods is just short of demand while industrialization is stable and not top-heavy on the GPs

I really want to make the steamers modifier work...

debt

Yeah there doesn't seem to be much of a way to encourage the AI to get out of a debt spiral without a default. Could non-top-tier countries not being able to afford much be intended? That might be the only way the investing-in-railways-all-over-South-America meme is even a thing, if everyone could build what they wanted all the time foreign investment wouldn't be as much of a thing. Moreover, 2nd-tier countries being able to afford world-class infrastructure, forts, shipyards, and massive armies and fleets wouldn't make much sense.

Hopefully opium being a money printer and the current system of most countries would ban opium in the 1900s would increase money supply in the early game while having less of an effect later on. I do need to the civilized country opium events so that a human player would actually want to ban opium now though.

[Xylephony]

Could non-top-tier countries not being able to afford much be intended?

Of course yeah, it absolutely is. My observation was more that the consequence of them not being able to afford any of that is basically that they will never ever save money, because they'll always spend it as soon as they can on their leftover infrastructure projects, and it makes them even less resilient to sudden economic downturns or wars. It's not something that's solvable, like I mentioned, but it's just frustrating.

I think you can probably re-implement all the economic reversions, if you wanted to including the opium demand you had previously rather than having opium as a money printer. Now that we know what we need to look at and when we need to be looking for it, it might be that those demand levels were actually viable and I was mistaking how much they impacted the global economy.

[ujtcelvn]

Yeah, CWE turned that into feature by having a "print money" decision which is as jank as it sounds. The development branch has a version with the opium and all the economy changes back in, I'll test it some more too.

[Xylephony]

So unfortunately I think the test I just did is a little FUBAR. I can say that the turning point for treasury reserves appears to be slightly later than I observed in my earlier tests with these economic changes, but that's not definitive because the rest of this test has been quite odd. And the most important thing, anyway, is that a surplus is being gathered at roughly the right time.

POPs seem more militant than usual, though. Especially given what just happened I'm trying to be more cautious about drawing sudden conclusions, but I saw anti-Qing rebels in 1840, Tunis was toppled to rebels in the same decade, Spain lost the Philippines in the 1860s and Russia entered a completely fatal classic-style rebel spiral starting in the late 1850s. I assume demand is the main driver here, but demand of what I'm not sure. I did see that several basic life goods were a hundred units or so short in the early game - fish, fruit, livestock and grain especially. I don't know if that could have this kind of impact, though; for uncivs, certainly, but I would think for civilized tags they would be high enough in the market priority to buy their goods needs. That said, for the same reasons that I wouldn't take the savings as definitive yet, I wouldn't treat this as definitive either.

Russia was one of the big problems in this test, with two main issues. First, with the P&E 4.0 changes for some reason that's utterly beyond me Russia is passing Ruling Party Only with some regularity - this is the second time I've seen it happen. I don't know how they're getting sufficient reactionaries in the UH to do it (unless the AI actually has logic for dissolving the UH?) but it's causing them to be supremely unstable, they can't do any gradual reforms until they break to rebels and then from there it's a rebel spiral into hell. I also wanted to ask about that as my second point, because in this test I saw Russia fall to Jacobins, go from an Absolute to a Constitutional monarchy, then immediately flip to Semi-Constitutional with Panslavics re-appointed as the ruling party. I want to say I've also seen this happen to some other absolute monarchy tags, though I couldn't swear to it. Is this an event EEM added somewhere, or is this something I never noticed before from HPM? Because I think it's keeping the Jacobins militant whenever Russia falls to rebels, and pushes them back into a spiral they might've avoided.

[ujtcelvn]

turning point for treasury reserves appears to be slightly later fish, fruit, livestock and grain

This is very curious because I did not decrease the values for these compared to HPM. (except opium not printing money anymore) It might be the 1% factory throughput from steamers - I removed that just now

Russia

There is an event (that was already in HPM) that lets you become Ruling Party Only if you have reactionaries in power and your upper house is >30% reactionary. Russia starts with an appointed and 21% reactionary upper house so getting to 30% and the event firing is quite possible. How immediate does the flip occur? It's probably the revolution event turning it into a constitutional monarchy but Russia still not having non secret ballots or secret ballots and then flipping back into Semi-Constitutional. The Panslavics returning is probably them winning the election that fires due to the revolution.

[Xylephony]

There is an event (that was already in HPM) that lets you become Ruling Party Only if you have reactionaries in power and your upper house is >30% reactionary.

Russia probably needs to have a setup event similar to the Papal States, then. Starting them with Panslavists has most of the countryside reactionary, and that's probably what's returning such high rates of reactionaries in the UH and getting it stuck like that. The event is fine, but it shouldn't be normal for Russia I think, especially given this is the period where Russia's tight grip over its population has already started to slip and several westernizing uprisings have already taken place. Maybe setup should just make Russian POPs 20% less reactionary? That would probably put most of them back into conservative alignment.

How immediate does the flip occur? It's probably the revolution event turning it into a constitutional monarchy but Russia still not having non secret ballots or secret ballots and then flipping back into Semi-Constitutional. The Panslavics returning is probably them winning the election that fires due to the revolution.

I would say it's usually within a month, so that's believable.

[ujtcelvn]

The Russian pops will still stabilize at around 38% reactionary even with a starting event. I think it's better if the Russia just starts with the conservative party, it's already called the Slavophiles with Moralism, Residency, and Jingoism which is pretty reactionary already. Also, both the Slavophiles and Nicholas I disliked serfdom so Russia should still be allowed to abolish serfdom early on if they really wanted to. If they start with Reactionary it's easy for them be stuck being reactionary forever until rebels overthrow them, unless I railroad it by adding a "Tsar Alexander II" event that forces the ruling party to be conservative or liberal. I don't really want railroaded monarch events like this because then to be consistent every country would need such an event which would make politics too railroaded and too bloated - for example, the Papal States should change to a conservative ruling party in 1846 after Pius IX becomes pope as well.

[Xylephony]

I agree with you RE: custom events, but I don't intend to make any further changes with the P&E patches regarding starting political parties (the 4.0 release itself was rounding up some final issues), so that shouldn't really be an issue that we'll run into. Properly modeling the setup for Russia and the Papacy would be the only custom events that would ever be needed.

I also take your point regarding keeping Russia Slavophile, but then what exactly is Panslavic meant to represent? I mean one could argue the obvious in that it's meant to represent a party which supports an aggressive pan-Slavic identity and support for Slavs abroad, but Russia did do that during the period, under both Nicholas and his successors. What position is there that's further to the right he was, both in terms of geopolitics and on clamping down on dissent and even basic services like education at home? I really think that Panslavic is the logical party to start is as.

I think the issue of the ruling party can be resolved without needing any particularly heavy-handed implementations by just making it that the Peasant's Reform switches the ruling party to Slavophile. That would avoid the need for any new event representing a specific monarch, which I agree should absolutely be avoided, and also go some way to representing Russia's long road towards modernization and gradual opening-up. The only thing that would then need to be resolved is the UH being switched to ruling party only. If you still think that'd be too heavy-handed with the modeling then switching back to Slavophile is an option, certainly, I think it just misses exactly how reactionary Nicholas was, and politically unconscious the serfs were at the time.

[ujtcelvn]

what exactly is Panslavic meant to represent

Maybe the super-reactionary pro-serdom nobles? Because the Slavophiles wanted to abolish serfdom.

[Xylephony]

I suppose that makes sense, if the political differentiation of Panslavist is pro-Serfdom. I can buy into that.

[Xylephony]

This most recent test looked very good. Iron was a bit too plentiful early game but it's possible that was just circumstantial. Aside from that, everything seemed to ride that perfect supply line just below demand, except for basic goods which were plentiful enough. Industrialization was stable across most states, even small-scale industry in poor nations, and the savings turn was in full swing already by 1875. That's just a single test, and not all the way to endgame, but it looks promising.

[ujtcelvn]

By the way which version are you using? The main branch or the development branch? Your Australia save file about the Red Turbans seems to be of the main branch that doesn't have the steamers modifier and with opium as a money printer but I want to make sure.

[Xylephony]

I grabbed development for my previous test, but I might've fucked up and grabbed main for the Australian test. The temp test (from the comment from 4 days ago) was 100% with the dev branch.

[ujtcelvn]

Yeah, you used the main branch for the Australia test.

The temp test (from the comment from 4 days ago) was 100% with the dev branch.

Oh sweet - I made it the master branch now.

[Xylephony]

Thanks--by the by I am still interested in helping but I will likely be busy for the next month or two. I will try checking back in and doing some more long-term economic tests when I can get free.

[ujtcelvn]

I've also been extremely busy recently and haven't had the chance to do many updates but I should be getting more free sometime soon too.

[Xylephony]

@ujtcelvn I played that India game almost to the end date and am still very worried about ship numbers. The big nations like Britain all have a large amount of ships, but there are GPs in the 1930s that have none, including Italy at 2 and Japan at 3! More worryingly IMO is that both nations are rolling in cash, Italy at 3 million and Japan at 14 million stockpiled at this point, but they absolutely won't spend any of it on a navy, even though their ruling party is and have been Jingoist for decades. I can't understand where the calculus is going wrong here, but even when cash is available it seems like some nations just don't want to build ships, which is a critical problem for an island nation like Japan.