Persist ALL THE THINGS

[WildWeazel]

Trees: World.generateTree(Location, TreeType) Sugarcane: reuse soil check to find height of plant, then check if block above is air Cactus: same as sugarcane, also check for solid blocks surrounding air block Pumpkins/melons: search for attached fruit, if none search for adjacent air block above dirt/farmland, remove stem from DB, add again when fruit is broken?? Cocoa: ???? Mushrooms: ????

[ttk2]

Coca just always grows out. Should be easy to figure out.

On Fri, May 29, 2015, 10:41 AM Travis Christian notifications@github.com wrote:

Trees: World.generateTree(Location, TreeType) Sugarcane: reuse soil check to find height of plant, then check if block above is air Cactus: same as sugarcane, also check for solid blocks surrounding air block Pumpkins/melons: search for attached fruit, if none search for adjacent air block above dirt/farmland, remove stem from DB, add again when fruit is broken?? Cocoa: ????

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16.

[erocs]

I wonder if the BlockChangeDelegate variant of generateTree will be needed to check for reinforcements. Hopefully it will just fire a growth event. Too bad it doesn't take a maturity level for the tree being generated.

[benjajaja]

I'm interested, but I'll have to take a better look at the code before I can tell you if I can do this or not. Will report back. Also mushrooms, they have some specific spreading rules.

[WildWeazel]

Yeah, I don't think fully replicating vanilla behavior will be worth it. Hopefully that won't cause any problems.

I can do it but I'd rather keep focusing on Factories/Contraptions if you want to take it.

[benjajaja]

I was just thinking about the farms for these crops work: Cacti: when it grows, the topmost block that grew is broken instantly. If we change this to persistent, then when the chunk loads after a long period, there will be MASSIVE amounts of instant growths and consequent block breaks and item drops. We could technically skip the block update and break events, but not the spawning of all the itemstacks.

I think that this may still be better than AFKing it performance wise, but the there would now be sudden lag spikes if the farm is big enough.

It may be worth exploring Contraptions to harvest cacti magically without performance impact, rather than creating huge amounts of blocks, break events, or itemstacks.

The same goes more or less for pumpkins, melons, sugarcane and mushrooms. There aren't that many autofarms like for cacti, but they do exist. Melons, pumpkins and sugarcane can be farmed with pistons from an interval or BUDs, and mushrooms with washing them away in an interval.

Ideas?

[ProgrammerDan]

Good analysis. Could the grouping of drop stacks be leveraged to reduce the performance impact by spawning dropped stacks instead of individual items?

Also, given the expense either way, perhaps finding the nearest hopper connect by water on a Y below the cactus, unimpeded by other blocks, would be feasible. Then stuff the hopper and any connected chests with cactus based on computed output.

[benjajaja]

We could group stacks yes, but don't count on putting them in hoppers - most farms collect items with water chutes into a central hopper.

[ProgrammerDan]

Yeah, that's exactly what I'm talking about, gipsy. Programmatically "follow" the water that's under the cactus source block, to the nearest hopper connected to the water. Etc.

Just figured that such a system might be less painful to the server then handling all the entity spawns on chunk load; could pre-compute association of cactus source to hopper/chest stack on chunk unload, then on chunk load, fill the chests/hoppers with items based on the expected rate of growth during the chunk's offline time.

[ttk2]

that sounds like a job for a contraption, maybe an area collector contraption could be possible? Like a longer range hopper, but if we make them too easy to sue everyone will use them and we might negate our gains.

[benjajaja]

Eh, I think following water flow is going down the rabbit hole too much, perhaps? You'd have to calculate that on each chunk load unless I'm wrong.

Contraptions could support something like working together with RB to do basically the same thing sans following water flow. Maybe "absorb all potential offline cactus drops from above and in radius X" on chunk load. If RB can expose both chunk crops (cacti etc) and offline time to contraptions, this could make a good compromise. Existing farms could be adapted easily, just replace collection point with this contraption.

BTW I have to post to contraptions repo 'cause I couldn't figure out ow to make them work at all :|

[ttk2]

I am not sure what state the contraptions code is, feel free to ask questions over there.

[ProgrammerDan]

@ttk2 I like that idea, it's a great thought for contraptions, although it's functionally identical to what I was suggesting.

@gipsy-king Nah, you'd only have to compute it on chunk unload, then apply the computation on chunk load, after which point vanilla behavior takes over and the computed behavior is inactive. Since it's on unload, you could spawn a FutureTask to handle the computation based on a snapshot of the chunks unloading, and allow the computation to "eventually" complete, with early cancelling of the job if the chunk re-loads before computation is done. That said, I like the contraptions idea. The "benefit" of my proposal would be no alterations to existing farms, but obviously that's not a priority in the grand scheme. Also, the Contraptions approach would be basically the same, but I'd guess more persistent.

Now we're getting into "wrong repo" territory, but with a contraption, you could suppress cactus growth and auto block break events (eliminating item spawn) and just have the item drop appear within the contraption. It should be relatively easy to define some system to mark out "area of control", but for the love of god don't make it a voronoi-style area-of-control; we're talking mechanisms, it should be a mechanically defined volume (placing lored blocks, "blueprint" loaded into the machine, something -- literally anything -- besides automatic subdivision based area control)

[ttk2]

How would RB and contraptions communicate if we wanted to do this. Obviously it requires coordination but we want to keep them separate.

On Tue, Jun 9, 2015, 8:03 AM Daniel Boston notifications@github.com wrote:

@ttk2 https://github.com/ttk2 I like that idea, it's a great thought for contraptions, although it's functionally identical to what I was suggesting.

@gipsy-king https://github.com/gipsy-king Nah, you'd only have to compute it on chunk unload, then apply the computation on chunk load, after which point vanilla behavior takes over and the computed behavior is inactive. That said, I like the contraptions idea. The "benefit" of my proposal would be no alterations to existing farms, but obviously that's not a priority in the grand scheme. Also, the Contraptions approach would be basically the same, but I'd guess more persistent.

Now we're getting into "wrong repo" territory, but with a contraption, you could suppress cactus growth and auto block break events (eliminating item spawn) and just have the item drop appear within the contraption. It should be relatively easy to define some system to mark out "area of control", but for the love of god don't make it a voronoi-style area-of-control; we're talking mechanisms, it should be a mechanically defined volume (placing lored blocks, "blueprint" loaded into the machine, something -- literally anything -- besides automatic subdivision based area control)

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16#issuecomment-110350716 .

[WildWeazel]

I think we could just expose a simple API to look up rate by plant and block/biome, basically a shortcut to what the click event handler does, and let Contraptions derive its output from that.

[ttk2]

In the meantime let's do other crops one at a time. Getting something done is better than waiting around forever.

On Tue, Jun 9, 2015, 8:56 PM Travis Christian notifications@github.com wrote:

I think we could just expose a simple API to look up rate by plant and block/biome, basically the and let Contraptions derive its output from that.

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16#issuecomment-110556380 .

[benjajaja]

Regarding plugin communication, @WildWeazel could RB emit a custom "chunk-load-harvest" event that contains data about how many blocks have been "spawned"? RB could then create these blocks, unless the event is cancelled by Contraptions presumably to fill the contraption. This would allow for a future implementation of contraptions while having persistence now.

I'm going to start with melon and pumpkin, then sugarcane. I don't think that there are many auto farms for these, although I have a tiny BUD autofarm for melons. Sugarcane is mostly farmed at an interval I think.

Cacti are a different story, because cactus farms are continuously dropping, so AFKing will always give more output than output from offline persistence, since the latter maxes out after the time it takes for all cacti to grow. I think cacti NEED contraptions, or players will continue to AFK cactus farms.

[WildWeazel]

Perhaps so but does that also account for the manual stick checks? Do they trigger a whole chunk update?

[ProgrammerDan]

@gipsy-king Although I agree you should go ahead and tackle melons and pumpkins, do note that there are quite a few auto farms for both, and some are quite massive.

[ttk2]

lets get what we can done while we discuss the rest.

[WildWeazel]

trees at least should be easy enough, ironically

[WildWeazel]

20 for cocoa

[ghost]

http://forum.civcraft.co/t/starting-soon-realisticbiomes-cocoa-persistent-crop-growth/457/2

I updated the jar and the config to the latest build. inb4 I borked something.

[ttk2]

it looks like Coco works, next?

[WildWeazel]

treeeees

[ttk2]

http://forum.civcraft.co/t/realisticbiomes-persisting-thread/484/1

made a discourse thread there.

On Wed, Jun 17, 2015 at 9:44 PM Travis Christian notifications@github.com wrote:

treeeees

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16#issuecomment-113018878 .

[benjajaja]

@WildWeazel I have melons and pumkpins working, I just have to clean up the code a bit and try to test some more.

Since RB will be tracking the fruits' ripeness from the stem, I decided to add another field to the crops table. The same can then be used for sugarcane and cactus.

Trees don't need this since they grow only once in their lifetime.

[benjajaja]

With pumpkin, trees and cactus we now have blocks that may depend on surrounding blocks being loaded. This could be a problem if the surrounding chunk is not loaded.

Pumkpin stems grow their fruit into a surrounding block, trees' structures grow outwards of their sapling's column, trees may be in a 2x2 grid for their large version, and cacti need to check surrounding blocks to know if they break or not. Any of those affected blocks may be outside of the block's chunk and not loaded.

Issues so far:

• Trees that generate on chunk border are cut off there if neighbor chunk is not loaded.
• 2x2 arrays of saplings may be split across chunks, these may not grow into large trees, depends on chunk load order.
• Pumpkins/melons aren't a huge problem, worst case they won't grow if the stem is in one chunk and their free spot in another. Some farms may never produce persistent fruits, at least in some spots.
• Cactus may grow next to a block and not break, if said block is in another chunk and not loaded.

Except cut-off trees, I haven't tested for these cases, but I'm pretty sure that they will happen.

@WildWeazel @ProgrammerDan any ideas? Maybe instead of growing the chunk in the same tick it is loaded, schedule a task so that if there are other chunks being loaded, they would be available?

[ttk2]

If RB tried to load adjacent chunks the question is if it will ever stop.

On Mon, Jun 29, 2015, 3:28 AM Benjamin Grosse notifications@github.com wrote:

With pumpkin, trees and cactus we now have blocks that may depend on surrounding blocks being loaded. This could be a problem if the surrounding chunk is not loaded.

Pumkpin stems grow their fruit into a surrounding block, trees' structures grow outwards of their sapling's column, trees may be in a 2x2 grid for their large version, and cacti need to check surrounding blocks to know if they break or not. Any of those affected blocks may be outside of the block's chunk and not loaded.

Issues so far:

• Trees that generate on chunk border are cut off there if neighbor chunk is not loaded.
• 2x2 arrays of saplings may be split across chunks, these may not grow into large trees, depends on chunk load order.
• Pumpkins/melons aren't a huge problem, worst case they won't grow if the stem is in one chunk and their free spot in another. Some farms may never produce persistent fruits.
• Cactus may grow next to a block and not break, if said block is in another chunk and not loaded.

Except cut-off trees, I haven't tested for these cases, but I'm pretty sure that they will happen.

@WildWeazel https://github.com/WildWeazel @ProgrammerDan https://github.com/ProgrammerDan any ideas? Maybe instead of growing the chunk in the same tick it is loaded, schedule a task so that if there are other chunks being loaded, they would be available?

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16#issuecomment-116521463 .

[benjajaja]

It should not, but maybe it could wait so that surrounding chunks have a chance to load. But I don't know if "waiting" is deterministic or feasible at all.

[WildWeazel]

What happens when a regular tree growth event causes it to grow into an unloaded chunk?

[erocs]

Do growth events even fire on the outer-most loaded chunk or do they only fire within the activation range?

On Mon, Jun 29, 2015 at 10:24 AM, Travis Christian <notifications@github.com

wrote:

What happens when a regular tree growth event causes it to grow into an unloaded chunk?

— Reply to this email directly or view it on GitHub https://github.com/Civcraft/RealisticBiomes/issues/16#issuecomment-116767933 .

[ProgrammerDan]

Good point, should just be within activation range

[benjajaja]

@WildWeazel @erocs good questions, couldn't find the answers. Strangely I can only reproduce this with 2x2 large trees, and it depends on where I stand relative to the chunk when connecting, presumably because it affects the chunk load order.

[ProgrammerDan]

Looking here: http://minecraft.gamepedia.com/Chunks seems to indicate that view distance as sent by the server is the controlling factor, so could have BUD anywhere inside that transmitted/loaded view distance.

@gipsy-king wonder if it would be possible to cache growth events for a chunk until every chunk connected to that chunk is loaded, then replay them.

[benjajaja]

@ProgrammerDan I don't know, maybe it's possible, but what if not all (expected) chunks load, or what if they just load because a player gets near them... seems like a can of worms to me :/

I figure that for melons and pumpkins it won't probably be an issue since it is a bit "random" in the sense that it depends on chunk load order which depends on where the player logs in or approaches. Worst case, a fruit doesn't grow that time.

For cactus it could be an issue, if a cactus block is created next to a block (in neighbour chunk which was not loaded). I don't know if in that case the cactus breaks (as if you place a block next to it) or if it stays (this does happen if you simply set a surrounded block to cactus). This could be a problem if cactus farms start growing "impossible" cactus blocks that are never broken, unless removed manually.

For trees, well, it's really ugly.

[ProgrammerDan]

Was thinking specifically trees, I don't think it matters for the other crop types (could be wrong).

Using a deferred mechanism would work great, here's a psuedo sketch:

1. Chunk loads, do the following:

   a. Inform chunk cache manager of new chunk load.

   b. Chunk cache manager checks neighbor chunks for (1) load status and (2) cache presence.

   c. If the new chunk fully completes a neighboring chunk (all neighbors are now loaded) which has a cache, create a new asynch task to replay all events in the cache

   d. If the new chunk loading is not fully complete, create a new cache for it.

2. As tree growth events occur, check if a cache exist. If it does, add to it and cancel the event. Otherwise, let the event procede.

3. When a chunk unloads, inform the cache manager.

   a. Chunk manager checks for neighbors that are now incomplete and creates a cache for them

   b. The chunk that unloaded, if it had a cache, that cache is discarded.

It's fairly straightforward unless I'm missing something obvious.

[ProgrammerDan]

or simpler:

Cancel tree growth events if the chunk they occur in is not fully surrounded by loaded chunks.

[benjajaja]

Cancel tree growth events if the chunk they occur in is not fully surrounded by loaded chunks.

For some reason I like that better :innocent:

I'm gonna try with a very simple compromise:

• if all surrounding chunks loaded, grow tree
• if not, schedule to check again after a second, to give chunks time to load if they are actually coming from the same "action" (player login, approach)
• if surrounding chunks are still not loaded, give up
• else spawn tree (but check that chunk still loaded, plant is there etc)

Sounds right?

[ProgrammerDan]

Sounds pretty ideal, it's a nice balance.

[ttk2]

go for it.

[WildWeazel]

My only concern with that is scaling, are you creating a new runnable for each chunk load event? They're loaded sequentially from the player outwards, right?

[ProgrammerDan]

Good point. Perhaps a new chunk that fails the check should test if a runnable is already "queued" and attach itself to that runnable. Yay threadsafe coding.

[benjajaja]

@WildWeazel @ProgrammerDan should I add mushroom persistance? The easy route would be to subsitute with huge mushrooms like trees and nerf small mushrooms to the point of being useless, but that would upset A LOT of players. Normally I wouldn't care but this is delicate in the sense of players trusting that what they build won't be rendered useless anytime. The upside would be that chopping down huge mushrooms, like trees, is very complicated to bot.

Emulating the algo to grow shrooms isn't too difficult either. It has to be a 9x9x5 bounding box centered on the shroom, and in that box there can be max 5 shrooms, and a free spot requires to have dirt/grass/mycelium and low light or something like that.

[ttk2]

how much lag is mushroom farming? if its not much efforts may be better centered elsewhere. Then again if its easy do it now.

[ProgrammerDan]

The mushroom spawning isn't the lag source. The issue is that mushrooms are harvested with flowing water released by pistons on a redstone clock. The water and pistons in large farms will be a significant lag source. I'm not aware of any designs that pop the mushroom using timed light sources, although that would be possible (and likely less laggy than flowing water).

So I don't see much value in persisting mushrooms, unless there's some means to harvest I'm not aware of, as the harvesting alone is the lag source, not the mushrooms.

[ttk2]

very well then, @gipsy-king we have interesting reports that they RB remove plant function might actually be removing random blocks.

[benjajaja]

@ProgrammerDan the lag source we're targetting with persistence is AFKing. That is, chunks being continuosly loaded over long period of time with all their possible lag sources.

@ttk2 that would be the block-update function, that updates the block(s) to whatever the plant's growth is supposed to be. When a plant gets removed (that is the spam log message from the other issue, fixing ASAP), it only gets removed from the tracking list of the plugin.

[ProgrammerDan]

@gipsy-king I know, I'm not making my point clear. My point is, collecting the plant is the lag source. That's going to be true regardless, your efforts would be in vain for mushrooms specifically. It doesn't use a fixed flowing water source for collection like cactus can or melons and pumpkins can. The water is flowed selectively... much of the lag is the infrastructure at harvest time, not during the larger AFK time.

[benjajaja]

@ProgrammerDan but the target of persistence is AFK, not much else. If you want to make it less laggy, we can do it for example with disabling mushroom spread entirely and allowing giant mushrooms to grow like trees (I don't remember who proposed that, I think it was in the forum discussion).

[ProgrammerDan]

Ok, fair point. I concede. Perhaps my comments could be reframed as "set ttk's expectations" so he doesn't interpret mushroom persistence with any behavior or lag change, as it won't provide it. Just lets people leave their farms alone, but they'll need to return in the same frequency as their water-flow harvest mechanisms to keep the same rate of return, so I hazard they won't stop AFK'ing.