Convert weather particles to be rendered in 3D as part of game world by using GraphicsObject/RuneLiteObject

[Billybishop]

Another enhancement related to immersion/seamlessness with the game world.

The goal of this enhancement is to convert the existing particle rendering approach from it's 2D implementation into one that utilizes the RuneLiteObject class, a modified GraphicsObject, in order to define the world particles (currently named RainDrop) as game world objects. To make this conversion possible the new RuneLiteObject implementation will require a simple 3D textured model with which to render, details on how to reuse a model from the Game's Cache is outlined below. Due to these changes the current 2D rendering methods used by the Weather plugin will become obsolete.

The most notable improvement of doing a proper GraphicsObject implementation of the weather particles, instead of rendering pseudo-3D particles directly to the screen as pixels, is that the new game world particle objects will naturally render as part of the Z-buffer for game models, which means particles rendered in the world space will be rendered within their natural position behind, or in front of other geometry depending on their 3D position in the world- that is to mean it provides natural occlusion that all world objects are affected by. Visibility occlusion (eg. preventing weather particles from rendering when behind a wall) is not easily achievable when rendering the particles as screen pixels, as to implement an occlusion technique for the screen pixels would require a form of ray or line casting, and manual checking of Z-buffer depth for intercepted models. Not only would this be an expensive approach, but it would require more work to get the same result. So the advantages are many: Save development time, better optimized rendering of weather particles, and more cohesive integration of weather particles in the game scene.

Thankfully after searching around the Runelite repositories I was able to find a good example that already makes use of RuneLiteObject and can be a reference for developing an implementation of 3D weather particles: https://github.com/runelite/runelite/blob/43e8a57cadc232c151c002cbfe5a7e4dd95e8b53/runelite-client/src/main/java/net/runelite/client/plugins/grounditems/Lootbeam.java#L41

In the linked example the author created animated 'Loot Beams' as objects in the game world by setting a model & model data to load from the game cache, lighting it, and applying an existing animation:

public Lootbeam(Client client, ClientThread clientThread, WorldPoint worldPoint, Color color, Style style)
{
    this.client = client;
    this.clientThread = clientThread;
    runeLiteObject = client.createRuneLiteObject();

    this.color = color;
    this.style = style;
    update();
    runeLiteObject.setShouldLoop(true);

    LocalPoint lp = LocalPoint.fromWorld(client, worldPoint);
    runeLiteObject.setLocation(lp, client.getPlane());

    runeLiteObject.setActive(true);
}

For the weather particle object, we could potentially create our own 'falling rain' Animation and bundle it with the plugin, then apply it as an animation to the object, but I'm not too familiar on how to go about creating an Animation. However the good news is it looks like we can use the #setLocation method in RuneLiteObject to update the object's position, so I think in the interim we should try using that to see if it suffices for 'animating' the particles as falling in the game world.

For the model of rain particles, I think we could reuse this model in the cache that represents the 'Needle' object, then resize, recolor and maybe set to some level of transparency:

'Needle' Model ID = 2770

I was able to find the Model ID number for the Needle object by searching through the OSRS model database using this tool on OSRSBox: https://www.osrsbox.com/tools/model-search/

You'll want to poke around the LootBeam example I linked above to understand how defining the full usage of a model is done, but it relies on the usage of Client#loadModel, it's usage described here:

/**
* Loads a model from the cache and also recolors it
*
* @param id the ID of the model
* @param colorToFind array of hsl color values to find in the model to replace
* @param colorToReplace array of hsl color values to replace in the model
* @return the model or null if it is loading or nonexistent
*/
@Nullable
Model loadModel(int id, short[] colorToFind, short[] colorToReplace);

If anyone knows how to use Sprites/Textures as game objects so we can further optimize this, as opposed to loading an actual model for particle objects, then that could be a more desirable approach and would love to know how that is done.

Below is an example image of 3D rain particles which includes floor collision rendering, for reference on what 3D weather particle implementation should ideally look like:

[bogstandard]

Wow, this is a fantastic idea. Coding for 3D rendering & spaces have always baffled me somewhat though, I'll have an experiment with this idea in the coming weeks. As always any outside contributors are welcome to offer code samples or implementation ideas. I'll mark this one in the ReadMe as a future plan.

[Billybishop]

So here's a way to determine whether a 3D particle is on a tile that has a roof or not, that way it won't render weather particles inside buildings, this is a prototype that expects a Particle object with a X Y Z coordinate scheme, so your current RainDrop object would need to be modified, or a new 3D RainDrop object created to be able to implement this:

Tile current_location = this._client.getLocalPlayer().getLocation();
Weather locational_weather = WeatherService.get_regional_weather(current_location);
int z_plane = client.getPlane();

this.render_weather_particles(locational_weather);

void render_weather_particles(Weather weather_data) {
    for (WeatherParticle weather_particle : weather_data.particles) {
        //Get the flags of the tile that our particle is on falling within
        int tile_flags = settings[weather_particle.x][weather_particle.y][z_plane];
        if ((tile_flags & Constants.TILE_FLAG_UNDER_ROOF) != 0)
            //Call graphics drawing method for weather particles from here
    }
}

[Billybishop]

I was thinking more about this enhancement the other day, and in order to handle occlusion of objects from the Z-buffer we would need to rethink the approach I originally described. So with that in mind I did a little research on how to use and manipulate an actual GraphicsObject to be used by the game engine for rendering models in the game world; I came across an implementation of that such idea which was done for the LootBeams plugin: https://github.com/runelite/runelite/blob/9966cad9ea37c045cf5c12efba542a0fe6f0bf96/runelite-client/src/main/java/net/runelite/client/plugins/grounditems/Lootbeam.java#L41

I also went ahead and updated the original enhancement to describe this new approach and how we could reference LootBeam for creating our own weather particle game object. If I come across some more free time I'll work on a prototype for this and document any progress here.

[bogstandard]

This type of effect is now provided by the spiritual successor to RL-Weather: 3D-Weather. Because this enhancement is provided by plugins elsewhere I'm closing this issue.

Because 3D-Weather now provides players with more modern effects, so RL-Weather will continue to focus on traditional older "crunchier" feeling effects.

[ScreteMonge]

Hey, I just happened to see a plugin change merged for RL-Weather through RL's discord. Given that it's been some time since RL-Weather had been updated, I assumed you had put the project away.

I'd just like to thank you and your plugin for inspiration on 3D Weather through multiple levels, both in this very thread as well as through examining your work. From using RLobjects to begin with, to guidance for how to handle sounds, where to source sounds, implementing lightning flashes, and probable future updates like using a WeatherAPI.

I know that you had very clearly wanted to implement 3D weather effects at some point, so I do apologize if it feels like I've stolen your opportunity. I've had you credited in the ReadMe since the plugin's release, if that means much to you.

[bogstandard]

Hey @ScreteMonge a pleasure to meet you! I'm not at all thinking you've stolen anything, it was an absolute unexpected delight to see your plugin and the credit to this one (I noticed by chance a few weeks ago). This project is often quiet because I don't have much time to work on it nowadays, so it's fantastic to see another project implement some of the unreachable ideas that I had got stuck on.

If anything 3D Weather has allowed me to rethink the focus of this project, aiming more for a proper "crunchy" old school feeling that's more suited for play in the classic UI layout.

If you need any advice or have questions on how I've implemented stuff I'm more than happy to help! By the way I've also got my other plugin Dice that uses some other tricks to create on-screen graphics in an efficient way if you're interested.

[ScreteMonge]

@bogstandard good to meet you too! Same with you, I'm very happy for both plugins to coexist and serve their relevant niches.

There's a very good chance I'll have questions, so thanks for the offer and resources!

[Billybishop]

Nice to see someone figured it out! Back when I was playing around with this idea and wrote up this thread I had trouble figuring out how to manipulate GraphicsObjects or a primitive mesh in general. At the time I don't think there was an easy way to rotate graphicobjects or mesh wrappers that was baked in, and was clueless as to where to start if I wanted to bake my own mesh and import it to the client for this purpose. Congrats on the release and merging of 3D weather. I look forward to see what you guys get up to next. I had fun exploring this.