EvEmu-Project / evemu_Crucible

Emulator for EvE Online's Crucible expansion
https://evemu.dev
170 stars 68 forks source link

Implement Sovereignty #90

Closed jdhirst closed 2 years ago

jdhirst commented 3 years ago

This is a stub feature for implementing player sovereignty in EVEmu. Part of project.

jdhirst commented 3 years ago

Things to resolve

Sovereignty Structures

Sovereignty System

Things to implement

Sovereignty Structures

Sovereignty System

jdhirst commented 3 years ago

@zhyrohaad Any ideas about how to resolve the global issues with the structures?

Also, do we already have something implemented where we can perform actions on the structures without a player being in the same bubble as the structure? Currently, the structure will jump to the next phase without executing the code in-between.

zhyrohaad commented 3 years ago

wow. sorry for the delay....i wasnt notified of this (not sure why notifications arent working for me)

Any ideas about how to resolve the global issues with the structures?

global structures...... will need the isGlobal attr set may have to adjust SE.isGlobal tag.....i dont remember how this is determined. it should take the attr data from the SE's Item object. i was messing with it at one point, but forgot where i left off. i think i had it fixed, and if so, SE.isGlobal will no longer exist. this was redundant and buggy.

side note: there is command to view objects in system and their attributes. i dont remember what it is offhand. you can also view locations of all static and most dynamic items in system map. to do this, enter .showall then preform a system scan using ship scanner. while scanner is open, change view to system map (F11?)

Also, do we already have something implemented where we can perform actions on the structures without a player being in the same bubble as the structure? Currently, the structure will jump to the next phase without executing the code in-between.

actions on a general timer? yes. use system tic. be careful adding to system tic; this can get expensive with too many checks/objects (you want to avoid looping thru all objects in system too often.....use existing tic call and add specific code to object that requires it.)

actions on objects using their own timers in a bubble.....no. bubbles go 'inactive' once all players have left. this also cancels bubble tic to save cpu cycles on server. while it is possible to add timers to items that require specific actions regardless of player presence, it is tricky.

another side note: i dont have official source set up on my dev box, and dont have time/inclination to do so. (dont like all the new shit) for me to begin developing again, i'll have to add changes to my source. then i have no way to test my updates to official. i dont have time to code anyway.

jdhirst commented 3 years ago

may have to adjust SE.isGlobal tag.....i dont remember how this is determined. it should take the attr data from the SE's Item object It definitely doesn't pull from the Item object, as those are set correctly according to my traces. TCUs, IHubs and SBUs are all correctly set as global in the item objects.

I've been trying to find how global objects actually become global in the entity system. What defines this? If I can figure that out, I can work backwards to figure out why its not being set.

there is command to view objects in system and their attributes. i dont remember what it is offhand. you can also view locations of all static and most dynamic items in system map. to do this, enter .showall then preform a system scan using ship scanner. while scanner is open, change view to system map (F11?)

Oh that's cool, I had no idea, I've been using GDB and checking attributes the hard way! 😅

actions on a general timer? yes. use system tic. be careful adding to system tic; this can get expensive with too many checks/objects (you want to avoid looping thru all objects in system too often.....use existing tic call and add specific code to object that requires it.)

I think we need onlining of all structures to be added to that timer as it breaks things otherwise. How would we go about adding structures in general to the system tic?

i dont have official source set up on my dev box, and dont have time/inclination to do so. (dont like all the new shit) for me to begin developing again, i'll have to add changes to my source. then i have no way to test my updates to official. i dont have time to code anyway.

No worries! I'll gladly pull in changes from your repo into staging and make sure things don't break too badly 😀

zhyrohaad commented 3 years ago

It definitely doesn't pull from the Item object, as those are set correctly according to my traces. TCUs, IHubs and SBUs are all correctly set as global in the item objects.

then something else is buggy here. my latest SE update was to fix the stupid "department of redundancy department" item data between the SE and II objects. i kinda remember there were a few things not cooperating, but details are fuzzy.

I've been trying to find how global objects actually become global in the entity system. What defines this? If I can figure that out, I can work backwards to figure out why its not being set.

few systems at work here...... SE objects shall(should) not contain specific item object data, only entity object data. isGlobal is an item object data. when Destiny State data is sent, item attributes are queried thru their II reference in SystemManager. (look for DestinySetState or something like that) the actual data is stored in the item's godma object (the attrMgr). when objects are added to a bubble, the data is queried the same way, using mostly same methods. if the object is NOT global when created/spawned/launched (via player in system) but IS global when player enters new system, then the SysBubble::Add() isnt working right. this can be tested by the following......add global item, leave system (jump out) and allow it to unload (should print msg in console. no log.ini switch). after system unloads, jump back in.
if the supposed Global object IS NOT on overview, the attribute set/query isnt right. (item object data error) if the supposed Global object IS on overview, then SysBubble::Add() isnt querying/sending attribute data correctly. (slight chance of other problems also)

also see SysBubble::Add() and SysBubble.m_entities map and associated methods for more info on how they are determined and sent.

there is command to view objects in system and their attributes. i dont remember what it is offhand. you can also view locations of all static and most dynamic items in system map. to do this, enter .showall then preform a system scan using ship scanner. while scanner is open, change view to system map (F11?)

Oh that's cool, I had no idea, I've been using GDB and checking attributes the hard way! 😅

look in the commands file. most of the things ive added are for data view/debug to avoid the hard way. LOL

actions on a general timer? yes. use system tic. be careful adding to system tic; this can get expensive with too many checks/objects (you want to avoid looping thru all objects in system too often.....use existing tic call and add specific code to object that requires it.)

I think we need onlining of all structures to be added to that timer as it breaks things otherwise. How would we go about adding structures in general to the system tic?

POS timers are POC. they are quite incomplete, which may be why it's not working as expected. however, they are (should be) already added to system tic in the m_ticEntities map when the object is added to the system. this is done on item creation via launch or system boot. from the system tic, Process() is called on all items that require it.
if the SE object code DOESNT have a Process() call (overridden from base SE class), then it wont get the tic.

can you give me more information about the structure online and how it breaks that you are referring to? i dont believe im following what you're trying to do here.

as a side note, Online is also an attribute managed by godma. it is accessed thru II reference (thru redirects...see bottom of II.h) MOST (99%, if not all) attributes are added/set/reset/queried/deleted correctly. this was verified after my item code and attribute mgr rewrite. if some attrib isnt working as expected, then the set/query methods may not be right. (the method calls, not the class code) this applies to all items.

No worries! I'll gladly pull in changes from your repo into staging and make sure things don't break too badly 😀

ok, that sounds good. i wasnt gonna ask, but you're already familiar with it as you've done it before. ;)

jdhirst commented 3 years ago

This message is mainly for keeping my thoughts organized, but I figured I'd share it here anyway.

So I've been debugging this some more using your commands and have found the following: isGlobal is definitely set on the item: image

However, TCU is shown as a non-global entity: image

So as I suspected, there is a disconnect between the item and the SE.

if the supposed Global object IS on overview, then SysBubble::Add() isnt querying/sending attribute data correctly. (slight chance of other problems also) Yeah, I think this is what is happening here, the item attribute is perfectly fine. The object is never global, either when it is spawned or after it is reloaded.

After some tracing, I found that we call addEntity for dynamic objects instead of adding using sBubbleMgr directly like we do with statics.

Statics:

        if (pSE->IsGateSE() or pSE->IsStationSE())
            sBubbleMgr.Add(pSE);
        if (pSE->IsBeltSE()) {
            sBubbleMgr.Add(pSE);
            m_beltVector.push_back(cur.itemID);
        }

Dynamics:

        AddEntity(pSE);

So I dove into AddEntity, and when we load the system and added a break right when we check if an object is global (so we can send a static ball), global is confirmed to be false for the TCUSE entity:

The check:

        if ((pSE->IsCOSE())
        or  (pSE->isGlobal())) {
            m_staticEntities[itemID] = pSE;
            if (m_loaded)   // only update when system is already loaded
                SendStaticBall(pSE);

Debug results:

(gdb) print *pSE
$3 = {_vptr.SystemEntity = 0x11de5d8 <vtable for TCUSE+16>, m_bubble = 0x0, m_system = 0x231c820, m_targMgr = 0xa21c3a0,
  m_destiny = 0x17bd8a0, m_services = @0x7fffffffcc40, m_self = {_vptr.RefPtr = 0x1134740 <vtable for RefPtr<InventoryItem>+16>,
    mPtr = 0xa21b030}, m_killed = false, m_radius = 1602, m_harmonic = -1, m_warID = 0, m_allyID = 99000000,
  m_corpID = 98000000, m_fleetID = 0, m_ownerID = 98000000}
(gdb) print pSE->isGlobal()
$5 = false

Since the item is fine and the entity is borked, now I need to look at the DynamicEntityFactory::BuildEntity() function, so there, we simply instantiate a new TCUSE object, derived from StructureSE, derived from SE.

Starting at the top, we see that TCUSE is setting global like this in the Init() function:

void TCUSE::Init()
{
    _log(SE__TRACE, "TCUSE %s(%u) is being initialised", m_self->name(), m_self->itemID());
    StructureSE::Init();

    // check for valid bubble
    if (m_bubble == nullptr)
        assert(0);
    m_bubble->SetTCUSE(this);

    // set global attribute
    m_self->SetAttribute(AttrIsGlobal, EvilOne, false);
}

However, this clearly isn't going anywhere. The global attribute is set correctly, that part is completely fine. However, this doesn't translate to the SE. One thought I had was that maybe we are already instantiating the SE based upon the item when the item doesn't have the IsGlobal attribute set to true? This would explain why the item seems to be fine but the SE is not.

I don't understand why running isGlobal() always seems to evaluate to false for TCUSE, even though everything else (including the InventoryItem that is being provided to instantiate the TCUSE object itself has that attribute returning true.

zhyrohaad commented 2 years ago

ok, the check for the (non-global) attrib uses some weird shit that i've had problems with....its not accurate. i probably should work on getting the actual check working instead of the base/derived overrides that are currently used. would be a good idea as there are other things that use it.

thinking back on this one, i dont think any structSE::Init() are called, or are not called correctly. i kinda dropped off at the Init() and it is very likely incomplete.

i am working on setting up my dev environment again and updating with latest developments, and will look into it more soon.

jdhirst commented 2 years ago

ok, the check for the (non-global) attrib uses some weird shit that i've had problems with....its not accurate. i probably should work on getting the actual check working instead of the base/derived overrides that are currently used. would be a good idea as there are other things that use it.

When you say the check, do you mean the function in InventoryItem? Because yeah that check does always seem to evaluate to false for the SE derrivative.

What exactly is the trigger for the client to always show something in the overview even when not in-bubble? Is it the attribute 1207 which causes it to be global? Because that is set according to the object attrlist.

When you mentioned:

if the supposed Global object IS NOT on overview, the attribute set/query isnt right. (item object data error)

Do you mean on the overview globally or at all? Because it is on the overview, just not globally (warping away causes it to vanish from overview)

thinking back on this one, i dont think any structSE::Init() are called, or are not called correctly. i kinda dropped off at the Init() and it is very likely incomplete.

I confirmed that those do get called as I put a breakpoint in there and made sure that it does get called when the structure is initialised.

i am working on setting up my dev environment again and updating with latest developments, and will look into it more soon.

Awesome, thanks! This one has been really breaking my brain lately haha

zhyrohaad commented 2 years ago

When you say the check, do you mean the function in InventoryItem? Because yeah that check does always seem to evaluate to false for the SE derrivative.

the check passed to the II object is redirected to attrMgr, but yes, that one. however, from the SE, it is hard-coded based on SE type. see SystemEntity.h and other SE.h files.

What exactly is the trigger for the client to always show something in the overview even when not in-bubble? Is it the attribute 1207 which causes it to be global? Because that is set according to the object attrlist.

so, entities are sent to client via AddBalls, as im sure you already know. this data is pulled from 2 sources, SystemManager for globals, and SystemBubble for dynamics. the check for globals are in both, sysmgr is for existing objects (planets, moons, stations, gates, etc), including dynamics already in system (like existing POS) and sent in Destiny's SetState data as Full State (as opposed to balls) this is found in SystemManager.cpp:1236 as part of the SystemManager::MakeSetState() method. it pulls globals from the m_staticEntites list, which uses multiple checks to populate, including the isGlobal check. SystemBubble::Add() also uses a similar check when adding objects, but will add "new" global balls using the "balls" SetState data. attributes for all objects are also sent with the SetState data.

that is the basic code flow, and the client uses this data to determine what objects are global or not.

When you mentioned:

if the supposed Global object IS NOT on overview, the attribute set/query isnt right. (item object data error)

Do you mean on the overview globally or at all? Because it is on the overview, just not globally (warping away causes it to vanish from overview)

overview globally.

i am working on setting up my dev environment again and updating with latest developments, and will look into it more soon.

Awesome, thanks! This one has been really breaking my brain lately haha

i can imagine....been there. ;)

jdhirst commented 2 years ago

I think we may have a different issue here; I remember that we were having issues with the structure code when I had originally define isGlobal to be true statically in the TCUSE class definition.

I did this again just now to test something, and this is what happened:

The object is global now in .list: image

However, the way it is being sent to the client is causing a client freak-out:

EXCEPTION #6 logged at  09/04/2021 9:33:00 ; caught by eve.ExceptionHandler
Context info: BluePyOS::PyError()
Caught at:
/client/script/remote/michelle.py(900) DoPreTick
Thrown at:
/client/script/remote/michelle.py(900) DoPreTick
/client/script/remote/michelle.py(1330) RealFlushState
/client/script/remote/michelle.py(1040) SetState
/../carbon/common/script/sys/cfg.py(1044) Hint
        self = <Instance of Recordset.EveLocations>
                       Key column: locationID, Cache entries: 19185
                       Field names: locationID, locationName, x, y, z, locationNameID
        key = 0
        value = [0, 'Territorial Claim Unit', -56965069440.0, 2717578085760.0, -6765923045760.0, None]
RuntimeError: ('Hint called with unsupported data type', <type 'list'>)
Thread Locals:  session was <Session: (sid:6669298093443447068, clientID:2888444, mutating:0, locationid:30000848, corprole:0xffffe07ffffff81L, userid:1, languageID:EN, role:0x63f8000280c40000L, charid:90000000, address:172.16.15.76:1066, userType:30, sessionType:5, regionid:10000010, constellationid:20000124, allianceid:99000000, warfactionid:0, corpid:98000000, shipid:140000177, solarsystemid:30000848, solarsystemid2:30000848, hqID:60000826, baseID:60000826, rolesAtAll:0xffffe07ffffff81L, rolesAtHQ:0xffffe07ffffff81L, rolesAtBase:0xffffe07ffffff81L, rolesAtOther:0xffffe07ffffff81L, genderID:0, bloodlineID:11, raceID:1, corpAccountKey:1000)>

EXCEPTION END

Unless I'm missing something, fixing the check so that isGlobal does return true, would simply result in this here, where it is indeed global, but its sent to the client in an invalid way, which causes the freakout.

Looking into that error more, seems like client function AddBalls() is not providing valid data to Recordset.Hint(). As for what that means specifically on the server side, not sure yet

zhyrohaad commented 2 years ago

as to the error.....

key = 0 value = [0, 'Territorial Claim Unit', -56965069440.0, 2717578085760.0, -6765923045760.0, None]

this is the problem here.....no UID. the key in this dict is the itemID, and first item in value is itemID. ofc you know a value of '0' is invalid. (i mention it here for those that dont know)

I think we may have a different issue here; I remember that we were having issues with the structure code when I had originally define isGlobal to be true statically in the TCUSE class definition.

they were all done this way, which is what i had removed, but havent pushed yet. base SE class has bool isGlobal() { return m_self->isGlobal(); } where item attributes (redirected through the m_self pointer) should be working properly.

as a side note, that .list method of mine is quite incomplete.....most globals are some form of StaticSystemEntity (SSE) or ItemSystemEntity (ISE) and will not have destinyMgr because they are non-mobile. the destiny state (Stop), current speed fraction (csf) and speed are all moot and wasting bandwidth (and cpu cycles).
but, i used common code for all items in that list....which puts me in that group. (make it work now, make it right later)

jdhirst commented 2 years ago

this is the problem here.....no UID. the key in this dict is the itemID, and first item in value is itemID. ofc you know a value of '0' is invalid. (i mention it here for those that dont know)

Yeah, I see that. So for some reason, the locationID is set to 0 when isGlobal evaluates to true.

After enabling a bunch of new log stuff, I found the following when now trying to generate the slimitem for the TCU:

09:49:35 [SE SlimItem] MakeSlimItem for StructureSE 0
09:49:35 [POS SlimItem] MakeSlimItem for StructureSE 0
09:49:35 [POS SlimItem] StructureSE::MakeSlimItem() - Territorial Claim Unit(0)
09:49:35 [POS SlimItem]       Dictionary: 8 entries
09:49:35 [POS SlimItem]        [ 0]   Key:     String: 'allianceID'
09:49:35 [POS SlimItem]        [ 0] Value:    Integer: 99000000
09:49:35 [POS SlimItem]        [ 1]   Key:     String: 'corpID'
09:49:35 [POS SlimItem]        [ 1] Value:    Integer: 98000000
09:49:35 [POS SlimItem]        [ 2]   Key:     String: 'ownerID'
09:49:35 [POS SlimItem]        [ 2] Value:    Integer: 98000000
09:49:35 [POS SlimItem]        [ 3]   Key:     String: 'warFactionID'
09:49:35 [POS SlimItem]        [ 3] Value:       None
09:49:35 [POS SlimItem]        [ 4]   Key:     String: 'typeID'
09:49:35 [POS SlimItem]        [ 4] Value:    Integer: 32226
09:49:35 [POS SlimItem]        [ 5]   Key:     String: 'posState'
09:49:35 [POS SlimItem]        [ 5] Value:    Integer: -2
09:49:35 [POS SlimItem]        [ 6]   Key:     String: 'itemID'
09:49:35 [POS SlimItem]        [ 6] Value:       Long: 0
09:49:35 [POS SlimItem]        [ 7]   Key:     String: 'name'
09:49:35 [POS SlimItem]        [ 7] Value:     String: 'Territorial Claim Unit'

So just as you said, the itemID is indeed 0.

Next, I created a few breakpoints in Structure.cpp to figure out why it works when isGlobal evaluates to false but not to true. So it seems that Init() doesn't get called when the object is global, so therefore the itemID is never set.

Log shows us that the entity is built correctly, but is never initialized, since the slimitem is built immediately without Init() being called:

10:16:16 [POS Trace] Created StructureItem for Territorial Claim Unit (140000190).
10:16:16 [SE Debug] Created SE for item Territorial Claim Unit (140000190) with radius of 1602.0.
10:16:16 [SE Debug] Created StructureSE for item Territorial Claim Unit (140000190).
10:16:16 [POS Trace] DynamicEntityFactory::BuildEntity() making TCUSE for Territorial Claim Unit (140000190)
10:16:16 [ServerInit] SystemManager::LoadSystemDynamics - 2 Dynamic System entities loaded for M-OEE8(30000848)
10:16:16 [ServerInit] SystemManager::LoadPlayerDynamics() - 0 Dynamic Player entities loaded for M-OEE8(30000848)

We add the entity to m_staticEntities instead of m_ticEntities, so therefore Init() is not called. So I added something new so that it can go through the list and see if an item is a Sovereignty entity on the static entity list, and then initialise it.

    // check for static entities which need to be initialized (such as sovereignty structures)
    for (auto cur: m_staticEntities)
        if (cur.second ->IsTCUSE())
            cur.second->GetTCUSE()->Init();
        else if (cur.second ->IsSBUSE())
            cur.second->GetSBUSE()->Init();
        else if (cur.second ->IsIHubSE())
            cur.second->GetIHubSE()->Init();

However, it seems for some reason that IsTCUSE() is not returning true for any item on m_staticEntities.

And the entity is indeed on the staticEntitiy list:

(gdb) print m_staticEntities
$13 = std::map with 103 elements = {<lots of uninmportant entities here>, [140000190] = 0xa105c00}

So now to question why that entity doesn't evaluate IsTCUSE() to be true.

Oh... turns out for some stupid reason I set it explicitly to false...

    virtual bool                IsTCUSE()               { return false; }

Huzzah! image

So now they are global :D

Now to just fix the other things for Sovereignty!

jdhirst commented 2 years ago

On to the next issue, I'm now looking at why everything stops working when you leave the system.

The first thing I did was add these sovereignty structures to the process tic as they weren't in there already (since they aren't part of m_ticEntities.

However, when you SetAnchor() them, they do indeed start to anchor, but if you leave the system while they are anchoring, they stop.

Logs for when it anchors correctly (while ship is in system):

11:31:38 [POS Trace] POS Mgr::Anchor()
11:36:39 [POS Debug] Module Territorial Claim Unit(140000467) Processing State 'Anchoring'

However, the Processing State 'Anchoring' message doesn't show up when not in the system. Since there is nothing special for the sovstructures, I would imagine this would also be true for all anchorable structures with timers including towers.

zhyrohaad commented 2 years ago

On to the next issue, I'm now looking at why everything stops working when you leave the system.

because the tic checks for empty system.
this was written before things needed tics in empty systems.

will have to add a check for active timers in system when empty.

as to the globals, im glad you find the error, but i dont really care for looping thru all statics in the off-chance some random object needs init.....theres a better way to do this. (i dont know how yet, becuase i still havent had time to look)

jdhirst commented 2 years ago

as to the globals, im glad you find the error, but i dont really care for looping thru all statics in the off-chance some random object needs init.....theres a better way to do this. (i dont know how yet, becuase i still havent had time to look)

I agree completely, this was merely a POC since I wanted to atleast get globals working so I could finish the rest of my implementation stuff since it was blocking that.

So I came up with a good way of solving this problem. Essentially, I create another list which has static entities which have operational actions (such as sov structures) which need to be handled regardless of whether players are in a system or not.

As such, this includes sov structures and Outposts (I plan on implementing outposts in 0.8.5 😄).

Now I can use this in place of everywhere I would use GetStaticEntities()

The added benefit of having such a thing is that now I can treat all of these types of objects like a set where I don't need to check for IsTCUSE() or IsSBUse() or IsIHubSE() etc when I need to refer to the whole group of them. This makes it extensible since when implementing new things in the future, I might need to add things to this list since there are probably more things that need to be handled when nobody is in a system.

For instance, any sov structures on a timer such as onlining, anchoring, etc must be able to complete this action when the system is empty or we will end up in an inconguous state where the entity may appear to be online/offline, but in fact the core actions which should have taken place (such as the actual creation of a new claim, or making a system become contested) never complete.

My implementation of solving this is by using m_delayTime from StructureSE and having the SystemManager UpdateData() function check to see if there is any operational static structure that has a delay time set, and avoid unloading the system if that is the case.

I ran into a weird problem I'm trying to debug now and will update once I figure it out.

jdhirst commented 2 years ago

So I changed everything to use my new list, and it all seems to work correctly.

Still running into the issue where the process stops when leaving the system. The system is not being unloaded and I can't seem to find another check where we see if the system is empty.

I modified SystemManager so that we don't unload a system which has a busy structure:

    // if system and jumpmap are both empty, set activity time for unload timer
    if (SafeToUnload())
        if (m_activityTime == 0)
            if (m_clients.empty())
                if (m_jumpMap.empty())
                    m_activityTime = sEntityList.GetStamp() -50;
    ManipulateTimeData();
}

// checks for if it is safe to mark the system for unloading
bool SystemManager::SafeToUnload()
{
    for (auto cur: GetOperationalStatics()) {
        //If there are any ongoing operations by operational static structures, we don't want to unload the system until this is complete
        if (cur.second->IsPOSSE()) {
            if ((cur.second->GetPOSSE()->GetProcState() == EVEPOS::ProcState::Unanchoring) or 
            (cur.second->GetPOSSE()->GetProcState() == EVEPOS::ProcState::Anchoring) or
            (cur.second->GetPOSSE()->GetProcState() == EVEPOS::ProcState::Offlining) or
            (cur.second->GetPOSSE()->GetProcState() == EVEPOS::ProcState::Onlining)) {
                return false;
            }
        }
    }
    return true; //by default, its always safe to unload
}

However, since the system is not actually unloading, something else must be causing the processing to stop.

zhyrohaad commented 2 years ago

However, since the system is not actually unloading, something else must be causing the processing to stop.

until i get set back up and walk thru it, i cant offer any suggestions/advise on that

jdhirst commented 2 years ago

until i get set back up and walk thru it, i cant offer any suggestions/advise on that

No worries, I'm also posting here to keep my thoughts organised and to document the investigation process 😀

zhyrohaad commented 2 years ago

...posting here to ... document the investigation process 😀

that's why i've been replying here instead of discord. ;) lil bit of insider knowledge can be helpful to someone eventually. lol

jdhirst commented 2 years ago

that's why i've been replying here instead of discord. ;) lil bit of insider knowledge can be helpful to someone eventually. lol

Yep, absolutely!

jdhirst commented 2 years ago

I can finally close this as completed, huzzah!