Closed tflo closed 1 year ago
BagSync doesn't actually store the full item information (including) the name of items. Instead what is stored is the ItemID. The reason for this is to make sure the information is accurate. Say Blizzard decides to change the name or specs of an Item. If I were to store that information, it would be inaccurate and would have to be updated. That would take extra time to do as I would not only grab the information from the server but also parse the incorrect data.
Instead what BagSync and many other addons do is pull the information from the server using the ItemID. This takes time and the more items there are the longer it takes. The server doesn't always send back every item that is requested, hence the "warning info box" about the server/cache to respond. It's literally waiting for the server to send back the information to BagSync. The number of items should decrease and not increase or reset. That is wrong and I'll need to investigate that.
Let me run a few checks and make sure that the system is working properly. Unless of course Blizzard changed something on their end. The BattlePet should have been listed when you searched for it.
The number of items should decrease and not increase or reset.
Just checked it again on another char: Started with around 4800 "warning" items, after a couple of Refreshs down to 1799, then, with the next Refreshs, 2809, 1802, 2682, 1802, 2675, 1791. Always bouncing between roughly 1800 and 2700.
Here is my database file, in case it helps. It's from several sessions after my post, but the Spinemaw Gormling is still not found:
The number of items should decrease and not increase or reset.
Just checked it again on another char: Started with around 4800 "warning" items, after a couple of Refreshs down to 1799, then, with the next Refreshs, 2809, 1802, 2682, 1802, 2675, 1791. Always bouncing between roughly 1800 and 2700.
Here is my database file, in case it helps. It's from several sessions after my post, but the Spinemaw Gormling is still not found:
I'll take a look. I have a feeling I know what is going on, but I need to double check.
I found the issue, it is related to how the BattlePet names are being treated. I'm putting in checks to ensure the name does get parsed.
Damn pets 🫤
My Spinemaw Gormling is happy now. Thanks!
The "not searched" number still doesn't go to zero and bounces (this time between 1220 and 2046). But maybe this is irrelevant, as long as stuff is found.
Lets try something. Go and login to each and every one of your characters on every realm. Make sure ro look in banks, guild banks, auctions, and mailbox for each character.
Then try doing a search on your primary character. If it still does it, then tell me the search term you are using and on what character. I would like to squash this issue with the refresh.
Sent with Spark On Feb 23, 2023, 10:04 AM -0500, Tom @.***>, wrote:
My Spinemaw Gormling is happy now. Thanks! The "not searched" number still doesn't go to zero and bounces (this time between 1220 and 2046). But maybe this is irrelevant, as long as stuff is found. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>
Go and login to each and every one of your characters on every realm
Will do that over the next few days (~20 chars).
I guess guild bank is enough to check on one toon for each guild, right? Or should I do that too on each?
Void Storage too, I guess? You didn't list it, but there are many pets (in "installer" form) in there.
You should only need to scan each unique or different Guild Bank once. Meaning a character is in the same guild, then you don't need to scan it. Only each different Guild and Guild Bank.
Void Storage is different for each character so it needs to be scanned yes. >.> I didn't realize you can put Pets into the Void Storage. That is an issue I need to address. I thought only armor and weapons can go in there. Otherwise I'm not sure what you mean by "installer" form, unless you mean Caged?
Also great googly moogly, according to BagSync Debug you have over 8000+ items across all your characters. 😱 😨 That's a lot LOL.....
I didn't realize you can put Pets into the Void Storage
Not pets themselves, but the "installers", if they are 1) soulbound and 2) not unique. Like these, for example:
you have over 8000+ items across all your characters
Yeah, the guys are such hoarders, can't do anything.
Otherwise I'm not sure what you mean by "installer" form, unless you mean Caged?
Oops, I missed that part. But I think the screenshot explains it. The item category is Miscellaneous > Companion Pets, as opposed to Battle Pets (I think), which are the caged ones. You can't put the caged ones in the Void Storage.
I didn't realize you can put Pets into the Void Storage
Not pets themselves, but the "installers", if they are 1) soulbound and 2) not unique. Like these, for example:
Oh those! Yes believe it or not those are actually treated as regular items and not BattlePets. They actually have an ItemID and everything unlike BattlePets. They are not Caged Pets actually, it's an item that just adds to the pet to your Pet Journal. That's it really. Caged Pets are pets that you already have and are transformed to that generic [Pet Cage] itemID. So it would make sense that you can put the ones that are companion pets with an itemid in the Void Storage.
Try this version of BagSync that I just wrote up. I haven't pushed it live yet as I'd like you to test it. It should help with the Server query and responses when having a crazy (ludicrous) amount of items to go through. It may take a few refreshes but eventually should bottom out at zero left and the warning should disappear when doing a search.
OK, installed it, thanks.
I will tell my guys that you called them crazy 😝
Did a search for the good ol' Spinemaw Gormling with the test version:
It showed 5000+ "not searched" items. 1st click on Refresh: 1427, 2nd click on Refresh: nothing! (aka zero)
Did a miscellaneous other search: No "not searched" warning at all right from the start!
Looks like you solved the issue 😁
On a different toon: 6010 --> 1421 --> 0
Back to the first test toon: Same behavior as before.
So, to sum it up, I get 5000+ unsearched items on the first search after login, which can be brought down to zero (aka no warning) with two Refreshes.
And a search without a "not searched" warning is something I never had before! Go figure, I considered it normal, until I noticed that BS does not find things.
I guess this fix has the potential to also fix the mail issue. BS often was not seeing items in the mail box, where the other addons (Armory, Stash, TSM) had no issues (or rarely issues). I wanted to post that issue next, but I think it is a good idea to verify again with the new version.
Nope, doesn't seem so. Just mailed a pet to an alt and did a BS search immediately after: Mail location is not listed in the BS tooltip (only occurrences in bank, auction, bags, guild bank).
I will open an issue for that later.
PS: In this case, Armory and TSM are failing too. Only Stash sees the pet in the mail location.
Nope, doesn't seem so. Just mailed a pet to an alt and did a BS search immediately after: Mail location is not listed in the BS tooltip (only occurrences in bank, auction, bags, guild bank).
I will open an issue for that later.
PS: In this case, Armory and TSM are failing too. Only Stash sees the pet in the mail location.
You have to physically check the mail with the character receiving it for it to show up in BagSync. Meaning that BagSync doesn't preemptively add the item to outgoing recipient in the Database. The only time the Mailbox ever gets scanned is when you physically check it with a character. If you were to login to the recipients character and check the mailbox it would then show up. Many addon authors including myself don't like doing prediction stuff. For it to work the way you suggested, I'd have to hook into the "MAIL_SENT". Then do loop through ALL the characters on ALL the realms and match the recipient and realm. Then go to their MailBox DB (if they even have one) and add it manually into their DB list.
I'd have to check, but I believe that the MAIL_SENT doesn't send complete information other then a flag that mail was actually sent.
Meaning that BagSync doesn't preemptively add the item to outgoing recipient in the Database.
This is unfortunate, because it is valuable information.
Many addon authors including myself don't like doing prediction stuff.
Technically you are certainly right, though I wouldn't call it 'prediction' in this case, because the probability that the sent item will be in the recipient's mailbox is almost 100%.
Actually this means that BS is treating a mailed item equally to a destroyed/vendored/consumed item (i.e., removed from the DB). From a user's point of view, this is definitely suboptimal, simply because the item still exists (with almost 100% probability).
Just checked it with different non-pet items. Armory, TSM, Stash all show them as in the recipient's mail box, immediately after sending and without switching to the recipient char. So, I guess, the mentioned failure of Armory and TSM in my previous post was probably due to the "irregular" nature of the item (pet cage).
But good to know that this is the intentional behavior of BS, no point in opening an issue then.
After all, there must be a reason why I'm using different inventory addons side by side 😌
Back to the previous issue of this thread:
I guess I can stop my visit-all-storage-locations-on-all-toons action now, right? Seems you fixed the "not searched items" warning with the posted version of BS. (See this post and the following, in case you missed them.) I tried a couple of more times in the meantime, and the warning reliably goes away after 2 Refreshes, always.
I guess I can stop my visit-all-storage-locations-on-all-toons action now, right? Seems you fixed the "not searched items" warning with the posted version of BS. (See this post and the following, in case you missed them.) I tried a couple of more times in the meantime, and the warning reliably goes away after 2 Refreshes, always.
Not necessarily as I want to ensure that there aren't any issues with the DB. I had made some changes previously to BattlePets and want to make sure it's being treated properly on your end.
Also it's not 100% if there is a chance that a person is sending the item to another user/player unrelated to their characters. To be fair you did say "almost" but 100% implies that it's a definite thing, lol in that scenario it would be like 99%. Therefore you'd have to check to see if the recipient is actually in the DB. In any case I'm reviewing the MAIL_SENT and MAIL_SUCCESS events to make sure that Blizzard didn't change anything that I'm not aware of. If they do send the recipient as part of the event or at least allows a search of send items before transmission, then technically it is possible. It's been a very long time since I review the mail functions and I don't recall them making any significant changes to it, but it's possible.
If there are changes, I don't see an issue with adding the SendMail stuff to BagSync. I'll keep you posted.
Not necessarily as I want to ensure that there aren't any issues with the DB
OK, going on with it then (I'm already done with ~70% of the toons).
Also it's not 100% probability if there is a chance that a person is sending the item to another user/player unrelated to their characters.
True, sorry I was speaking only of mails to alts without saying it. Indeed, mailing an item to a non-account char (other player) actually removes it from the account, so it is correct to remove it also from the DB in this case.
Hmm it appears that you can now GetItemLinks before mail is actually sent and get a success response. They must have changed this as before you weren't able to do this. This means technically that I can actually store the information on the recipients DB entry. I'm going to start working on the code for that.
LOL the code exploded in my face, but I think I'm on the right track. 😆
the code exploded in my face
Yeah, that's always a good sign 🙃
You could also check how the others are doing it. Here, for example, the mail part from Stash:
Oh I usually write it up and it's not that different from the Mail Scan code that I wrote up. I just didn't have access to grab the SendMail before a success command. The reason the code blew up was because it didn't like the BattlePet I was testing with LOL. Go figure, it's resolved now. So some progress.
I sent you the snippet from Stash in particular because the author is good with battle pets 😉 (it's by gello, the author of Rematch).
Try this version of BagSync. It should now properly track the items that you send to a character. Note: Obviously this will only work as long as there is a Character entry in the BagSync database. Meaning you had to at least logged in to that character once EVER so BagSync could have made an entry in the database for that character.
Meaning the following:
Otherwise if there is an entry in the DB for BagSync then it should work with no problems. BagSync.zip
Installed…
Seems to work fine, with simple stuff:
…and even with pets:
Damn, was this really necessary? What is now with my "After all, there must be a reason why I'm using different inventory addons side by side"?! 😖
Just joking. Congrats, the addon keeps getting better and better! And of course, thanks for fixing bugs so quickly and for responding to my suggestions!
Still working on visit-all-storage-locations-on-all-toons, should be ready tomorrow or maybe later this evening/night.
Have you noticed the missing stats and the "PetLevel 0" in pet tooltip above?
This seems to happen only with tooltips from the BS search frame. This is the correct tooltip, from the pet cage in the bags:
Or is this normal because the search doesn't differentiate between pet levels and variants/quality?
Have you noticed the missing stats and the "PetLevel 0" in pet tooltip above?
This seems to happen only with tooltips from the BS search frame. This is the correct tooltip, from the pet cage in the bags:
![]()
Actually that is intended. The reason for this is because BagSync and many other addons only store the SpeciesID. (I checked and Stash also just store the speciesID). The other data is hardcoded in the special BattlePet links that Blizzard generates on a server basis (which btw become invalid once a server reboots). These links contain some of the stats data required to properly display the BattlePet properly. The issue is that on MANY API code functions that Blizzard provide for us, the only option is to pass the SpeciesID or PetID. The PetID is a reference to the index in your Pet Journal where the stats are. That is how Blizzard grabs the stats information to display in the tooltips. The problem is the PetID is not very reliable as those indexes do change. So instead when you pass the SpeciesID it doesn't know which PetID is being referred to and generically just shows everything with 0 stats****. This is pretty normal for addon authors. Otherwise we'd have to store every variant and their stats and you can easily have 30 of the same SpeciesID with different levels and stats. It's just too much to have to deal with and at the end of the day using the SpeciesID is just easier. In all honesty working with BattlePets is a pain as Blizzard didn't really think things through or implementing in a way that is easier for addon authors to work with. There is is going to be some limitations.
It may be possible to store some of the stat link only for displaying in BagSync. But honestly I don't really know if it's really worth it. Since you really just want to see who has the BattlePet on what character and the locations it can be found.
Outside of BagSync though in the Blizzard UI the tooltip should work perfectly as it can communicate properly with the Blizzard servers for the information.
Armory actually shows every found instance, which allows to differentiate pet quality and level in the tooltip:
OK, I finished the visit-all-storage-locations-on-all-toons. I didn't visit the Void Bank on a few toons that don't have one yet. One char is fresh and has never visited his bank (nothing in the bank).
Warning behavior is still the same: Number starts high and goes away after two Refreshes.
Here the file after a reload on the last toon:
But honestly I don't really know if it's really worth it. Since you really just want to see who has the BattlePet on what character and the locations it can be found.
This isn't entirely true:
I sell mainly level 25 pets on the AH. When I have sold one, and there isn't another L25 in the AH toon's bank, and no spare L25 in the pet collection, I need to know if and where I have another L25.
If I only have the information that 5 alts have the pet in their bank, without level, then I would have to log in to each of these 5 alts to see if it is a L25 or a L1.
Actually the problem doesn't exist for me, but only because I know that I have most of the L25 on one or two specific toons (besides the AH toon). The other toons have exclusively L1 pets. And in case of doubt, I still can do a search in Armory.
So, just to demonstrate that the level info is in fact a very useful info.
But honestly I don't really know if it's really worth it. Since you really just want to see who has the BattlePet on what character and the locations it can be found.
This isn't entirely true:
I sell mainly level 25 pets on the AH. When I have sold one, and there isn't another L25 in the AH toon's bank, and no spare L25 in the pet collection, I need to know if and where I have another L25.
If I only have the information that 5 alts have the pet in their bank, without level, then I would have to log in to each of these 5 alts to see if it is a L25 or a L1.
Actually the problem doesn't exist for me, but only because I know that I have most of the L25 on one or two specific toons (besides the AH toon). The other toons have exclusively L1 pets. And in case of doubt, I still can do a search in Armory.
So, just to demonstrate that the level info is in fact a very useful info.
This is true, but I only show one result in the Search window per item as BagSync users get the info from the tooltip hover. How would I display 5 of the same pet in various levels when only one item is ever shown? I would have to make significant changes to BagSync to allow for something like this. I don't think I want to invest that much time and effort into making those changes. What I will do however is look into the petData stats, perhaps there is something there I can work with, but considering how some functions only return a SpeciesID it may be difficult. I'm not going to rule anything out and it's worth investigating, but I cannot promise you I'll implement something like this.
Armory actually shows every found instance, which allows to differentiate pet quality and level in the tooltip:
![]()
OK, I finished the visit-all-storage-locations-on-all-toons. I didn't visit the Void Bank on a few toons that don't have one yet. One char is fresh and has never visited his bank (nothing in the bank).
Warning behavior is still the same: Number starts high and goes away after two Refreshes.
Here the file after a reload on the last toon:
Thank you, I will take a look at the data and see if I notice any discrepancies. In terms of the refreshes, that will be normal as remember BagSync queries the server for all the item information at least once. Blizzard has a set limit of queries you can perform at a time. Hence why a few refreshes are required. It's only at Login and should persist in the cache for the remainder of the gaming session. Normally folks don't see that warning screen as they don't have 8000+ items. The average I've seen is around 2000 to 3000.
How would I display 5 of the same pet in various levels when only one item is ever shown?
Yes, see the screenshot of Armory. This is of course only possible if the search lists every found instance.
I don't think I want to invest that much time and effort into making those changes.
I did not ask that from you because, as said, for me personally this is not a problem because I happen to know which alts are holding the L25s. So, if I find a pet on that alt, it must be a L25. The example was just to show that the level info in search results makes absolutely sense. (And, to a minor degree, also the quality and breed info.)
So please don't bother with it, I imagine it's a lot of work!
Have you noticed the second part of my post with the new DB file after having finished visiting all locations on all toons?
Oh, missed your second post, so forget the last part of my post above.
Blizzard has a set limit of queries you can perform at a time. Hence why a few refreshes are required.
Ah, OK. This would have been my next question 😌
It's only at Login and should persist in the cache for the remainder of the gaming session.
Yep, can confirm this.
But, wouldn't it be possible to do the additional refreshes automatically when needed? Or do you need an hardware event for that?
But, wouldn't it be possible to do the additional refreshes automatically when needed? Or do you need an hardware event for that?
Unfortunately it's a built in restriction on the WOW client. There is no real way to trigger an automatic refresh. Though any new items would fall under the query limit if new ones were added. So technically a refresh from the server wouldn't be necessary after the initial cache is filled on Login. It's a lot of items for WOW server to parse.
No, what I meant is something like this:
Just a raw hack, but it seems to work fine. For obvious reasons I would throw in a counter to limit the automatic runs to 3 or so, in case something goes wrong.
But, of course, I can as well click the button 2 times myself per session, not a thing.
Ah yes I normally don't do stuff like that because I've had people yell and scream at me about "OMG BAGSYNC CAUSES LAG" or something like "SO MUCH LAG" on this or that... or something to that nature. Or zomg I have like 10,000 items and I have a tiny micro-second frame delay or lag because I'm searching through all my characters across ALL my BattleNet toons because I have the option enabled in BagSync (even though I put a warning about it). Yes this has happened before.
Your hack would technically work, but I would have to literally put an option for it to disable it as folks would complain regardless. I know I sound sarcastic or over the top here, but I've seen it all. 😆
The good news is I found a bug in regards to the BagSync External Tooltip while looking into the BattlePet data. So that's a nice find!
people yell and scream at me about "OMG BAGSYNC CAUSES LAG"
Well, those people should consider that the auto-refreshes (even with the C_Timer at 0.5 or 1s) will most certainly be faster then clicking the button twice manually 😉
And if they have only 3000 items, then the auto refresh will not run anyway, as it is tied to the countWarning > 0
.
people yell and scream at me about "OMG BAGSYNC CAUSES LAG"
Well, those people should consider that the auto-refreshes (even with the C_Timer at 0.5 or 1s) will most certainly be faster then clicking the button twice manually 😉
And if they have only 3000 items, then the auto refresh will not run anyway, as it is tied to the
countWarning > 0
.
Technically you are correct and it wouldn't affect some people, but the few that do... just you wait. I see a future ticket.
I also have a dumb or stupid idea. Perhaps I can make some sort of Ctrl_rightclick or shit_rightclick or some stupid button combination so that when you click on an item in the BagSync Search, it pops out a external tooltip with a breakdown of all the items and their locations and character stuff. It wouldn't require me to reprogram an entire search algorithm or window for it. That or some other weirdo way to do it. Eh I'll think about it.
There is a thing I don't understand:
Today I was searching for a pet and I tried BS's search. It didn't find anything:
Then I did the same search in Armory. Armory found the pet, and in the pet's tooltip BS is correctly listing the pet locations (2× guild, 1× AH; I verified it):
I guess that it didn't find anything has to do with the "A total of [n] items were not searched… still waiting…" warning, right?
But obviously BS knows about the location of the item (see tooltip count), so why does it not use that information for the search?
I'm probably failing badly in understanding how BS works, but why does it need to wait for the server to respond at all if the items are cached?
PS:
Clicking the Refresh button several times in intervals of 5 or 10s brings the "not searched" count in the message down, but never completely. When it has reached a certain minimum (e.g. 1576), it goes up again on the next refresh, then down to the minimum again, and up again, etc.