[BUG-8658] Since last weeks group chat fixes, about 20% of all sent chat posts in all group chats never post at all. Problem is not group specific.

[sl-service-account]

Steps to Reproduce

Attempting to communicate in a group chat

Actual Behavior

Posting messages in group chats fail

Expected Behavior

I expected the chat to go through or get a failed to post notice

Other information

The most recent change to the group chat code has resulted in post failing to go through far more often with no notice of the failure. this happens repeatedly, you may get a couple of messages through and then nothing for another few minutes. In the case of our support groups it has made helping users very frustrating for the people doing support as well as the users. This also happens using the LL standard viewer. EDIT I had one occasions in Firestorm Support English where I saw a post from a team member on one account, but not on another (accounts logged in on separate computers.) "[14:57:04] Amaya Summers: (FS64 4.7.0f os) Spock is your computer a 64 bit or a 32 bit OS ?"

Links

Duplicates

• [ ] https://github.com/secondlife/jira-archive/issues/16274
• [ ] https://github.com/secondlife/jira-archive/issues/16285

Original Jira Fields

| Field | Value | | ------------- | ------------- | | Issue | BUG-8658 | | Summary | Since last weeks group chat fixes, about 20% of all sent chat posts in all group chats never post at all. Problem is not group specific. | | Type | Bug | | Priority | Unset | | Status | Closed | | Resolution | Duplicate | | Reporter | Ed Merryman (ed.merryman) | | Created at | 2015-03-01T21:23:09Z | | Updated at | 2015-03-05T19:49:43Z | ``` { 'Business Unit': ['Platform'], 'Date of First Response': '2015-03-01T15:38:09.426-0600', "Is there anything you'd like to add?": 'The most recent change to the group chat code has resulted in post failing to go through far more often with no notice of the failure. this happens repeatedly, you may get a couple of messages through and then nothing for another few minutes. In the case of our support groups it has made helping users very frustrating for the people doing support as well as the users. This also happens using the LL standard viewer.', 'Severity': 'Unset', 'System': 'SL Simulator', 'Target Viewer Version': 'viewer-development', 'What just happened?': 'Posting messages in group chats fail ', 'What were you doing when it happened?': 'Attempting to communicate in a group chat', 'What were you expecting to happen instead?': 'I expected the chat to go through or get a failed to post notice', 'Where': 'Location does not matter.', } ```

[sl-service-account]

Mister Acacia commented at 2015-03-01T21:38:09Z

I have observed that the queue appears to be processed for a minute or two, then stops for two or three minutes. There are no errors, no warning that my message is still being processed, the messages are simply dropped.

[sl-service-account]

Paige Theseus commented at 2015-03-01T21:39:03Z, updated at 2015-03-01T21:39:34Z

Today was especially difficult. I have begun numbering my re-attempts, because when the post is dropped (with no system notice as we used to receive), it does not post later, which was the old chat lag behavior. My high number so far today was a successful post on the 7th repost (meaning the original and the first six retries were dropped). On nearly every post, I have to make a retry today, and the average for me is that I succeed at posting on the third retry (meaning the fourth overall attempt).

Chat lag was virtually eliminated several times since the work began, but as it is now, it was better before it was ever changed at all.

[sl-service-account]

Miro Collas commented at 2015-03-01T21:42:55Z

Things were pretty much going splendidly for quite some time. Yes, we had times when chat was totally "broken", but when it worked, there was no lag, no dropped messages.

Now ... well, what's been said above. Thing is, you aren't told if a message has been dropped. So, you wait a bit, repost, wait a bit .... frustrating.

[sl-service-account]

Whirly Fizzle commented at 2015-03-01T21:48:54Z, updated at 2015-03-01T21:56:09Z

Confirmed.

This is affecting all group chats and it is persistant and it is new behaviour. There are no times now when the number of dropped group chat messages is better or worse - the problem is persiatntly there in all group chats.

If you do not see your group chat message post within a couple of seconds, it will never post. It is behaving as if the timeout to discard lagged chat posts has been set to only a few seconds. (I thought it was supposed to be 2 minutes?).

I'd say the 20% in the issue title is about right - roughly 1 in 5 posts to group chat will never be posted and silently fail. This is a much higher failure rate then in the past.

[sl-service-account]

Soho SpiritWeaver commented at 2015-03-01T21:56:39Z

I have had great difficulties with a number of my groups. Several of us try to start a group chat and it never initializes. This is new behavior in the past week or two.

[sl-service-account]

Robert0 Siamendes commented at 2015-03-01T21:59:22Z

I am now getting 90% of my posts being dropped. Once in a while I will get the message dropped message but very rare. Please get this fixed.

[sl-service-account]

Minuet Voir commented at 2015-03-01T22:00:53Z

For the past few days especially, I consider it practically a miracle if any of my posts make it through. Firestorm Support English, Firestorm Beta Testers, KittyCats Addicts, for example.

[sl-service-account]

Marianne McCann commented at 2015-03-01T22:03:44Z, updated at 2015-03-01T22:10:24Z

Been seeing this a lot in The Arcade - Gacha Events trade chat today (opening day and a group with nearly 15,000 members). About 1 in 4 sends of mine actually arrive. The others vanish with no notice.

[sl-service-account]

leelee78 commented at 2015-03-01T22:08:33Z

Absolutely infuriating when trying to help someone and the chat won't go through. Right now 2 groups I'm in have broken chat. Whatever LL "fixed" broke it worse.

[sl-service-account]

sexy.jinn commented at 2015-03-01T22:20:59Z

this is happening alot for me

[sl-service-account]

Amaya Summers commented at 2015-03-01T22:21:04Z, updated at 2015-03-01T22:36:21Z

I am also seeing the same that has been described. I also ran into a total group failure with this group > 2ca73182-eed2-4881-9e61-9af595e44b4e it opens but I sent a message and nothing happened. no warnings or messages of any kind. March 1, @ [14:01] there abouts.

[sl-service-account]

AleiaSerenity Sapphire commented at 2015-03-01T22:25:06Z

In one of the larger groups, we found that turning off group chat for everyone except for a few roles that has group chat and anyone who was still in said group when it was turned off, worked flawlessly. This is really frustrating when you a mod in a very very busy group and you can't do anything in said group since the messages gets dropped more than it goes through. At this point it's easier to just message someone who needs the help or has a question about something instead of trying to answer and help in group chat.

[sl-service-account]

Astrid Veriander commented at 2015-03-01T22:58:37Z

Group chat for the SL Frees & Offers group with over 47,000 general members and +250 merchants broke down starting Tuesday 2/24. We are experiencing approximately 90% of chat messages dropped by the server, without warning. This is affecting all group members and is highly frustrating. We are used to network latency and general lag issues, but the recent server side changes have broken chat completely for large groups.

[sl-service-account]

Mister Acacia commented at 2015-03-01T23:11:43Z

Just to dissuade any thoughts that this only affects one viewer. I am experiencing the sane kind of posting failure in the SL Viewer

Second Life 3.7.24 (297623) Dec 19 2014 14:34:39 (Second Life Release) Release Notes

You are at REDACTED Second Life Server 15.01.23.298475 Retrieving...

CPU: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (3000 MHz) Memory: 11999 MB OS Version: Linux 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 Graphics Card Vendor: NVIDIA Corporation Graphics Card: GeForce GT 530/PCIe/SSE2

OpenGL Version: 4.4.0 NVIDIA 331.113

libcurl Version: libcurl/7.38.0 OpenSSL/1.0.1h zlib/1.2.8 J2C Decoder Version: KDU v6.4.1 Audio Driver Version: FMOD Ex 4.44.31 Qt Webkit Version: 4.7.1 (version number hard-coded) Voice Server Version: Not Connected Built with GCC version 40603 Packets Lost: 12/9025 (0.1%)

[sl-service-account]

RadonStarblade commented at 2015-03-01T23:25:32Z

A lot of messages are being dropped, it is a real pain to have to send im's for everything.

[sl-service-account]

Anastasia Horngold commented at 2015-03-01T23:33:35Z

This "new" chat lag started sometime in the last week or so, after many weeks of vastly improved group chat performance. The new version of lag is very unpredictable. Posts will sometimes go through instantaneously for a few minutes. Suddenly they stop going through at all. I sometimes repost the same line numerous times, and as Paige mentioned above, I've started numbering my reposts. I rarely get any error message about the post being delayed. (I have gotten one such error message since this new problem began.) After some minutes, an avalanche of chat will sometimes appear, but not all; what appears has no apparent rhyme or reason. Then posts will go through immediately for a few minutes; then chatlag will start again. This is making it virtually impossible to have any kind of meaningful conversation. The situation is actually worse than the "old" chat lag, where chat was often just tortuously slow, but messages tended to be delivered (eventually) much more often than now.

[sl-service-account]

OMAG commented at 2015-03-02T00:16:20Z

The bigger or busier the Group, the higher the rate of failure to post, regardless of the Viewer (LL, Firestorm, Radegast), between 90% and 10%... when you can actually log into the Group Chat

[sl-service-account]

Trina Stromfield commented at 2015-03-02T01:13:35Z

A lot of us are experiencing the same thing, my chat hardly ever goes through and im tired of it...

[sl-service-account]

Billie Marchant commented at 2015-03-02T01:21:00Z, updated at 2015-03-02T01:21:34Z

I have been noticing over the past few weeks, more and more messages not even showing up in some group chats, not even dropped or any warning, just never showing. This is an ongoing bug with group chats going back years now and still we are told that its improved!

[sl-service-account]

Flossy Andrew commented at 2015-03-02T01:42:14Z

I have had several messages dropped in group chat. No message that it was dropped, nothing. They just didn't go through.

[sl-service-account]

Leslea Aldrin commented at 2015-03-02T04:31:45Z

For the past several days, messages have been dropped suddenly(in various chats), and it appears that the chat has totally stopped- no one else posts in a chat that normally is relatively continuous. There is no warning, no comment, and no delayed posting.

[sl-service-account]

Antoinette Lioncourt commented at 2015-03-02T07:17:00Z

All i can add is to agree with what's been said in the above comments, it's an ongoing issues that seems to have really hit a breaking point since Tuesday Feb 24. However i do seem to be noticing a steady increase in occurrences of dropped messages since then, and i'm tending not to post in affected groups as it's frustrating, and quite often pointless trying to send the same post numerous times in the hope that ONE gets thru.

[sl-service-account]

narisa commented at 2015-03-02T08:50:54Z

Hope its fixed very soon, its spoiling my enjoyment of being on SL and not being able to MOD group too

[sl-service-account]

Ashira Torok commented at 2015-03-02T09:02:46Z, updated at 2015-03-02T09:06:31Z

I dont know what they did whit that update but it broken the groupchat We experience the same issues as mentioned by others The group is WoW skins and we have over 37600 members this is frustrating

[sl-service-account]

Ellen Sideways commented at 2015-03-02T13:26:06Z

please help us, Lindens, this is the way we communicate. and SL is all about communicating!

[sl-service-account]

Nisa Maverick commented at 2015-03-02T14:38:50Z, updated at 2015-03-02T14:39:14Z

I can only confirm and agree with all the comments made above. It seems to be that the chat lag is back more or less to square 1. as if no improvements have been made at all.

I experienced virtually no chat lag since the initiall improvements were made with the exception now and then group chat would "lock up" and fail for a while. These latest improvements have made it worse than ever especially as no message is received and like most have to repeat my message 3 or 4 times before it actually appears.

It is very frustrating.

[sl-service-account]

Paige Theseus commented at 2015-03-02T16:37:21Z

It seems that someone changed something this morning. I have not had any group chat post fail or even delay. It's working very well since I logged in shortly before 7:00AM SLT. I would describe the difference as "night and day."

[sl-service-account]

Alexa Linden commented at 2015-03-02T17:05:21Z

Thanks everyone. I've passed this along to the team.

[sl-service-account]

Anastasia Horngold commented at 2015-03-02T18:04:29Z, updated at 2015-03-02T18:35:46Z

It's MUCH better this morning (from about 7 to 10 SLT, so far), although I have had 2 posts dropped and have seen a couple of delays of a few seconds. Still vastly better than yesterday.

Update 1/2 hour later: 3 posts dropped in a row, with no error message.

[sl-service-account]

Blurrific commented at 2015-03-02T19:16:59Z

The groups started acting up around Thursday last week. Throughout the entire weekend: Groups either wouldn't allow me to post, or wouldn't open correctly. Me and my friends were experiencing the same issues. Not sure what's causing the issue? Maybe heavy amounts of traffic on the server? I don't know. But I know before we were getting alerts if messages were failing to send, and now we're not even getting those. I really hope this is fixed soon. Me and my friends don't normally use nearby chat to socialize, so groups are needed for our communication.

[sl-service-account]

Doktor Seuss commented at 2015-03-02T19:22:00Z, updated at 2015-03-02T19:23:32Z

regardless of group, though I have noticed a couple of groups, the larger ones I am interested in, seem to be hit hardest with lost messages.

That being said I have also had problems with my own very small group with 3 people logged in out of 12 total and each of us could only talk to one other person and none of us could talk so that all three saw the messages.

Even an error would be better than wondering.

I have also noticed that sometimes, maybe once in ten attempts the message you send eventually makes it to everyone multiple times, as in 5-10 times so fast it is freaky. (on the small group we use for our land mentioned above)

[sl-service-account]

Paige Theseus commented at 2015-03-02T20:26:43Z

The issue has returned, full force, now. Nearly every post must be re-attempted manually multiple times before it will show in a group chat.

[sl-service-account]

imawetodd commented at 2015-03-02T20:34:16Z

Severe and unpredictable group chat lag,sometimes stalling out groups completely for upwards of 20 mins making it impossible to give viewer support resulting in irate and disappointed users. Multiple re posts doesn't help and there is no message or indication of the posts being dropped,it's pretty much hit or miss, with the greater percentage being miss.It seems to affect large and small groups seemingly the same as if all chat is being throttled at intervals.Very disruptive in both viewer support as well as the general enjoyment of residents.

[sl-service-account]

AcaciaCreationsSupport commented at 2015-03-02T20:50:11Z

I'm seeing some delivery issues. I have three accounts logged in (two different PCs, three different viewers, three different regions) to test this issue, Avatar A would post and only Avatar B would see it. Someone else would post and only Avatar C would see it.

[sl-service-account]

AliKat Steampunk commented at 2015-03-02T22:21:14Z

This says 20% but for me, no matter where I'm at in SL, it's been closer to 90% of messages aren't posting.. It doesn't matter which computer or device or viewer I use either..

[sl-service-account]

ElinCD commented at 2015-03-03T00:53:14Z

Confirmed, and the number of members in a group doesn't seem to be a factor as I have those issues just as much in the land group for my private home with only 5 members as in some 1000+ members groups I'm in.

[sl-service-account]

Leighlee Trafalgar commented at 2015-03-03T18:47:51Z

I have been unable to chat in groups like SL Frees and Offers nearly 100% of the time. Really disappointing when one of the best things about SL IMO is the chatting and it cant be done

Please fix SL once and for all you would think since the population is down about 42% that you could manage to get it together

FAIL

[sl-service-account]

Skylar Farshore commented at 2015-03-03T19:03:22Z

Been unable to send a message through for several days now. Some people seem to have no problems getting messages to go through, whereas others have been unable to get a post through for days at a time. Please fix this.

[sl-service-account]

Cetan Stoneshield commented at 2015-03-03T21:34:38Z

I have the same issue with the added problem of other people in chat seeing my posts without them showing up in MY chat box

[sl-service-account]

Ayesha Askham commented at 2015-03-03T22:47:18Z

Yep, this has been happening for me, posts just vanish into the aether with no message from the chatserver.

[sl-service-account]

KyleFlynn commented at 2015-03-03T23:30:57Z

It's as if, at times, whatever server(s) are processing group chat just take a break. ALL group chat just disappears at certain times.

It's my firm opinion that group chat is one of the most social and enjoyable aspects of Second Life. I'd rather more server resources be devoted to group chat than any other task, even avatar motion. I know that this is an enormously interactive system, but it's difficult for me to understand why simple Unicode text is so difficult to process when 3D graphics seem to be handled with relative ease.

On yet another note, how do you (Linden Labs) expect businesses to adopt this system, when their primary tool would most likely be group chat. I mean, open forum lecture halls are nice, but conference calls and group chat would be the bread-and-butter of what they'd be doing. The fact that you can't make this work flawlessly is truly mind-blowing to me.

Regards

[sl-service-account]

Irish Breen commented at 2015-03-04T00:00:57Z

Seems to me as if this has been happening for over a year now. Its almost as if the more we get updates, the worst it it. Please LL fix this. Or Phoenix Team or SLgo, someone , please just make it work right, its messing with everything!

[sl-service-account]

Anastasia Horngold commented at 2015-03-04T00:11:22Z, updated at 2015-03-04T00:11:37Z

The group chat lag that appeared this past week is in full force this afternoon, 2-4 pm SLT. Posts have to be re-entered multiple times, and still often never show. There are no error messages.

[sl-service-account]

CaitnaBrionnae commented at 2015-03-04T00:18:59Z

Chat has been VERY laggy, but by widely varying amounts and with no apparent relation to general lag issues as has been the case other times. It's also not notifying on message fail, and the chat lag varies majorly, from a few seconds, to 3 to 4 MINUTES. This is a major problem and needs addressed ASAP.

[sl-service-account]

Ashira Torok commented at 2015-03-04T01:08:09Z

Its still going We are not able to send messeges in WoW Groupchat none of my messeges are showing We need a solution

[sl-service-account]

Ayesha Askham commented at 2015-03-04T15:08:12Z

I'm simply going to say that this must be the single worst gamebreaker for a couple of years Linden Lab. This needs to be repaired, or at least your latest patch backed out ASAP. At least before small groups were less affected. Now ALL groups are borked irrespective of size, and the total lack of chatserver response will unquestionably lead to more multiple posting and even MORE load.

[sl-service-account]

CountryCouture commented at 2015-03-04T15:11:35Z

Even my store group looses chat messages and we have only 200 or so members. It gets pretty sad when I have to post a comment 4-5 times!

[sl-service-account]

Antoinette Lioncourt commented at 2015-03-04T17:07:57Z

shall we cut to the chase? LL you take REAL money from REAL people for using a 'world' which has for it's mean reason for existence.....the means to communicate with one another..we do so, during these spells of transfering money to one another, we DO communicate. However once this communication is lost or taken away, we get frustrated and are wary of communicating with one another by way of trading, never mind the fact we need to be able to TALK/CHAT. You are, and have been WELL aware of this issue, and seem oblivious to the feelings of paying customers at ALL levels, by NOT showing a decent response in some message to us....or even fixing it. People have tried contacting 'help' and may as well have been talking to some other organisation...can we see some form of response?...even if it's an acknowledgement of this ongoing issue?

[sl-service-account]

Whirly Fizzle commented at 2015-03-05T00:43:38Z

This problem was discussed at Tuesdays Simulator user group

Commenting on the problem at the Simulator User Group meeting on Tuesday, March 3rd, Simon Linden summarised the situation thus:

In short, yes, it’s cranky, and yes, we’re (as in I am) looking at it … the chat server itself is actually running better than before, believe it or not. A back-end service it relies on, what we call “agent presence” [used to help locate someone on the grid], seems to be having new problems, so the changes may have added load to those servers and is causing problems, or something else unexpectedly changed … [So] some people don’t get the messages when chat is failing … it’s dropping sending some updates and messages when it times out with some other internal requests.

Ref: https://modemworld.wordpress.com/2015/03/04/sl-project-updates-week-101-server-general-news/

[sl-service-account]

Astrid Veriander commented at 2015-03-05T01:41:20Z

Thanks for that info. Well if "cranky" means having the vast majority of chat messages fail, I'd hate to see what angry looks like. I think I preferred it when it ran worse than now, meaning when it sort of worked with acceptable latency for the majority of users most of the time. In any case, we haven't seen any improvement since the issue started or the opening of this jira. It takes upwards of 10 attempts to get a message relayed by the server and even then they are not being transmitted to all chat participants. Would be nice too if LL changed the status of this jira and responded instead of making an offhand comments at a user group meeting.