search

Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
195 stars 9 forks source link

Stuck Gather Jobs for Certain Users #2062

Open ProbablePrime opened 3 years ago

ProbablePrime commented 3 years ago

Describe the bug

Certain users:

Are experiencing Gather Job / Asset Loading, failing after long play sessions within Neos. At this point nothing new in the world will load:

Users joining their worlds will also not be loaded into their cloud avatars on joining.

Relevant issues

To Reproduce

I'm sorry I don't really know how to provide reliable reproductions here.

Erin, did say that the more they load the faster it tends to happen. Spawning in large objects or videos might exacerbate it. It doesn't matter if they are hosting or not either.

Expected behaviour

The users affected, would love to be able to continue playing without having all of these issues. Right now they have to restart roughly every 4-6 hours.

Screenshots / Video

image

Bug information (please complete the following information):

Additional context

All of these users use Spectrum for their internet provider. Erin has attempted to correct the issue by changing their DNS settings which did seem to improve things for awhile but the issue still happens just a little less likely within the session.

Here's a log: DESKTOP-6QDVSG7_-2021.4.17.185-_2021-04-20_20_11_32 (1).log

I took a look at various items in the log and I see some unusual messages but nothing that immediately jumps out.

Erin also has looked at their Gather Jobs Debug window and they see it full to the brim of Queued gather jobs. none of them seem to be starting. They all just look stuck.

I'm making this issue on behalf of the users, as i've been in the worlds where this has happened and experienced the frustrations of those users.

I apologize, for making such an ambiguous issue but I feel it deserves to be looked at even if it is closed later. Thank you.

Reporters:

I'm Erin. Kitten_of_Codes

Frooxius commented 3 years ago

Hmm that's a bit odd. Could you get me screenshot of what the top of the gather job dialog looks like? Particularly the gather jobs in bold, which are the current ones.

With the current one it's hard to say what's blocking the queue, because the ones below are simply waiting in the queue, but I'd need to know which ones is the system trying to handle at the moment.

I'll see if I can find more from the log, but having this info could help, I don't see much in there at first glance.

ProbablePrime commented 3 years ago

image - Here's the top of another occurrence

Additionally heres another log and a text based dump of the Gather Jobs window. Blobcore (2).txt DESKTOP-6QDVSG7_-2021.2.28.1210-_2021-03-03_15_30_26.log

ProbablePrime commented 3 years ago

I asked Erin to keep an eye on this with the new build re-writing the Local Database systems I thought this might have changed or improved a bit but its still happening for them.

Here's a new log and gather jobs dump: DESKTOP-6QDVSG7_-2021.4.24.849-_2021-04-24_15_43_33.log Content.txt

Frooxius commented 3 years ago

I did some changes in 2021.5.24.1203 which should help reduce the thumbnail gather jobs spam a bit. Let me know if it helps in this case!

Frooxius commented 3 years ago

Hmm I'm looking into this some more, but it's a bit confusing. It seems that some of those stuck gather jobs are just waiting for the cloud variant to be generated so they're being retried. How does this behave exactly? Are no jobs progressing at all? Or is it just slowing down the progression significantly?

From the logs it doesn't seem like they're really "stuck", they're just being retried a lot.

If this happens again, is it possible to observe the gather jobs for a minute or a few and record a video? I'll also add some extra logging in the next build to help with this.

ProbablePrime commented 3 years ago

How does this behave exactly? Are no jobs progressing at all? No Jobs appear to process, Erin is stuck seeing no models or assets load. Users with saved(favorited) cloud avatars will load in invisible. Models will be invisible.

On rare occasions some assets will recover but never all of them.

In some cases guests of the world will take photos with the Neos Camera of items that are being discussed as taken photos will Load for them.

I'll remind them to get new logs :)

ProbablePrime commented 3 years ago

DESKTOP-6QDVSG7 - 2021.5.25.1225 - 2021-05-27 18_30_00.log

Here's a new Logfile, there hasn't been much change in this issue. It might take longer to occur now but it certainly does still occur.

seif1 commented 3 years ago

DESKTOP-HRE9Q8J_-2021.6.9.396-_2021-06-09_01_21_48.log

I believe I have also encountered this issue, log file attached. Experience the same sort of thing in the debug menu, and assets not seeming to load, seems to occur pretty rarely, and only when joining a world in my experience.

Frooxius commented 3 years ago

@seif1 Do you have a screenshot of what GatherJobs looked like for you when that happened? What exactly was going prior to this in Neos? Were you loading worlds and so on? It's a little bit harder to analyze what has been going on without context.

It seems like there's huge amounts of jobs that have started in parallel, which is a bit strange, but I'm not too sure what was happening in Neos during the time.

seif1 commented 3 years ago

So, I don't have a screenshot but the GatherJobs pannel looked similar to https://github.com/Neos-Metaverse/NeosPublic/issues/2062#issuecomment-823768578 this earlier comment by ProbablePrime, it seems to occur when joining a world in my experience, and I don't think I have seen it at any other time.

(from the log file I provided, that was just after starting to join a world, it appeared things were not loading, so I restarted my neos, and that makes the problem not occur for a while in my experience, usually I'm not loading a lot of worlds though, and mainly just joining 1 which I stay in for a long period, only really switching when the previous session is closing, and usually only have the world I currently have focused, and my cloud home open at any given time)

I also did close my game pretty quickly after noticing that things didn't appear to be loading, so I may be mistaken about this specific instance being part of the issue, though I have experienced it before and stayed for much longer without things properly loading, though I do not have the logs from that time, I can say that the Gather Jobs panel looked similar to the linked comment.