Teleport failures & HG Teleport Failures

[Ana-Green]

After testing it several times W/we noticed that local teleports do fail from time to time, it can not find the region while it is there, solution:: logoff and log back in again. Same for Hypergrid once HG teleported for example to OSGrid it seems impossible to get back, teleport back, it shows the region http://hg.dsgrid.nl:8002:Space Station but that's about it. Solution: logoff and log back in again, that will bring you if you are lucky back to the welcome area.

Regards, Ana Green

[NoahStarfinder]

Good morning,

We are aware of this issue relating to intermittent inability to be able to teleport to regions even though the grid believes the region is online. We are also aware of a few grids using the mainstream OpenSim having similar issues as well. We hope to have a fix for this shortly.

To set the record straight on what happens when you log off and back on:

When you log off all you are doing is ending one session of your signed in avatar (or agent in this case). By logging in again your only creating a new session telling the grid your online. This is not a real fix to the issue and in fact doesn't solve the problem with the grid not being able to locate the region correctly. The correct solution is to restart the region having issues, this will ensure the region reregisters with the grid and is seen by the grid.

This issue is related to a problem with the region presence and region persistence which we are already working on as part of our refactor process.

Best regards, Noah Starfinder Governing Body Vice Chair (North America) Core Developer In Galaxy Services and Features Development, Maintenance Second Galaxy Development Team

[Ana-Green]

Right, but my Dad (Frank) has his entire Grid up with Virtual Universe 2.0.1, I think that was not a smart move. Better a small test Grid and use something stable for his "production Grid" if we can call it that until Virtual Universe 2.0.1 has become a bit more stable, that is what I told Dad what I would do but if the DB tables are altered he has to start from the beginning.

Best regards, Ana Green

[emperorstarfinder]

I think its time to dispell some incorrect thinking here.

First, the idea that there will ever be a 100% stable grid architecture is like saying a pig can fly. The reality is, just like a pig cannot fly, so there is no such thing as a 100% stable grid. OpenSim (regardless of how good or bad it is and despite its many flaws which we could all debate) will never be a fully stable architecture because it is in what we call continuous development. It is also why they have not really ever left the Alpha status when it comes to the development stages. The same could even be said for Secondlife which while most of its backend is stable, it also has its instabilities as a grid architecture.

Second, As both Noah and I have explained to Frank (along with others), the Virtual-Universe-NXG repository is considered to be in the very early development process of the Second Galaxy Development Team. Our understanding is that people who use this repository for their grid do so at their own risk and understand that as we progress with its development, there likely will be many significant changes that may either improve on what OpenSim did, may move the repository away from being compatible with OpenSim, or may be similar to OpenSim in some cases. We are happy to see people using the repository to run grids and provide us information on issues they may find. While we recommend it be for testing purposes we cannot dictate how the end-user chooses to run their grid anymore then you can. If Frank chooses to run it as a production grid that is a choice he will have to make. Our team will always be there to support those who use our architecture which in many respects is much more then some of the other projects out there can say.

Third, Database Changes likely will happen over time as we do refactoring. However, when we make a change to the database the software will always have the ability to make those changes to the database on its own without the need for the end-user to actually have to alter their database tables manually. When it comes to virtual world grids, if you do not know what you are doing in the database tables, you should never alter them manually. This is because you could alter a table that is needed a certain way by the architecture and accidentally cause errors that are not real errors.

Fourth, We will never put something in our open source architecture that we will not put in our proprietary grid (Second Galaxy). Generally, when we commit something to the Virtual Universe repositories it means one of the following:

a) We have already deployed it on Second Galaxy and its stable; b) It has been deployed on our internal development grid and has been deemed stable and is pending deployment with the upcoming Monthly Server Deploy to Second Galaxy; or c) It has not yet been deployed but is waiting for deployment with accompanying features that are not yet ready.

It should be noted that there will always be some things that make it into Second Galaxy that are proprietary that will not be included in the Virtual Universe architecture for obvious reasons unless we decide to make available an open source version of that feature.

It should also be noted that it is entirely possible that we will have inherited from OpenSimulator a lot of issues which may still be occurring, but that we ourselves may not necessarily be aware of right away. In those cases, as we learn of those issues we will fix them. In some cases, some errors that we do not yet know about will likely get corrected or made moot during the refactoring process. Some of those issues may even be issues arising from earlier versions of OpenSimulator that were never actually fixed or have been given rebirth by means of some of the current development activity going on at OpenSim.

As to this issue, Noah is correct it is a known issue. In fact, OpenSim's developers have even acknowledged this issue via their mantis. We also know other grids (such as Metropolis Grid) which uses their own variation of the OpenSimulator architecture, has noticed similar problems. Unfortunately, this issue happens when developers do not fully test their commits thoroughly before they commit them to their repositories. There is sadly a lot of this going on in the OpenSimulator architecture which dates back several versions at the very minimum.

It, however, will help us determine if our suspicions of the cause are correct by setting the console log levels to all in both Robust and the region consoles via the command set log level all This way we can see what info it is printing along with any potential errors it might be passing.

@NoahStarfinder according to OpenSim Mantis: http://opensimulator.org/mantis/view.php?id=8382 the problem does not exist at least on Mono 4.6 in Linux but appears to be an issue in later versions of Mono. But Frank is running on Windows Server (I believe either 2008, 2012, or 2016) in x64 bit.

Best Regards, Emperor Starfinder CEO Founder Governing Body Chair Core Developer Work Group Leader Grid Architecture Development Work Group Grid Architecture Development, Maintenance Second Galaxy Development Team

[NoahStarfinder]

@emperorstarfinder A quick speed read through that issue finds some interesting information. I will have to do a thorough read of it in the morning. I also find it interesting (but not surprising) that folks are saying it works when compiled against Mono 4.6 in Linux but when compiled against later versions of Mono the problem appears. I will try to reproduce that myself with 0.9.0.1 and 0.8.2.1 and see if that proves to be the case just to cover our basis per our internal discussions. I will also spin up a Windows VM and test against .Net 4.5.2, 4.6, 4.6.1, and 4.7.2 and see if the same is apparent there. If so then it means something has changed and was a usual Microsoft change that has caused a headache.

Best regards, Noah Starfinder Governing Body Vice Chair (North America) Core Developer Work Group Member Grid Architecture Development Work Group In Galaxy Services and Features Development, Maintenance Second Galaxy Development Team

[Ana-Green]

@all I agree. that is me Ana, not Frank, to say its time to dispell some incorrect thinking in global. Why in the first place using Opensim Master 9.1 while OpenSim Master 8.2.1 is much more reliable then OpenSim Master 0.9.0, .01, .1 and it might be so that in this hobby world nothing is stable, you said "It should also be noted that it is entirely possible that we will have inherited from OpenSimulator a lot of issues which may still be occurring" that is absolutely a fact, while using as fundament OpenSim 0.9.x knowing its already not stable and slow and merge issues, yet it had not the issues and still does not have the issues what Virtual-Universe-NXG now has, on top while search starts to fail and Teleports start to fail what is not the case with OpenSim 0.9.1 Master so you did not inherit that from OpenSimulator 0.9.1 ... Then if you had used OpenSim Master 8.2.1 I think you would have a much better fundamental code base as now. Then if you have that knowledge that it is entirely possible that you have inherited from OpenSimulator issues why use it then? Why not start with a code from nothing make flow cards and work it out to a stable Simulator and a better Grid structure and forget OpenSim! My Dad pays 850,55 USD for 4 servers a month yes you read that right, and then to run something on there what basically never will work regardless how fast these servers are, why is PBX software then Stable? I work for NASA and yes they have 100% stable Simulator like the 661XPC range NASA scientists have reached a breakthrough in computer modeling that allows them to simulate what gravitational waves from merging black holes look like. The three-dimensional simulations, the largest astrophysical calculations ever performed on a NASA supercomputer, provide the foundation to explore the universe in an entirely new way. Navigation software that cannot be some "maybe working version" that's why I said incorrect thinking in global creation of stable software is key. I can go on but I know that that would make no sense, you said also "First, the idea that there will ever be a 100% stable grid architecture is like saying a pig can fly. The reality is, just like a pig cannot fly, so there is no such thing as a 100% stable grid. OpenSim (regardless of how good or bad it is and despite its many flaws which we could all debate) will never be a fully stable architecture" of course not, you need to hire professional programmers, and that are multiple program parts to create a stable 100% working Simulator.

And believe me nor NASA or CISCO takes a NO for an answer.

Best regards, Ana Green

[emperorstarfinder]

Let's keep in mind here, what is in the repository now is not what will be the final version of the next generation of Virtual Universe. Whether we use 8.2.1 or the 9.0.x series isn't something that is really all that relevant because we could go back and ultimately say that no 8.2.1 is not stable and we could go further back and say we may ultimately use a base from 0.6.9 and bring it forward.

As we refactor the code there will be many changes and a lot of what is in the codebase now may not even exist (at least in current form) when we complete the refactoring process. So let's not assume this will be the final iteration of the next generation of code. In fact, most people don't know unless we specifically say whats in our internal repositories what the latest code is. Our internal repository version will always be ahead of what is on GitHub itself.

When commenting on the issues being posted here let us keep the comments relevant to the issue and not post about the software or other companies that have no relevance to what we are doing here.

NASA and Cisco also have no relevance to what we are working on here.

Best Regards, Emperor Starfinder CEO Founder Governing Body Chair Core Developer Work Group Leader Grid Architecture Development Work Group Grid Architecture Development, Maintenance Second Galaxy Development Team

[Ana-Green]

NASA and Cisco also have no relevance to what we are working on here. You got that right and they have no relevance with OpenSimulators for a good reason.

Best regards, Ana Green

[Ana-Green]

Presence issue prevents teleport(s), Teleport failed, Failed to verify user Presence in the grid for Frank Orbis, Access Denied to region DSG into the Dark. While agent with UUID a69be955-d44c-440e-a31e-66349ee2f17 is Presence in region DSG Plaza Market Estate ID 100 is Presence. Action manual logs out and log back in will solve it for now as it will happen again.

Regards, Ana Green

[Ana-Green]

Extended - Presence issue prevents teleport(s), in the DB table Presence Robust, I found that agent with UUID a69be955-d44c-440e-a31e-66349ee2f17 has been removed/deleted what means that the agent is still online but not in the DB table Presence Robust present. That explains the message Teleport failed, Failed to verify user Presence in the grid for Frank Orbis, Access Denied to region DSG into the Dark. See part log related to this issue/bug. Region.log

[NoahStarfinder]

Good afternoon,

I think we are going to find here that all this is related to regions getting desynced from the grid. When you teleport to a region that the grid thinks is offline teleports will either fail or let you teleport to the region but then teleporting to another region will cause the teleport failure.

The way to fix this (temporarily until we can find a more permanent fix which we re working on) is to do as follows:

1. In the region console that runs the region that your teleporting from when the teleport failure occurs use the change region command to that region Example: change region DSG Plaza

2. Once you are in that regions instance do the restart command: restart

3. As the region restarts watch the grid server. If the restart is successful you should see DSG Plaza has successfully registered with the grid.

That will temporarily fix the issue and teleports to that region will work again as they normally should.

We do know as we indicated before that other grids (Metropolis Grid included) are having similar issues but they appear intermittently at best.

As soon as we feel comfortable with a potential fix we will push it to the repository but rest assured we are working on this. Additionally when this happens again please provide us with a copy of both the region and grid logs. I would recommend that you set your console log levels to all (Command is: set log level all). THis will give us both the information logging and the debug logging. This will also help us understand what the grid is thinking at the time.

In fact I recommend for the time being to always have the log levels set to all while we are in the process of working through all of the various issues (both new and inherited) as we work to craft the code.

Best regards, Noah Starfinder Governing Body Vice Chair (North America) Core Developer Work Group Member Grid Architecture Development Work Group In Galaxy Services and Features Development, Maintenance Second Galaxy Development Team

[emperorstarfinder]

Because this issue will be fixed in the course of the new source code layout and cleaned up code, I am closing this issue as it will now be moot. At least in the short term.