search

Leonhard96L / NVGSIM

0 stars 0 forks source link

D-WORLD test on FTD/FNPT #18

Open Leonhard96L opened 1 year ago

Leonhard96L commented 1 year ago

To test D-World on our FTD/FNPT we need an employee in Linz or we can do it remotely

Leonhard96L commented 1 year ago

Organize a appointment with Ivan.

Leonhard96L commented 1 year ago

We will make a remote session for installation on 11.07 or 12.07. I will give you then the link and the password for remote access.

imiletovic-ms commented 1 year ago

Let's try on 11.7, if there are problems that cannot be resolved immediately we then have 12.7 as a backup. Is that ok? Before next week I will inform here how to perform the installation of D-WORD/D-SIM remotely.

Leonhard96L commented 1 year ago

Yes we can plan 2 days. Thank you.

imiletovic-ms commented 1 year ago

I discussed here with Martijn about the D-WORLD/D-SIM (remote) installation. We can make a build and configuration necessary to run our helicopter (AS532) simulation. We need some more information beforehand though:

Leonhard96L commented 1 year ago
  1. Actuators from Moog. Controlls are Custombuild from Reiser but you can get the input via LAN.
  2. We have 5 IG Computers. One Master and 4 Slaves (for each projector). The GPU´s are GTX 1080. The CPU´s are i3 8350k. And we have 32GB RAM storage. But we have to look if we have enough storage. How much storage does the software need?
  3. We have to look what our hardware can handle. So much weather as possible... Brownout and whiteouts would be intersting.
  4. All computers are accessable. We will checkout the ip adresses.
  5. Yes we have a share network drive available for all computers. Storage is 500GB
imiletovic-ms commented 1 year ago

Thank you!

I discussed further here also with Mark about the scope. I think the goal of the remote session should be to get D-SIM/D-WORLD up and running a.s.a.p. with the Varjo XR augmented reality goggles as a demonstration. Correct?

In that case, we can try with only one computer with approximately 100-150GB of free disk space for D-WORLD. If that is not enough for performance, we can try to offload some processes like the flightmodel to another computer. We will deliver only The Netherlands terrain database for now, as that is what we can share without a license.

D-SIM doesn't support MOOG actuators out-of-the-box, only Brunner and basic joystick controls. Longer-term if you want to run D-WORLD with your existing simulator controls an interface is needed to get the control inputs to D-SIM. Also a different configuration is necessary to drive projectors. If you want to use the flightmodel in D-WORLD with the Varjo XR, we can use a basic joystick if you have one available for now.

Leonhard96L commented 1 year ago

Hey Ivan, thank you! Exactly this is the goal. Ok than we will try with one computer. Good idea! 100-150GB is no problem. At the beginning the Netherlands terrain database is sufficient. In further time we can organize some aerialpictures from Austria.

Ok, it would be great if we can use the already inbuild controlls. We do not have a basic controller available, but i will check if i can organize one for the session.

imiletovic-ms commented 1 year ago

Great! Then I will arrange that there is a working configuration for D-WORLD and an XR3 running on one computer. At the moment it looks like I'm going to need the majority of Tuesday 11.7 for testing the configuration and uploading it to our Azure server. Is it ok if we plan the remote session for Wednesday 12.7 then? What time would be convenient for you to start?

For the joystick, we use the Logitech X3D. It has has a slider that is by default mapped to the collective control of the helicopter. in D-SIM. But this really this is only necessary if you want to fly the model in D-WORLD. The question is also how feasible that is with your current hardware. We will see... :)

Leonhard96L commented 1 year ago

12.7 is totally ok. We can start whenever you want but earliest at 08.00 a.m. I will talk with my collegues if we need to fly the model... Yeah let´s try this old hardware :)

Leonhard96L commented 1 year ago

Hi Ivan, the logitech X3D was not available in the shop. Instead we bought the THRUSTMASTER T.16000M FCS-Joystick. Hope this will be ok. If not then we can just test D-World without flying. And tell me when you want to start tomorrow. Thank you! We are very excited about the test :D

imiletovic-ms commented 1 year ago

I think the Thrustmaster stick will also work. I can be online by 9.00 tomorrow if that's ok? Also it looks like we will be needing two PC's for D-SIM and D-WORLD. One will be used for D-WORLD with the XR3, while the other will run a separate instance of D-WORLD for the physics as well as the other processes. Could you prepare two computers for this? It would also be good if you could hook up a monitor to the 2nd PC, but I understand this may be difficult. Finally, could you look up your network prefix, subnet ID's and the MAC addresses of the PC's as well? Thanks!

Leonhard96L commented 1 year ago

Ok great. 0900 is ok. We have already the running scientific project and therfore we have a extra powerfull PC (i7-13700 and RTX4090) which is already connected with the XR3 and already connected with a seperate Monitor. Then we can use the powerfull PC for D-WORLD with the XR3. And we can use an other not powerfull PC to run the physics.
We have a remote access tool for our computers. But not for the powerfull PC. Do have an recommendation for a remote freeware?

I will check out the network stuff tomorrow in the morning and i will write it in github.

Thank you!!

Leonhard96L commented 1 year ago

Hi Ivan, this is the link for remote access to our PC´s but not for the power PC: https://fnpt1.mysim.cloud/

username: rsky pwd:     EjCUBLJrALiMr7JaKo2Te4KX9Um5jE

Leonhard96L commented 1 year ago

The Network stuff for the not powerfull PC (for physics): 172.22.0.20 SNM: 255.255.0.0 28-80-23-B3-6D-CA

For the powerfull PC (D-World): 172.22.4.96 SNM: 255.255.0.0 58-11-22-BE-8F-D7

imiletovic-ms commented 1 year ago

Perfect Leonard, thank you! I'll be on Teams in about 10 minutes and then I will try to login through your link. Did you get the Google Drive share I sent you yesterday with the D-WORLD asset bundles?

Leonhard96L commented 1 year ago

You are welcome. Yes i got it. I already moved and unpacked on the power PC for D-World

imiletovic-ms commented 1 year ago

Ok, great! Both PC's will need the asset bundles though, so it should be on the remote access (physics) PC as well.

imiletovic-ms commented 1 year ago

But we can arrange that during the Teams call as well, I'm coming online now

imiletovic-ms commented 1 year ago

The remote installation of D-SIM/D-WORLD was successful! Very happy about that. Let me know in case you encounter any problems with the installation from now on.

I will discuss further here the possibility to couple the CLS controls in the FTD/FNPT simulator to D-SIM. However, this morning I had a brief discussion about this with Martijn. The work required could be minimal if the inputs from the CLS are somehow already recognized as joystick inputs in Windows. Is this something you could get more information on from your side? For example, how are the inputs currently coupled to X-Plane?

Also for the instruments, a relatively easy option would be to stream the instruments as overlays in VR. Maybe you saw a demo of this setup when you visited us here in Soesterberg? There would still be a see-through mask where the instrument panel is, but the D-SIM instrument windows will be placed over the real instruments in VR. This will improve the quality over using the real instruments especially in low light conditions (as you saw this morning, it is quite blurry otherwise). Any buttons/switches/controls on the instrument panels will still be visible in the see-through mask and can be used naturally.

I will keep you updated in case I have new information!

Leonhard96L commented 1 year ago

Ivan thank you very much!! I will check how the inputs are coupled with Xplane.

I will check it out with my collegues if this could be a option, to use overlaing pictures.

Leonhard96L commented 1 year ago

Hi Ivan, sorry for the late response i was in holiday last week.

I discussed the topics with Michael and Raimund this morning.

  1. We checked out what we get: From X-Plane we only get the cartesian coordinates (x,y,z) and the euler-angles (psi, theta, phi) of the aircraft. The rest of the parameters are coupled with the flightmodel which is running on a linux pc. Is this enough for D-World? Beacause at the end we would fly with the already existing flightmodel from Reiser. And would use D-World for the Vision.

Here i have a script (SimData.zip) which can read the X-Plane parameters. It was written from a programmer from the scientific project from which we have the XR3-Glasses. Maybe it could help you.

  1. We would use the pictures of our inbuild panels without overlaying pictures just with the outcutted cockpit. We would solve the problem with the light conditions by ourself.
imiletovic-ms commented 1 year ago

Hi Leonhard,

Hope you had a great holiday! And thank you for following up on the issues now that you are back.

1) I believe the approach you suggest here is different than what we had in mind originally. Namely that the CLS inputs would be coupled to D-SIM and therefore our flightmodel. I.e. just replacing the joystick you currently use with the CLS inputs. What you suggest would bypass our flightmodel and would only use D-WORLD as a render engine for your current flightmodel. This should be possible, maybe we can run D-SIM (it supports linux too) on your flightmodel PC and feed the position and orientation of the aircraft to D-WORLD directly through D-SIM from the flightmodel PC to the render PC. I will check with Martijn what he thinks about this. Unfortunately the code you provided is unreadable. It seems to be binary (i.e., compiled) code and not the source code.

2) Ok, that's fine and should also be the easier approach if you want to use your existing flightmodel.

Leonhard96L commented 1 year ago

Hi Ivan,

thanks a lot for your answer. Exactly we would use D-World as render engine and fly with our flightmodel. This what i sended you is a xplane plugin. I already asked the guy which has written the program, if he can give us the source code. If i get something i will send you immediately.

Leonhard96L commented 1 year ago

Hi Ivan,

i talked with the programmer which has written the script. He told me that it wouldn´t make much sense to send the source-code. Instead he wrote me a email with some usefull links:

TXT.pdf

All the information about X-Plane development can be found here: https://developer.x-plane.com/

IMHO, there are two ways to create a data interface with X-Plane. As said, if you just want to read some parameters/datarefs, you might prefer the second possibility described below.

  1. Plugin to retrieve/forward data
  1. X-Plane data publishing
Leonhard96L commented 1 year ago

Hi Ivan,

i just want to ask you how it´s going on with D-WORLD on our FTD/FNPT?

BR

imiletovic-ms commented 1 year ago

Hi Leonhard! I'm sorry I have not yet responded, I must have missed the previous comment because I was on a holiday.

I discussed this with Martijn before my holiday and I think it won't be difficult. D-SIM has support for interfacing with X-Plane, so it should be possible to simply read the data (position & attitude) from your X-Plane instance directly into D-SIM and pass it on to D-WORLD. This way you will use your existing (X-Plane) flightmodel with D-WORLD without needing to install D-WORLD on the flightmodel PC.

I will again discuss the details with Martijn tomorrow, and will let you know if/what more information we need from you to make this work.

Leonhard96L commented 1 year ago

Hi Ivan,

it´s fine. Thank you for the update.

imiletovic-ms commented 1 year ago

Today I discussed again with Martijn about using D-WORLD on your simulator. According to him, the best solution would be to read the UDP packets that X-Plane is receiving from your Reiser flightmodel. It is relatively simple to intercept these UDP packets in D-SIM. They are described to some extent in the document you linked, but it would be more helpful to also have some real data for testing here.

Are you familiar with the program Wireshark? This program can be used to monitor network traffic on a PC. If you run Wireshark while X-Plane is running on the Master PC you can capture the data transmitted and received as described here. Even better would be if you can capture while flying around somewhere in the Aviano (IT) or Woensdrecht (NL) area in X-Plane. Then we can read the captured data and replay it here in D-WORLD.

If you need assistance with this, I would be happy to schedule a remote session some time soon. Let me know!

Leonhard96L commented 1 year ago

Hey Ivan,

thank you again. Sounds great. I will try that asap. If i need some help i will let you know. Thank you

Leonhard96L commented 1 year ago

Hey Ivan,

today i tried to read the UDP packages with wireshark. I recorded a file, by filtering the port 49010. This port was defined as data input port in xplane. I flew i littlebit around in the Sim while i was recording. I only flew in Austria, because i had not enough time. Sorry for that. Can you check this out and tell me if this is the right traffic?

I would repeat this flyout in Aviano (IT) and Woensdrecht (NL) on Friday.

I had to zip this file because github didn´t want wireshark file. dump_port490010.zip

Leonhard96L commented 1 year ago

Hi Ivan, i just want to ask if you already tested the data. And you can say me if that is the right data channel. When it´s the right one i can fly again in Aviano and Woensdrecht. BR

imiletovic-ms commented 1 year ago

Hi Leonhard,

I have checked the file you sent and it looks like parsing the data isn't difficult as the host computer is broadcasting CIGI packets containing both timing and position information. However, at the moment we are very busy with finalizing the NVG simulator and I'm also pre-occupied with a different project so I don't have much time to look into it further. I propose that we come back to this issue after delivery of the NVG simulator.

Leonhard96L commented 1 year ago

Hi Ivan,

thanks for your answer. That is ok for us to. We have also a lot of things to do. Can you please tell me after the delivery of the NVG sim when you continue with this topic so i would support you.

imiletovic-ms commented 9 months ago

Hi Leonhard, let's pick this up when I'm back from my holiday (in Austria!) in two weeks. I will contact you to schedule a meeting where we can make a start with coupling the FTD to D-WORLD by using the X-plane network data (master IG) as a starting point.

Leonhard96L commented 9 months ago

Hi Ivan,

we will have access to the output of the Flightmodell from Reiser. Next week some Guys from Reiser will be here and they will help us. So we do not need to extract the data via Xplane anymore. We can extract it directly from the flightmodel and can send it to D-WORLD.

Leonhard96L commented 9 months ago

Hi Ivan,

we have now acces to the flightmodel data. You could extract data over google protobuf.

This is what i got from a Reiser engineer:

Fwd Eventauslese für Forschungsprojekt.zip

Can you do something with that?

If not we have to extract the Data from Xplane.

Would you have time for a remote session on monday?

Leonhard96L commented 9 months ago

Hi Ivan,

now we are able to set up the transmission of XPlane parameters to the XR PC via UDP (Port 49002). This should enable us to utilize the values in DWORLD as intended.

I've attached a sample file for your reference. Additionally, you can find a link detailing the structure of the packet. Please feel free to reach out whenever you have something to share or if you need further assistance.

I'll be at the simulator tomorrow morning. In the afternoon, my colleague will be on-site.

http://www.nuclearprojects.com/xplane/xplaneref.html

xplaneUDPcapture49002.zip

Good evening.

imiletovic-ms commented 9 months ago

Hi Leonhard, thank you for the file! Can you also provide a list of exactly which variables you are transmitting (in order if possible)? Yesterday I looked at the Wireshark dump you sent me through WhatsApp and when I tried to convert some of the binary data to numbers according to the link they didn't make a lot of sense. I will see if I can find something else online, otherwise I will download and install X-Plane myself and see if I can learn something about the data pattern there.

I think I will need at least today and a part of tomorrow to write and test locally a program to read the X-plane data and also to make a D-SIM configuration file that uses the 2nd XR PC to render the VR world based on the position and attitude in X-plane. Maybe aim for Thursday to try and deploy it on your side?

As discussed previously, this method will have a problem with terrain mismatch between D-WORLD and X-plane, so landing in D-WORLD may not be accurate/possible at all.

Best regards.

Leonhard96L commented 9 months ago

Hi Ivan,

I saw there is also the possibility to extract the Parameter in a txt file which is located in the xplane main folder. We have recorded some values. You see the the dump and the txt in the appendix.

So i have chosen the following params via UDP and to txt: -Pitch, roll, heading -angle of attack, sidslip -Latitude, longitude, altitude -All planes latitude -All planes longitude -All planes altitude.

The txt file is more detailed. UDP.zip

I will not be here the next days until Monday, but Raimund is here on Thursday. So you could do a remote Session with him.

Leonhard96L commented 9 months ago

Sorry for disturbing you again. I have tried again with some other params selected. These are more useful: -Pitch, roll, heading -Magnetic Compass -Latitude, longitude, altitude -Location, velocity , distance traveled

UDP.zip

https://www.x-plane.com/kb/data-set-output-table/ under this link you find a detailed description.

Keep in mind that not all Variables in the files are effected by the flightmodel. But in the recording should be all varibles which are changed in xplane by the flightmodel.

imiletovic-ms commented 9 months ago

Hi Leonhard, thank you. I made some progress in figuring out the data format of the UDP stream, now I'm trying to correlate the UDP data to the output txt you sent so that hopefully I can discover a fixed pattern.

Leonhard96L commented 9 months ago

Great. Thank you!

imiletovic-ms commented 9 months ago

I think I figured it out! It's not really intuitive, but each selection of parameters in the X-Plane output screen corresponds to a set of data with up to 8 values in it. You selected 4 outputs, but each has a different number of actual values. E.g., pitch, roll, heading is only 3 values but the last one (pos, vel, dist. travelled) is 8 values in total. These are always transmitted as rows of 8 numbers + 1 index (position in output table in X-plane), even if not all 8 numbers are actually written by X-plane.

This means it is important to "fix" the selection of outputs in X-plane from now on, because if it changes the reader software in D-SIM will no longer work as the values will in the UDP data stream will not be interpreted correctly. Is this the final selection of parameters you want to transmit to D-SIM?

Leonhard96L commented 9 months ago

Hello, this is Michael. Leonhard is going to fly so I am taking over for now. Right now we are also looking into interfacing our motion system over telemetry data but this may not work. Therefore I would say we probably won't have to change the selection of outputs of the UDP stream.

Is the selection that is currently configured enough for your script or do you need additional attributes? Additionally, is there a way we could later modify your reader script in case we do have to change the configuration again?

imiletovic-ms commented 9 months ago

To keep the software on our side somewhat generic, could you reduce the outputs to just:

-Pitch, roll, heading -Latitude, longitude, altitude

This is sufficient for now to render the aircraft in D-WORLD and also keeps the UDP data network traffic minimal.

Leonhard96L commented 9 months ago

ok, I'll do that. Do you need another Pcap with the new attributes?

imiletovic-ms commented 9 months ago

Hi Michael, thanks a lot! Another Pcap could be useful for debugging, so yes if it's not a big effort for you.

Leonhard96L commented 9 months ago

xplaneUDPcapture49002_4.zip filter with udp.port==49002

Leonhard96L commented 9 months ago

Hi Ivan, thank you again for yesterday. In the next days we will test the xr3 googles and we will try to set the correct postion and testing the greenscreen.