An error occurred while reading the broadcast ephemeris file by“GnssRinexNavigation2OrbitClock”

[IamLittleMonster]

Description

Dear developers,

When reading the broadcast ephemeris, sometimes incorrect broadcast ephemeris records may be read in. Of course, the reason for the above situation is more likely due to errors in the broadcast ephemeris file itself.

The following three images are three consecutive records related to C30 in BRDM00DLR_S_20223390000_01D_MN.rnx which was downloaded from ftp://igs.ign.fr, from which it can be seen that the same "Toe" corresponds to two records. Unfortunately, GROOPS seems to have read the incorrect record(the second picture) during runtime.

If the source code is not modified, the result of comparing GnssRinexNavigation2OrbitClock's output with precision ephemeris is as follows:

I have found a solution to the above problem, specifically by modifying the 246th line in the gnssRinexNavigation2OrbitClock.cpp file (part for reading orbits) and the corresponding part for reading clock deviations.

The modified line is: if(selection == NULLINDEX || fabs(dt) <= currentDt)

After modifing this part, the orbit comparison is:

However, I believe this is only a temporary modification plan. The specific judgment of problematic ephemeris should be based on the comprehensive judgment of Epoch, Toe, and Transmission time of message in the broadcast ephemeris (highlighted in red in the below figure).

I hope developers can adopt my modifications or propose more effective ways of judgment.

Best regards, Hongzhan

GROOPS version

main (latest commit)

Operating systems

• [ ] Linux
• [X] Windows
• [ ] macOS
• [ ] Other (please specify in the description)

Log output

No response

[IamLittleMonster]

Another feasible solution is to check whether the records corresponding to the same Toe time for the same PRN are exactly the same, and if they are different, delete all records at that Toe time.

[zhedumi]

Thank you very much for the input! I had pondered about this error for quiet some time and checked over the computation algorithm several times but never thought It could be due to the TOE being the same in two epochs. And for all the other computations thanks to robust least squares the faulty errors would be removed anyway. (Plots are differences between broadcast ephem and igs final iirc)

@IamLittleMonster

Another feasible solution is to check whether the records corresponding to the same Toe time for the same PRN are exactly the same, and if they are different, delete all records at that Toe time.

Yeah I think this will be the easiest way to solve it but I will go through the rinex format description again especially the TOE etc and will modify the code. I will check tomorrow thoroughly this issue and by Friday I think a release can be made to fix this issue. So I apologize that this may take some more time! Also I kindly ask you if it is possible to provide me the same NAV file so I can use it for testing since otherwise I need to find a rinex file with this issue.

Cheers

Patrick

[IamLittleMonster]

@zhedumi I am very pleased to receive your reply. Here are several problematic broadcast ephemeris files, all of which are related to BDS satellites. What I can confirm is that C30 and C35 on 2022339 have the above-mentioned issues. I have not made any records on other days, but there will be BDS satellites with problems every day in the above documents. BRDM00DLR_S_20223390000_01D_MN.rnx.gz BRDM00DLR_S_20223410000_01D_MN.rnx.gz BRDM00DLR_S_20223430000_01D_MN.rnx.gz BRDM00DLR_S_20223450000_01D_MN.rnx.gz BRDM00DLR_S_20223490000_01D_MN.rnx.gz

The above files can be used for your algorithm verification. In my view, this may be an issue with the BDS broadcast ephemeris file itself. Additionally, my approach has temporarily resolved the issue and has not caused significant inconvenience. Finally, thank you again for your handling of this issue.

Cheers

Hongzhan

[zhedumi]

Thank you for the files!

Sadly it is not BDS only. I have seen these jumps in a few Galileo satellites as well. In Galileo I saw those jumps being reocurring especially for E14 and E18 which are in the incorrect orbit. These jumps occured more frequently. But since they are in the incorrect orbit I just assumed that that is due to the incorrect orbit.

That is why I went through the computation algorithm several times and could not figure out what the issue is. So I am very thankful that you took a closer looks at the files. I hope with the fix these jumps will be all together gone.

Cheers, Patrick

[zhedumi]

Hi,

Sorry but I probably will not be able to make a release today but I will need one more day so I can fix it on monday properly. I already wrote code that would do what would work for your issue but the issue is a bit more complicated than I would have thought. Just a check on seeing whether toe and toc is not really defined so it will take me a bit longer to figure out a proper way to handle this. This is due to the NavMessages themself not really providing a definition for some GNSS.

GPS LNAV p.134: toe is equal to toc of same LNAV Clock, Ephemeris, Integrity GPSL2 CNAV p 157: toe is twice in the raw navbits type 10 and 11 and must be the same as toc otherwise new data must be used. P206: toe and toc equal in same CNAV CEI data set. GPS L5 CNAV p. 38 : toe is not current or does not match toc (or within current curve fit). GPS CNAV2 P. 51: toe and toc are the same

Galileo FNAV: toc and toe are speratelly used for clock corrections and orbits. No relation given INAV: toc and toe are seperatelly used for clock corrections and orbit. No relation given

BeiDou D1 and D2: Only definition i found is that if toe changes then toc also has to change. Or atleast in the english version. No idea if the original language would have more information. BeiDou CNAV2 did not find anything for toe or toc within but I presume it is same designed as GPS CNAV2 and therefore must be the same. Weirdly tho beidou CNV2 has ToE seconds of week in its ephemeris but no weeks???? BeiDou CNAV3 Same issues as CNAV2. CNV3 has ToE seconds of the week also within.

For BeiDOu CNAV2 and CNAV3 I just assumed that toc = toe in the programm and now I am questioning this. I presume I am supposed to compute the GNSS weeks with tocs and then use ToE.

Rinex2 no definition regarding TOE and TOC relation. Rinex3 no definition regarding TOE and TOC relation. Rinex4 Only mention is GPS CNAV toe and toc must be equal and only toc is provided

So by the definition it is completely possible to have different TOC and TOE for some navigation messages or atleast it is not mentioned different (or I oversaw it). What I thought to be only a check between ToE and ToC My plan is to check the TOC changes and check if TOE also changed with it but Galileo has it not defined whether TOC changes also imply TOE changes. So with this in mind I am thinking about a more thorough validation check for the data and also more restructuring for better code maintainance which will take me at least monday todo.

In the file you provided the Ephemeris have so much issues. Were the signals spoofed? Sometimes the TOE did not change but half of the ephemeris changed. Was this by any chance from the signal changing epochs and receiver not able to handle it properly?

But incase you want to still have code and cant wait till monday where I will have everything fixed, tested and validate here is some code where it will throw out Navigation data which have not the same TOC and TOE here is the code which you can use for the time being. gnssRinexNavigation2OrbitClock.txt

Just be aware that for Rinex4 it throws out a lot more than for Rinex3. interestingly for Rx4 it finds a lot more toe/toc differences in BDS than in RX3 as seen by the following 2 files which is just converting the broadcast from IGS for 2022. toc_toe_2022_rx4.txt toc_toe_2022_rx3.txt

Cheers

Patrick

[IamLittleMonster]

Hi, Sorry for replying to you later. Let me share my opinion below:

1. For CNAV2 and CNAV3 of BDS, it seems that they did not provide information on BDS week;

2. My initial idea was to exclude problematic records through Toe and Toc, specifically to determine whether the numerical difference between Toe and Toc is within a certain threshold range;

3. In my opinion, whether Toe and Toc are equal may not be the most important thing, because we only need to use Toc to calculate clock bias and Toe to calculate orbits at time points;

4. The broadcast ephemeris file I am using comes from igs.ign.fr, and here is its header information: I'm sorry, I'm not quite sure how the broadcast ephemeris was generated. The download address for my broadcast ephemeris is 192.134.134.6:21, and the specific path is/pub/igs/data/2022/DOY;

5. If we want to improve the robustness of GROOPS, we do need a function to preprocess the broadcast ephemeris (the function I can think of is to exclude inconsistent records based on whether the records of the same PRN and Toe are the same). The current version seems to have used the records after reading them.

6. Thank you for providing me with the documents, which will help me with my next steps.I don't have an urgent need for the latest version of code at the moment, so you can slowly develop the corresponding parts ^ - ^. For solving the above problem, I only modified two lines of the source code (as I mentioned earlier), and this is the file I modified: gnssRinexNavigation2OrbitClock_zhz.txt

Have a good weekend

Hongzhan

[zhedumi]

Hi,

I adapted the source and added a sathealth check and whether it should be removed. Incase of same TOE appearences I decided to just remove all entries incase same TOE have different ephemeris. The decision on which TOE is the correct one explodes quiet much when taking everything into account and this seems the most straightforward solution. Especially considering that broadcast ephemeris have a valid range up to hours some entries missing really does not hurt that much in my opinion. For PPP I would also just recommend using the precise orbit data anyway. I had hoped to use IODE or AODC but I had the weird thing that there are same TOE but different AODC and then it is just guessing around which is correct. Furtheremore, it seemed that in most cases the double TOE has been removed due to the sat health flag being in the unhealthy state anyway.

But I would love to hear if that is alright with you or if some other way would be more preferable. Since I usually use the orbits only as a prior value for other stuff I have less insight on what some people might want with the broadcast rinex files. I will keep looking into it though so there should be another update in the future.

The code modified also bit of refactoring. gnssRinexNavigation2OrbitClock.cpp.zip

Cheers,

Patrick

[IamLittleMonster]

Hello, In my opinion, the improvement plan you mentioned is currently quite suitable. The purpose of using a broadcast ephemeris is also to provide initial values, and the absence of a certain epoch will not have a significant impact on subsequent work. Because the orbit can be improved in accuracy through subsequent dynamic and observation models.

Thank you for your attention to this issue.

Cheers,

Hongzhan

[zhedumi]

In case I forget to mention it there is a complete overhaul being in progress of the program to read the broadcast ephemeris since the code is becoming less and less maintainable and I have gained a lot more knowledge about the data within. I will see if I can make it as fast as possible.

cheers Patrick

[IamLittleMonster]

Okay, thank you for your attention to this issue. I look forward to the updated program code.

cheers Hongzhan