GeoscienceAustralia / ginan

The Australian Government, through Positioning Australia (part of Geoscience Australia), is funding the design, development and operational service of a Global Navigation Satellite System (GNSS) position correction system - the Ginan service and toolkit. The application of the Ginan correction service by a GNSS device has the potential to increase positioning accuracy from meters to centimetres across Australia. The suite of software systems in this repository (the Ginan toolkit) will be used to create the service. It is available now under an open source licence. Ginan will give individuals and organisations no-cost access to the Ginan software and service as a public good.
https://geoscienceaustralia.github.io/ginan/
Other
229 stars 108 forks source link

Error on example rt_ppp_example.yaml #119

Open jiargei opened 1 week ago

jiargei commented 1 week ago

Describe the bug I tried to run the example rt_ppp_example.yaml but it fails. I changed the input sources (other CORS and BCEP).

To Reproduce Steps to reproduce the behavior:

  1. Go to 'exampleConfigs'
  2. Wait until "Synced X receivers"
  3. See error

Expected behavior Should start processing stations.

Configuration files

pea_output.txt

pea: malloc.c:2617: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.
Aborted (core dumped)

Operating environment (please complete the following information):

These may be found in the 'Compilation details' section of the pea's output.

Additional context Add any other context about the problem here.

smcclusky commented 1 week ago

Can you create and provide us with a backtrace using gdb on the core file that was produced by that crash (core dump) please. We would like to be able to see exactly which function and line of code produced it.

smcclusky commented 1 week ago

Thanks for the comment. We are aware of the slow memory leak issue you raised. We don't strictly consider this a "leak", but believe the slow increase in memory usage arises due to the map/s that contain the SV ephemeris information, slowly growing in size with time. We will take another look at this issue and see if there is a way that these slow increase in memory usage can be mitigated.

seballgeyer commented 1 week ago

Hi Jürgen,

I think the issue has been identified and addressed in the develop-weekly branch previously (https://github.com/GeoscienceAustralia/ginan/issues/106). We recommend switching to this branch to pull the latest fixes and re-run the example configuration to confirm the fix. Additionally, a new release with this and other improvements is expected within the next few weeks, providing a more stable update for all users. Let us know if you encounter any further issues in the meantime.

Best regards, Sebastien

jiargei commented 1 week ago

Hej Sebastien!

Thanks for the update, I switched to the develop-weekly branch. Unfortunately the sigma values are now higher than before. Now it is more than 500 meters, it was ~5 meters.

Kind regards, Jürgen

seballgeyer commented 1 week ago

Hi Jürgen,

Yes, I just realized this too. I am looking at it now.

Sebastien

jiargei commented 1 week ago

Hej Sebastien!

I switched back to branch release-v3.1.0. The sigma values are now in sub-meter range, but it takes several hours to get there...

+STATES/PPP
*    2024-10-29 08:59:35                 REC_POS     AMST             0   4122998.5403211          0.36205304        0.05485011                     0x117c8ed0                                          
*    2024-10-29 08:59:35                 REC_POS     AMST             1   1094890.4266538          0.42046421        0.30902426                     0x117c8ed0                                          
*    2024-10-29 08:59:35                 REC_POS     AMST             2   4726256.8256289          0.25640758        0.03903920                     0x117c8ed0                                          
*    2024-10-29 08:59:35                 REC_POS     MRDF             0   4123612.2445520          0.36277859        0.06034817                     0x10ece710                                          
*    2024-10-29 08:59:35                 REC_POS     MRDF             1   1186651.4980470          0.41900771        0.31457385                     0x10ece710                                          
*    2024-10-29 08:59:35                 REC_POS     MRDF             2   4703895.7527488          0.25676796        0.03884333                     0x10ece710                                          
*    2024-10-29 08:59:35                 REC_POS     NSDL             0   4096426.1308565          0.36633576        0.05708903                     0x1110af80                                          
*    2024-10-29 08:59:35                 REC_POS     NSDL             1   1239622.1316339          0.42082199        0.30886839                     0x1110af80                                          
*    2024-10-29 08:59:35                 REC_POS     NSDL             2   4713482.4674074          0.25934142        0.03881486                     0x1110af80                                          
*    2024-10-29 08:59:35                 REC_POS     STPO             0   4101619.4977111          0.36261136        0.05509162                     0x11347b20                                          
*    2024-10-29 08:59:35                 REC_POS     STPO             1   1147812.7175879          0.42000558        0.31016811                     0x11347b20                                          
*    2024-10-29 08:59:35                 REC_POS     STPO             2   4732251.5544407          0.25641895        0.03822631                     0x11347b20                                          
*    2024-10-29 08:59:35                 REC_POS     WIEN             0   4085136.0975974          0.36311141        0.05655818                     0x115844f0                                          
*    2024-10-29 08:59:35                 REC_POS     WIEN             1   1200313.3849334          0.41936398        0.31121022                     0x115844f0                                          
*    2024-10-29 08:59:35                 REC_POS     WIEN             2   4733345.0473014          0.25650217        0.03837182                     0x115844f0                                          
-STATES/PPP

Just wanted to notice you about that, it might help finding the issue.. in Postprocessing the PPP-works really well!

Kind regards, Jürgen

seballgeyer commented 1 week ago

Hi Jürgen,

we managed to find the issue in the develop_weekly branch, we are doing some more check, but it was introduced at the last commit, so the previous commit (aaed31fcddbcd44c74aa990246c43eeee9fe08f8) is safe to use.

The config file as provided needs to have the following changes:

  1. replace receiver_options -> models -> phase_bias with receiver_options -> models -> code_bias
  2. turn off (remove) all the section estimation_parameters -> satellites
  3. we also added
    receiver_options:
    global:
        [...]
        clock_codes: [AUTO, AUTO]
        zero_dcb_codes: [AUTO, AUTO]

doing so, I have a sigma less than 1m after a few minutes and about 5cm after 15 to 20 minutes (first epoch is 11:29) image

Best regards Sébastien