georust / rinex

RINEX and GNSS data processing :artificial_satellite:
Apache License 2.0
63 stars 14 forks source link

Improve performance #230

Closed gwbres closed 2 months ago

gwbres commented 2 months ago

This branch improves overal performances severely, especially in PPP opmode. By improving the way we interface to the RTK core, we reduce the processing time severily. PPP resolution of test_resources/ESBCDNK-2020 (24hr) is reduced from 2'30 to 3.5sec on my computer.

NB: this will remain open until we're 100% sure data precision is preserved.

lnicola commented 2 months ago

For the record, I can't build this:

    Checking rstats v2.0.1
error: no rules expected the token `"point being added is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:447:43
    |
447 |             return re_error("arith",here!("point being added is coincident with gm"))?;
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"point being removed is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:458:43
    |
458 |             return re_error("arith",here!("point being removed is coincident with gm"))?; 
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"point being removed is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:469:43
    |
469 |             return re_error("arith",here!("point being removed is coincident with gm"))?; 
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"radius: invalid subscript"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecvec.rs:264:47
    |
264 |             return re_error("DataError",here!("radius: invalid subscript"))?;
    |                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"sigvec given no data"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecvec.rs:303:43
    |
303 |             return re_error("empty",here!("sigvec given no data"))?;
    |                                           ^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro
gwbres commented 2 months ago

For the record, I can't build this:

    Checking rstats v2.0.1
error: no rules expected the token `"point being added is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:447:43
    |
447 |             return re_error("arith",here!("point being added is coincident with gm"))?;
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"point being removed is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:458:43
    |
458 |             return re_error("arith",here!("point being removed is coincident with gm"))?; 
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"point being removed is coincident with gm"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecg.rs:469:43
    |
469 |             return re_error("arith",here!("point being removed is coincident with gm"))?; 
    |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"radius: invalid subscript"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecvec.rs:264:47
    |
264 |             return re_error("DataError",here!("radius: invalid subscript"))?;
    |                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

error: no rules expected the token `"sigvec given no data"`
   --> /home/grayshade/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rstats-2.0.1/src/vecvec.rs:303:43
    |
303 |             return re_error("empty",here!("sigvec given no data"))?;
    |                                           ^^^^^^^^^^^^^^^^^^^^^^ no rules expected this token in macro call
    |
    = note: while trying to match end of macro

The Georust RINEX toolsuite does pass on my side and on the github worker. But I do face the same problem when compiling the RTK core by itself, which is almost unbearable. Therefore I only use the RTK core through the RINEX toolsuite, which is anyways my main application of it to this day. This appeared about last december.

@ChristopherRabotin

rstat is a dependency that Nyx has to get rid of. Have no idea why it behaves so strange nor what it does

ChristopherRabotin commented 2 months ago

Oof, yeah, I often have issues with rstats. The crate is dependent on another one which is maintained by the same person, and he'll update the dependency but only make a patch version change but it should be a minor version number change.

Can you try to run "cargo update"? That usually solves the issue for me.

On Sat, Apr 13, 2024, 12:34 gwbres @.***> wrote:

@.**** commented on this pull request.

In rinex-cli/src/positioning/interp/time.rs https://github.com/georust/rinex/pull/230#discussion_r1564185114:

@@ -0,0 +1,220 @@ +use crate::cli::Context; +use gnss_rtk::prelude::{Epoch, SV}; +use std::collections::HashMap; + +use super::Buffer as BufferTrait; + +struct Buffer {

  • inner: Vec<(Epoch, f64)>, //TODO +}
  • +impl BufferTrait for Buffer {

I was interested in having this Trait because we need both spatial and time interpolation in this topic, therefore they share the same logic. Especially how we manage in the internal fifo against data gaps.

Note that data gaps are not correctly handled in the current implementation. This proposed form is a first solution, probably too stringent (does not tolerate any gaps, since we reset the fifo when a gap appears), but we will improve that in the future (simple detail)

— Reply to this email directly, view it on GitHub https://github.com/georust/rinex/pull/230#discussion_r1564185114, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABEZV2GWV2T4FA3XES7HNS3Y5F3DRAVCNFSM6AAAAABGFSDLPKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSOJZGMZDQOJYHE . You are receiving this because you were mentioned.Message ID: @.***>

lnicola commented 2 months ago

Can you try to run "cargo update"? That usually solves the issue for me.

Thanks, that worked. I only tried cargo update -p rstats, but that here! macro is from.. indxvec, which is the scariest crate name I've ever seen :roll_eyes:.

gwbres commented 2 months ago

@ChristopherRabotin, @lnicola

thank you guys !