search

rawify / GPS.js

A NMEA parser and GPS utility library
https://raw.org/article/using-gps-with-node-js-and-javascript/
MIT License
253 stars 61 forks source link

GSV merge is inconsistent, "blinking" satsVisible / it disappears and reappears. #32

Closed VigibotDev closed 3 years ago

VigibotDev commented 4 years ago

Hello,

I get random OK/NOK on state.satsVisible object, it disappears and reappears a lot of time, I use 10Hz refresh rate, my NMEA look OK, some GPS are OK, some GPS I get the "blinking" satsVisible.

I think this is this TODO

    // TODO: better merge algorithm:
    // 1. update every sat and mark as updated.
    // 2. If last msg, delete all unmarked sats & reset mark
    if (data['type'] === 'GSV') {

      var sats = data['satellites'];
      for (var i = 0; i < sats.length; i++) {
        collectSats.push(sats[i]);
      }

      // Reset stats
      if (data['msgNumber'] === data['msgsTotal']) {
        state['satsVisible'] = collectSats; 
        collectSats = [];
      }
    }
VigibotDev commented 4 years ago

GPS where it work :

cat /dev/serial0 | grep GPGSV 

$GPGSV,3,3,09,31,54,221,*42
$GPGSV,3,1,09,02,00,034,,04,11,315,,05,13,056,,09,01,347,*78
$GPGSV,3,2,09,16,16,294,,18,47,159,,21,03,201,,25,28,114,*73
$GPGSV,3,3,09,31,54,221,*42
$GPGSV,3,1,09,02,00,034,,04,11,315,,05,13,056,,09,01,347,*78
$GPGSV,3,2,09,16,16,294,,18,47,159,,21,03,201,,25,28,114,*73

GPS where it do not work :

cat /dev/serial0 | grep GPGSV 

$GPGSV,3,1,11,02,01,034,,04,11,315,,05,13,056,,09,01,347,,0*6C
$GPGSV,3,2,11,16,16,294,,18,47,160,,21,03,201,,25,28,114,28,0*66
$GPGSV,3,3,11,26,54,297,,29,55,058,,31,54,221,,0*58
$GPGSV,3,1,11,02,01,034,,04,11,315,,05,13,056,,09,01,347,,0*6C
$GPGSV,3,2,11,16,16,294,,18,47,160,,21,03,201,,25,28,114,28,0*66
$GPGSV,3,3,11,26,54,297,,29,55,058,,31,54,221,,0*58
VigibotDev commented 4 years ago

I missed Glonass on my grep :

Do not work cat /dev/serial0 | grep GSV $GAGSV,1,1,00,074 $GPGSV,3,1,12,04,09,308,,05,14,049,,09,03,341,,16,23,297,,069 $GPGSV,3,2,12,18,55,153,,20,01,152,,21,11,202,,25,21,119,29,069 $GPGSV,3,3,12,26,63,297,,27,05,252,,29,47,061,,31,47,213,,060 $GLGSV,2,1,08,66,07,345,,67,08,034,,73,52,155,,74,82,274,,073 $GLGSV,2,2,08,75,24,322,,83,36,039,,84,80,233,,85,21,223,,079 $GAGSV,1,1,00,074 $GPGSV,3,1,12,04,09,308,,05,14,049,,09,03,341,,16,23,297,,069 $GPGSV,3,2,12,18,55,153,,20,01,152,,21,11,202,,25,21,119,29,069 $GPGSV,3,3,12,26,63,297,,27,05,252,,29,47,061,,31,47,213,,060 $GLGSV,2,1,08,66,07,345,,67,08,034,,73,52,155,,74,82,274,,073 $GLGSV,2,2,08,75,24,322,,83,36,039,,84,80,233,,85,21,223,,079 $GAGSV,1,1,00,0*74

Work : cat /dev/serial0 | grep GSV $GPGSV,3,3,10,27,05,252,,31,47,213,7C $GLGSV,2,1,08,66,07,345,,67,08,033,,73,51,155,,74,82,272,6D $GLGSV,2,2,08,75,25,322,,83,35,039,,84,81,233,,85,21,223,66 $GPGSV,3,1,10,04,09,308,,05,14,049,,09,03,341,,16,23,297,77 $GPGSV,3,2,10,18,55,153,,20,01,152,,21,11,202,,25,21,119,7C $GPGSV,3,3,10,27,05,252,,31,47,213,7C $GLGSV,2,1,08,66,07,345,,67,08,033,,73,51,155,,74,82,272,6D $GLGSV,2,2,08,75,25,322,,83,35,039,,84,81,233,,85,21,223,66 $GPGSV,3,1,10,04,09,308,,05,14,049,,09,03,341,,16,23,297,77 $GPGSV,3,2,10,18,55,153,,20,01,152,,21,11,202,,25,21,119,7C $GPGSV,3,3,10,27,05,252,,31,47,213,7C $GLGSV,2,1,08,66,07,345,,67,08,033,,73,51,155,,74,82,272,6D $GLGSV,2,2,08,75,25,322,,83,35,039,,84,81,233,,85,21,223,66 $GPGSV,3,1,10,04,09,308,,05,14,049,,09,03,341,,16,23,297,77 $GPGSV,3,2,10,18,55,153,,20,01,152,,21,11,202,,25,21,119,7C $GPGSV,3,3,10,27,05,252,,31,47,213,7C $GLGSV,2,1,08,66,07,345,,67,08,033,,73,51,155,,74,82,272,6D $GLGSV,2,2,08,75,25,322,,83,35,039,,84,81,233,,85,21,223,66 $GPGSV,3,1,10,04,09,308,,05,14,049,,09,03,341,,16,23,297,77 $GPGSV,3,2,10,18,55,153,,20,01,152,,21,11,202,,25,21,119,7C

infusion commented 4 years ago

Hi Pascal,

thanks for reporting this issue. When running your first not working block, I get

{ errors: 0, processed: 1 }
{ errors: 0, processed: 2 }
{
  errors: 0,
  processed: 3,
  satsVisible: [
    { prn: 2, elevation: 1, azimuth: 34, snr: null, status: 'in view' },
    {
      prn: 4,
      elevation: 11,
      azimuth: 315,
      snr: null,
      status: 'in view'
    },
    {
      prn: 5,
      elevation: 13,
      azimuth: 56,
      snr: null,
      status: 'in view'
    },
    {
      prn: 9,
      elevation: 1,
      azimuth: 347,
      snr: null,
      status: 'in view'
    },
    {
      prn: 16,
      elevation: 16,
      azimuth: 294,
      snr: null,
      status: 'in view'
    },
    {
      prn: 18,
      elevation: 47,
      azimuth: 160,
      snr: null,
      status: 'in view'
    },
    {
      prn: 21,
      elevation: 3,
      azimuth: 201,
      snr: null,
      status: 'in view'
    },
    {
      prn: 25,
      elevation: 28,
      azimuth: 114,
      snr: 28,
      status: 'tracking'
    },
    {
      prn: 26,
      elevation: 54,
      azimuth: 297,
      snr: null,
      status: 'in view'
    },
    {
      prn: 29,
      elevation: 55,
      azimuth: 58,
      snr: null,
      status: 'in view'
    },
    {
      prn: 31,
      elevation: 54,
      azimuth: 221,
      snr: null,
      status: 'in view'
    }
  ]
}
{
  errors: 0,
  processed: 4,
  satsVisible: [
    { prn: 2, elevation: 1, azimuth: 34, snr: null, status: 'in view' },
    {
      prn: 4,
      elevation: 11,
      azimuth: 315,
      snr: null,
      status: 'in view'
    },
    {
      prn: 5,
      elevation: 13,
      azimuth: 56,
      snr: null,
      status: 'in view'
    },
    {
      prn: 9,
      elevation: 1,
      azimuth: 347,
      snr: null,
      status: 'in view'
    },
    {
      prn: 16,
      elevation: 16,
      azimuth: 294,
      snr: null,
      status: 'in view'
    },
    {
      prn: 18,
      elevation: 47,
      azimuth: 160,
      snr: null,
      status: 'in view'
    },
    {
      prn: 21,
      elevation: 3,
      azimuth: 201,
      snr: null,
      status: 'in view'
    },
    {
      prn: 25,
      elevation: 28,
      azimuth: 114,
      snr: 28,
      status: 'tracking'
    },
    {
      prn: 26,
      elevation: 54,
      azimuth: 297,
      snr: null,
      status: 'in view'
    },
    {
      prn: 29,
      elevation: 55,
      azimuth: 58,
      snr: null,
      status: 'in view'
    },
    {
      prn: 31,
      elevation: 54,
      azimuth: 221,
      snr: null,
      status: 'in view'
    }
  ]
}
{
  errors: 0,
  processed: 5,
  satsVisible: [
    { prn: 2, elevation: 1, azimuth: 34, snr: null, status: 'in view' },
    {
      prn: 4,
      elevation: 11,
      azimuth: 315,
      snr: null,
      status: 'in view'
    },
    {
      prn: 5,
      elevation: 13,
      azimuth: 56,
      snr: null,
      status: 'in view'
    },
    {
      prn: 9,
      elevation: 1,
      azimuth: 347,
      snr: null,
      status: 'in view'
    },
    {
      prn: 16,
      elevation: 16,
      azimuth: 294,
      snr: null,
      status: 'in view'
    },
    {
      prn: 18,
      elevation: 47,
      azimuth: 160,
      snr: null,
      status: 'in view'
    },
    {
      prn: 21,
      elevation: 3,
      azimuth: 201,
      snr: null,
      status: 'in view'
    },
    {
      prn: 25,
      elevation: 28,
      azimuth: 114,
      snr: 28,
      status: 'tracking'
    },
    {
      prn: 26,
      elevation: 54,
      azimuth: 297,
      snr: null,
      status: 'in view'
    },
    {
      prn: 29,
      elevation: 55,
      azimuth: 58,
      snr: null,
      status: 'in view'
    },
    {
      prn: 31,
      elevation: 54,
      azimuth: 221,
      snr: null,
      status: 'in view'
    }
  ]
}
{
  errors: 0,
  processed: 6,
  satsVisible: [
    { prn: 2, elevation: 1, azimuth: 34, snr: null, status: 'in view' },
    {
      prn: 4,
      elevation: 11,
      azimuth: 315,
      snr: null,
      status: 'in view'
    },
    {
      prn: 5,
      elevation: 13,
      azimuth: 56,
      snr: null,
      status: 'in view'
    },
    {
      prn: 9,
      elevation: 1,
      azimuth: 347,
      snr: null,
      status: 'in view'
    },
    {
      prn: 16,
      elevation: 16,
      azimuth: 294,
      snr: null,
      status: 'in view'
    },
    {
      prn: 18,
      elevation: 47,
      azimuth: 160,
      snr: null,
      status: 'in view'
    },
    {
      prn: 21,
      elevation: 3,
      azimuth: 201,
      snr: null,
      status: 'in view'
    },
    {
      prn: 25,
      elevation: 28,
      azimuth: 114,
      snr: 28,
      status: 'tracking'
    },
    {
      prn: 26,
      elevation: 54,
      azimuth: 297,
      snr: null,
      status: 'in view'
    },
    {
      prn: 29,
      elevation: 55,
      azimuth: 58,
      snr: null,
      status: 'in view'
    },
    {
      prn: 31,
      elevation: 54,
      azimuth: 221,
      snr: null,
      status: 'in view'
    }
  ]
}

That means, it follows the current algorithm: Collect the sats until message number 3 and then set the new collection on state.satsVisible. That means sats 2, 4, 5, 9, 16, 18, 21, 25, 26, 29, 31 are available after the third message, then for message 4 and 5 it remains the same, on message 6 it sets the new collection (which contain the same sats) again. So, I don't see any "blinking" here, if you mean sats appear and disappear regularly. Could you explain it more deeply what behavior you would expect?

The TODO note is, to integrate new sats for each arriving message and add a dirty flag to all other sats that have not updated in the current cycle yet. So implementing it would make the state object a bit more up to date, but I suppose it would force a blinking behavior much more.

VigibotDev commented 4 years ago

I try to capture a longer GPS trace, it's very hard for me because the satsVisible is essentialy empty on the non working GPS.

This GNSS is UBLOX and have the GPS+Glonass+Galileo and NMEA 4.10 enabled.

VigibotDev commented 4 years ago

I managed to write two file, I captured a lot of blink for you !!!

RAW NMEA DATA from serial port : https://www.serveurperso.com/temp/gps.raw.txt

RAW console.log of your gps.state object : https://www.serveurperso.com/temp/gps.state.txt

Please help ! I can make more testing to help debug !

VigibotDev commented 4 years ago

Both files are saved at the same time when there is the problem that appeared :)

VigibotDev commented 4 years ago

I have a realtime website with online robots, some use GPS and show a map with GPS position.

The TRX4 (UBLOX NEO M8N) robot have the GPS sats "blink" problem; The Bubot (old UBLOX NEO 6) robot don't have any "blink" problem.

You can look here there is no ads, it's a non commercial website :)

https://www.vigibot.com/

On TRX4 inside Telemetry (page bottom) the numbers of sats Blink some time:)

infusion commented 4 years ago

Okay, now I see the problem! Interesting problem, it blinks between the glonass and gps sats!

BTW: What a cool website you have there!

VigibotDev commented 4 years ago

Your GPS.js parser live inside all users Raspberry PI robot client and it's easy to auto-update all :) If I can help for the testing/debug feel free to ask here

infusion commented 4 years ago

Okay, v0.6.0 landed. Hope this fixes your problem! :)

VigibotDev commented 4 years ago

You are fast ! Can I use npm update gps.js or I need to clone the git version to test ?

infusion commented 4 years ago

It's published on npm also already

VigibotDev commented 4 years ago

Cool, I just wget https://raw.githubusercontent.com/infusion/GPS.js/master/gps.js on the robot and I get continuous fixed 20 satellites !!!!

VigibotDev commented 4 years ago

In about 2 hours I make a real outdoor test ! because now the GPS is indoor and no fix. only sats in view:) I'm remote connected I can't put my hand on the antenna to check the code !!!

VigibotDev commented 4 years ago

It's okay I waited about 10 minute on the webinterface and I see 19 satellite for few second and return to 20! this is close!

infusion commented 4 years ago

Well, the library shows only what it gets from the receiver. If it changes between 19/20, chances are you saw a satellite change.

VigibotDev commented 4 years ago

Last "problem" npm install --save (empty directory) don't install me the last version. I get the package.json with "gps": "^0.5.3"

infusion commented 4 years ago

--save is obsolete these days. It saves as default. When I test it...:

➜ npm i gps
+ gps@0.6.0
added 1 package from 1 contributor and audited 1 package in 2.277s
found 0 vulnerabilities

Could you test it elsewhere?

VigibotDev commented 4 years ago

Same problem, here is the complete procedure on a PI 4 robot :

root@raspberrypi:/usr/local/vigiclient# ls -lha
total 72K
drwxr-xr-x  2 root root 4.0K Jul 29 16:34 .
drwxr-xr-x 11 root root 4.0K Jun  6 21:25 ..
-rw-r--r--  1 root root  32K Jul 28 05:39 clientrobotpi.js
lrwxrwxrwx  1 root root   15 Jun  6 21:25 processdiffaudio -> /usr/bin/ffmpeg
lrwxrwxrwx  1 root root    8 Jun  6 21:25 processdiffusion -> /bin/cat
lrwxrwxrwx  1 root root   15 Jun  6 21:25 processdiffvideo -> /usr/bin/ffmpeg
-rw-r--r--  1 root root 1.7K Jul 15 20:43 sys.json
-rw-r--r--  1 root root  23K Jul 10 18:37 trame.js
-rwxr-xr-x  1 root root 1.6K Jun 26 23:25 vigiupdate.sh

(Strange "sudo" required for serialport even in root !!!!)

root@raspberrypi:/usr/local/vigiclient# sudo npm i gps i2c-bus pca9685 pigpio serialport socket.io-client stream-split
...
...
...
npm notice created a lockfile as package-lock.json. You should commit this file.
+ stream-split@1.1.0
+ socket.io-client@2.3.0
+ i2c-bus@5.2.0
+ serialport@8.0.8
+ pigpio@3.2.3
+ pca9685@4.0.3
+ gps@0.5.3
added 5 packages from 5 contributors, removed 1 package and updated 19 packages in 58.047s
root@raspberrypi:/usr/local/vigiclient# ls -lha
total 3.8M
drwxr-xr-x   3 root root 4.0K Jul 29 16:36 .
drwxr-xr-x  11 root root 4.0K Jun  6 21:25 ..
-rw-r--r--   1 root root  32K Jul 28 05:39 clientrobotpi.js
drwxr-xr-x 100 root root 4.0K Jul 29 16:36 node_modules
-rw-r--r--   1 root root 3.7M Jun 12 22:12 node_modules.tar.gz
-rw-r--r--   1 root root  741 Jul 29 16:36 package.json
-rw-r--r--   1 root root  36K Jul 29 16:36 package-lock.json
lrwxrwxrwx   1 root root   15 Jun  6 21:25 processdiffaudio -> /usr/bin/ffmpeg
lrwxrwxrwx   1 root root    8 Jun  6 21:25 processdiffusion -> /bin/cat
lrwxrwxrwx   1 root root   15 Jun  6 21:25 processdiffvideo -> /usr/bin/ffmpeg
-rw-r--r--   1 root root 1.7K Jul 15 20:43 sys.json
-rw-r--r--   1 root root  23K Jul 10 18:37 trame.js
-rwxr-xr-x   1 root root 1.6K Jun 26 23:25 vigiupdate.sh
root@raspberrypi:/usr/local/vigiclient# cat package.json
{
  "name": "vigiclient",
  "version": "1.0.0",
  "description": "Raspberry PI robot client for Vigibot.com",
  "main": "clientrobotpi.js",
  "dependencies": {
    "gps": "^0.5.3",
    "i2c-bus": "^5.2.0",
    "pca9685": "^4.0.3",
    "pigpio": "^3.2.3",
    "serialport": "^8.0.8",
    "socket.io-client": "^2.3.0",
    "stream-split": "^1.1.0"
  },
  "devDependencies": {},
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/vigibot/vigiclient.git"
  },
  "author": "Serveurperso",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/vigibot/vigiclient/issues"
  },
  "homepage": "https://github.com/vigibot/vigiclient#readme"
}
infusion commented 4 years ago

Try setting "gps": "^0.6.0" and do and npm update.

VigibotDev commented 4 years ago

It work, But I need one shot install command for automatic scripts that generate the update pre-compiled archive for everyone.

root@raspberrypi:/usr/local/vigiclient# npm update
npm WARN npm npm does not support Node.js v10.19.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
npm WARN npm You can find the latest version at https://nodejs.org/
+ gps@0.6.0
updated 1 package in 4.696s
VigibotDev commented 4 years ago

I do a manual trick to update everyone... Updating gps 0.5.3 to 0.6.0 done ! All GPS work

VigibotDev commented 4 years ago

This is strange, I can put the robot inside a Microwave oven and I always get 20 sats !!! I think Visible sats are remembered by the GPS, I switch to ActiveSats for the realtime view, VisibleSats is not usefull.

infusion commented 4 years ago

Can you confirm, that the sats are not visible in the GSV message? How does the SNR change when you grill your robot?

VigibotDev commented 4 years ago

For now I can only confirm the VisibleSats never cleaned when I put the robot inside a non reception zone (microwave).

I can make you two dump tonight like previsous when I move the robot.

infusion commented 4 years ago

That would be nice, I double checked the code and after 3sec it should clear all visible sats that have not be seen anymore.

VigibotDev commented 4 years ago

I cannot test for now because UBLOX M8N I received does not remember the settings, it seems a forgery:( I ordered other to try them....

infusion commented 3 years ago

Please re-open if it is still relevant.