search

hickey / meshchat

MeshChat for AREDN (in Lua)
GNU General Public License v3.0
21 stars 3 forks source link

[Feature]: Drop the extra API only package and use sysinfo.json to get the same data #14

Open aanon4 opened 1 year ago

aanon4 commented 1 year ago

Contact Details

tim.j.wilkinson@gmail.com

Enhancement Type

API improvements

What is your idea or what can be improved?

The generic sysinfo.json interface in AREDN can be used to get all the same information the current api-only package provides. I assume that approach predates it. Might be good to dump the api-only and switch over.

hickey commented 1 year ago

@gerner and I were just discussing this basic concept this evening. It will take a bit of work, but looks doable. Thanks for the thought.

gerner commented 1 year ago

Just a few notes:

The sysinfo.json api takes several 0/1 valued query params to get extra info. Here's an example (that I've edited for brevity) from a node on my local network where I've asked for "link_info" which is, as of a few weeks ago, just a processed form of the olsrd API (shown below):

http://192.168.68.106/cgi-bin/sysinfo.json?link_info=1

{
  "lon": "-122.38914047574394",
  "sysinfo": {
    "uptime": "1 days, 22:11:31",
    "loads": [
      0.14999999999999999,
      0.14000000000000001,
      0.16
    ]
  },
  "interfaces": [
    {
      "name": "br0.3",
      "mac": "94:83:C4:2D:46:A8"
    },
    {
      "mac": "94:83:C4:2D:46:A8",
      "name": "br-lan",
      "ip": "10.106.53.57"
    }
  ],
  "api_version": "1.11",
  "lat": "47.574943408790766",
  "meshrf": {
    "ssid": "AREDN-10-v3",
    "channel": "-2",
    "status": "on",
    "freq": "2397",
    "chanbw": "10"
  },
  "tunnels": {
    "active_tunnel_count": "1"
  },
  "link_info": {
    "10.203.61.200": {
      "helloTime": 0,
      "lostLinkTime": 0,
      "linkQuality": 0.68600000000000005,
      "vtime": 20000,
      "linkCost": 1.4921880000000001,
      "linkType": "RF",
      "hostname": "KK7LZM-cpe210-1",
      "validityTime": 33216,
      "previousLinkStatus": "SYMMETRIC",
      "currentLinkStatus": "SYMMETRIC",
      "rx_rate": 57.799999999999997,
      "neighborLinkQuality": 0.97599999999999998,
      "noise": -95,
      "symmetryTime": 13653,
      "seqnoValid": false,
      "lossMultiplier": 65536,
      "pending": false,
      "lossHelloInterval": 2000,
      "asymmetryTime": 166135078,
      "tx_rate": 39,
      "hysteresis": 0,
      "seqno": 0,
      "lossTime": 664,
      "expected_throughput": 16,
      "olsrInterface": "wlan0",
      "lastHelloTime": 0,
      "signal": -65
    }
  },
  "grid_square": "CN87tn",
  "node": "KK7LZM-ar300m16-2",
  "node_details": {
    "model": "GL.iNet GL-AR300M16",
    "mesh_gateway": "0",
    "firmware_mfg": "AREDN",
    "board_id": "GL.iNet GL-AR300M16",
    "firmware_version": "3.23.8.0"
  }
}

and the olsrd api. That's not exposed on the WAN interface by default, so I had to ssh into the node to get this:

http://KK7LZM-ar300m16-2.local.mesh:9090/links

{
    "pid": 3479,
    "systemTime": 1700067245,
    "timeSinceStartup": 166475840,
    "configurationChecksum": "3999fcf6",
    "links": [
        {
            "localIP": "10.45.70.167",
            "remoteIP": "10.203.61.200",
            "olsrInterface": "wlan0",
            "ifName": "wlan0",
            "validityTime": 38228,
            "symmetryTime": 18665,
            "asymmetryTime": 166494505,
            "vtime": 20000,
            "currentLinkStatus": "SYMMETRIC",
            "previousLinkStatus": "SYMMETRIC",
            "hysteresis": 0,
            "pending": false,
            "lostLinkTime": 0,
            "helloTime": 0,
            "lastHelloTime": 0,
            "seqnoValid": false,
            "seqno": 0,
            "lossHelloInterval": 2000,
            "lossTime": 1665,
            "lossMultiplier": 65536,
            "linkCost": 1.023438,
            "linkQuality": 1,
            "neighborLinkQuality": 0.976
        }
    ]
}

The sysinfo.json script just hits this olsrd api and processes the output to fillout the link_info section. It has some other features you can turn on to get even more info. It's pretty nifty.

hickey commented 1 year ago

I am probably missing a couple of things, but the other night I played with the OLSR API and found I could not get enough information from it to support MeshChat. It would probably help if there was a modicum of documentation for the API.

I dug through the source to find the API endpoints to query them, but I could only find information about the services being broadcasted off the local node not all the services on the mesh. Without seeing all the services on the mesh there is no way to interrogate for other nodes serving the same MeshChat zone.

The only advantage I see (currently) to using the OLSR API is for self-discovery of the the MeshChat zone. Sort of unfortunate as this would have been a great way to remove the need for producing the API package.

It would be great if someone could show me that I have misunderstood something. I will go back through the OLSR source to see if I can find a way to retrieve the services information for the entire mesh.

gerner commented 1 year ago

What do you need the from the API? It would help me to understand the scenario or at least have a few bullet points of the key pieces of info. I appreciate the effort.

Is this something that's available in the sysinfo.json API is it? If it's info that's more generally useful for any AREDN node, or the kind of thing that any service running on an AREDN node might need or want to advertise, and it's not already in sysinfo.json, an update there might be worth while.

hickey commented 1 year ago

In looking through the code, I only see /var/run/services_olsr being read. It is actually read in two locations (one in meshchat and the other in meshchatlib.lua) but they are really doing nearly the same thing. So there is some refactoring to do.

So it appears the only real thing that is being pulled from OLSR is a list of nodes that are running the MeshChat zone. The way this is done is to look at every service that is listed in the OLSR database and look for matches to "http://:/meshchat|tcp|.*". The host and port are then saved for the matches and used in a few spots. Most specifically this info is useful for meshchatsync.

gerner commented 1 year ago

If all you need is where lines get read from /var/run/services_olsr, which looks like code in the api like:

print("Content-type: text/plain\r")
print("\r")
if query.action == "meshchat_nodes" then
    local pattern = "http://(%S+):(%d+)/meshchat|tcp|" .. str_escape(query.zone_name) .. "%s"
    for line in io.lines("/var/run/services_olsr")
    do
        local node, port = line:match(pattern)
        if node and port then
            print(node .. "\t" .. port)
        end
    end
else
    print("error no action")
end

then I think you can get the same info here:

curl localhost:9090/

{
  "pid": 3488,
  "systemTime": 1700370926,
  "timeSinceStartup": 709936,
  "configurationChecksum": "3999fcf6",
  "plugins": [
    {
      "plugin": "olsrd_nameservice.so.0.4",
      "parameters": {
        "service": "http://KK7LZM-ar300m16-2:8080/meshchat|tcp|MeshChat-W7AW",
        "10.45.70.168": "dtdlink.KK7LZM-ar300m16-2.local.mesh",
        "name": "KK7LZM-ar300m16-2",
        "name-change-script": "/usr/local/bin/olsrd-namechange",
        "timeout": "300",
        "interval": "30",
        "sighup-pid-file": "/var/run/dnsmasq/dnsmasq.pid"
      }
    }
  ]
}

I heavily edited the response here. It includes a TON of other info.

hickey commented 1 year ago

Unfortunately, not quite.

If I look at the services_olsr file and just grep out "meshchat" I get the following:

http://WT0F-1:8080/meshchat|tcp|FLMeshChat  # my own service
http://kl7aameshpi:80/meshchat|tcp|AKMeshChat   #10.214.202.154
http://W3VN-HAP-101-KNOX:8080/meshchat|tcp|TNMeshChat   #10.159.118.116
http://WT0F-VM:8080/meshchat|tcp|FLMeshChat #10.7.15.63
http://KD4WLE-Tik2:8080/meshchat|tcp|FlMeshChat #10.156.253.68
http://K1KY-HAP-450-P2:8080/meshchat|tcp|MeshChat   #10.214.177.59
http://K4PWO-CPE510-101-MBORO:8080/meshchat|tcp|TNMeshChat  #10.14.14.190
http://KI4LMR-RKM5XW-110-LOVE-130TWR-235:8080/meshchat|tcp|TNMeshChat   #10.170.45.72
http://K1KY-HAP-300-P1:8080/meshchat|tcp|TNMeshChat #10.44.105.52
http://KM4TBQ-HAP-101-KNOX:8080/meshchat|tcp|TNMeshChat #10.110.43.244
http://KD4KCD-Server:80/meshchat|tcp|TNMeshchat #10.218.186.10
http://AK4GO-RKM5-102-LIPSC-SE:8080/meshchat|tcp|MeshChat   #10.252.204.79
http://KA1TOM-HAP-103-LAKE:8080/meshchat|tcp|TNMeshChat #10.190.165.242
http://K6QS-RKM5-N106VU-SW-253:8080/meshchat|tcp|MeshChat   #10.170.174.211
http://KL7PMC-PMCN-RKM33G26-355:8080/meshchat|tcp|AKMeshChat    #10.44.55.80
http://KD4WLE-Titusville:8080/meshchat|tcp|FlMeshChat   #10.60.23.87
http://KS1O-Server2:80/meshchat|tcp|TNMeshChat  #10.66.205.64
http://n2mh-web:80/meshchat|tcp|- MeshChat-WX   #10.73.73.65
http://KL7PMC-PMCS-RKM33G26-145:8080/meshchat|tcp|AKMeshChat    #10.44.55.81
http://N4OEM-HAP-100-METRO-OEM:8080/meshchat|tcp|TNMeshChat #10.146.227.127
http://KA1TOM-HAP-250-KNOX:8080/meshchat|tcp|MeshChat-5133  #10.155.210.96
http://KL7AA-SUMMIT-RKM3SEC120-310:8080/meshchat|tcp|AKMeshChat #10.70.220.248
http://KC4JIR-1-HP-Laptop:8080/meshchat|tcp|MeshChat-4013   #10.118.96.122
http://K1KY-GLUSB150-600-MR2:8080/meshchat|tcp|TNMeshChat   #10.68.15.30
http://ab4nx-arednpi:80/meshchat|tcp|AtlMeshChat    #10.183.75.128
http://N4TDX-Gate1:8080/meshchat|tcp|FlMeshChat #10.235.209.76
http://kc4jir-arednpi:80/meshchat|tcp|GAMeshChat    #10.5.11.198
http://K1KY-AR750-750-P1:8080/meshchat|tcp|TNMeshChat   #10.20.81.183
http://kv4atv-meshchatpi:80/meshchat|tcp|WX-NWS-SE-SKYWARN  #10.110.87.172

That is everything that is on the mesh that I am connected to. In reality the code will filter down to the MeshChat zone that it is part of and just return a couple of the above.

If I grep for "meshchat" in what is returned from the OLSR API, I get the following:

        "service": "http://WT0F-1:8080/meshchat|tcp|FLMeshChat",

Now the OLSR API does return entries for every node on the network and one could go query every OLSR API on the network to determine which ones have a MeshChat that is a member of the specific zone, but that would be an incredible waste of resources and cause the performance to go straight into a burning dumpster.

gerner commented 1 year ago

How about this one?

http://192.168.68.106/cgi-bin/sysinfo.json?services=1

{
  "lon": "-122.38914047574394",
  "sysinfo": {
    "uptime": "0 days, 12:08:25",
    "loads": [
      0.20999999999999999,
      0.17000000000000001,
      0.16
    ]
  },
  "services": [
    {
      "ip": "10.45.70.167",
      "name": "MeshChat-W7AW",
      "protocol": "tcp",
      "link": "http:\/\/KK7LZM-ar300m16-2:8080\/meshchat"
    },
    {
      "ip": "10.165.194.150",
      "name": "N6LKA MeshChat",
      "protocol": "tcp",
      "link": "http:\/\/N6LKA-VM-1:80\/meshchat"
    },
    {
      "ip": "10.154.68.14",
      "name": "MeshChat-W7AW",
      "protocol": "tcp",
      "link": "http:\/\/N7OEP-1-SEATTLE:8080\/meshchat"
    },
  ],
}

I only included part of the response to illustrate multiple nodes running meshchat showing up.

This comes from here: https://github.com/aredn/aredn/blob/main/files/www/cgi-bin/sysinfo.json#L158-L160 Which pulls from here: https://github.com/aredn/aredn/blob/main/files/usr/lib/lua/aredn/info.lua#L313-L347 That's pulling from /var/run/services_olsr

If you add a link_info=1 query param, you'll get the mapping from ip address to host name.