search

jaiarobotics / jaiabot

Jaiabot source code
Other
20 stars 2 forks source link

Feature/app for api/jaia-1460, jaia-1517, jaia-1550, jaia-1459, jaia-1514, jaia-1516 #925

Closed tsaubergine closed 3 weeks ago

tsaubergine commented 2 months ago

Web endpoints

This PR implements a new REST API for accessing fleet data and sending commands. It is intended to supercede the preliminary API in the JCC server, which has been renamed to "/jaia/v0" to avoid conflict when running the new REST API and JCC on the same port, that is "/jaia/status" -> "/jaia/v0/status", etc.

This PR implements https://jaia-innovation.atlassian.net/browse/JAIA-1463 and most child issues, except as noted below in Follow-on work

Merged PRs

This PR merges in the following separate PRs for final review:

Follow-on work

Hosting

The Apache config has been updated to serve /jaia using this new REST API for ports 80 and 443 (https), and /jaia/v0 and / with the JCC server on port 80 (http only).

REST API - Overview

The new API directly introspects from the new APIRequest Protobuf message for input and APIResponse for output. It accepts GET requests for the subset of requests that only have singular (non-repeated) non-recursive (no submessages) data. This is generally suitable for simpler requests (e.g. Stop all bots, query status, etc) but not for more complicated MISSION_PLAN type commands.

It also accepts POST for all requests, using JSON (which will be required for the more complicated messages).

api_key can be omitted for local connections, if we want.

There are 2 variants of the API:

Bot and Hub Status

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

10.23.1.11/jaia/status

New Input

10.23.1.11:9092/jaia/v1/status/all?api_key=4vS6s2jnulxVjrKSB-__tQ

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": True}, "status": True, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Task Packets

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

10.23.1.11/jaia/task-packets?startDate="2023-10-18 09:04:00"&endDate="2023-10-22 09:04:00"

New Input

10.23.1.11:9092/jaia/v1/task_packets/all?start_date="2023-10-18 09:04:00"&end_date="2023-10-22 09:04:00&api_key=4vS6s2jnulxVjrKSB-__tQ

OR (POST)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1/task_packets/all', json={ "start_date": "2023-10-18 09:04:00", "end_date": "2023-10-22 09:04:00", "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": True}, "task_packets": { "start_date": "2023-10-18 09:04:00", "end_date": "2023-10-22 09:04:00" }, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Task Packet Count

This section is out of date, use jaia doc rest_api for the correct precise syntax

Removed (though could be added in if important)

Metadata

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

10.23.1.11/jaia/metadata

New Input

(for hub1 metadata)

10.23.1.11:9092/jaia/v1/metadata/h1?api_key=4vS6s2jnulxVjrKSB-__tQ

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"hubs": [1]}, "metadata": True, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Stop All Bots

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

# importing the requests library
import requests

API_ENDPOINT = "http://10.23.1.11/jaia/all-stop"

# define the headers for the request
headers = {'clientid': 'backseat-control', 'Content-Type' : 'application/json; charset=utf-8'}

resp = requests.post(url=API_ENDPOINT, headers=headers)

# extracting response text
pastebin_url = resp.text
print("The pastebin URL is:%s"%pastebin_url)

New Input

10.23.1.11:9092/jaia/v1/command/all?type=STOP&api_key=4vS6s2jnulxVjrKSB-__tQ

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": True}, "command": { "type": "STOP"}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Activate All Bots

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

# importing the requests library
import requests

API_ENDPOINT = "http://10.23.1.11/jaia/all-activate"

# define the headers for the request
headers = {'clientid': 'backseat-control', 'Content-Type' : 'application/json; charset=utf-8'}

resp = requests.post(url=API_ENDPOINT, headers=headers)

# extracting response text
pastebin_url = resp.text
print("The pastebin URL is:%s"%pastebin_url)

New Input

10.23.1.11:9092/jaia/v1/command/all?type=ACTIVATE&api_key=4vS6s2jnulxVjrKSB-__tQ

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": True}, "command": { "type": "ACTIVATE"}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Single Waypoint Mission

This section is out of date, use jaia doc rest_api for the correct precise syntax

Old Input

# importing the requests library
import requests

API_ENDPOINT = "http://10.23.1.11/jaia/single-waypoint-mission"

# define the headers for the request
headers = {'clientid': 'backseat-control', 'Content-Type' : 'application/json; charset=utf-8'}

data = {'bot_id': 1, 'lat': 41.661849, 'lon': -71.273131, 'dive_depth': 2, 'surface_drift_time': 15,'transit_speed': 2.5, 'station_keep_speed': 0.5}

resp = requests.post(url=API_ENDPOINT, json=data, headers=headers)

# extracting response text
pastebin_url = resp.text
print("The pastebin URL is:%s"%pastebin_url)

New Input

Note that this is trivially extendable to a multiple waypoint mission by repeating "goal"

~Can't use GET as we need to pass embedded message with repeated fields~

(POST)

#!/usr/bin/env python3
import requests
res = requests.post('http://127.0.0.1:5000/jaia/v1/command/b1', json={"type":"MISSION_PLAN","plan":{"goal":[{"location":{"lat":41.661849,"lon":-71.273131},"task":{"type":"DIVE","dive":{"max_depth":2},"surface_drift":{"drift_time":15}}}],"speeds":{"transit":2.5,"stationkeep_outer":0.5}}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

OR (POST full APIRequest)

#!/usr/bin/env python3
import requests
res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"bots": [1]}, "command": {"type":"MISSION_PLAN","plan":{"goal":[{"location":{"lat":41.661849,"lon":-71.273131},"task":{"type":"DIVE","dive":{"max_depth":2},"surface_drift":{"drift_time":15}}}],"speeds":{"transit":2.5,"stationkeep_outer":0.5}}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

Testing

To test first start a simulation:

cd jaiabot/config/launch/simulation
./all.launch

Then start the REST API code with a key (this key is used in the test scripts):

export JAIA_REST_API_PRIVATE_KEY=4vS6s2jnulxVjrKSB-__tQ
cd jaiabot/src/web/rest_api
./run.sh

Then run either of tests

cd jaiabot/src/web/rest_api/test
./test.sh

or simply use a web browser to test the long API (GET).

jason-jaia commented 1 month ago

Is v1 short for API version 1?

jason-jaia commented 1 month ago

What is hub metadata? And how does it relate to metadata that gets recorded into the goby logs? As a user, I guess I would expect it to be the same or close to.

tsaubergine commented 1 month ago

Is v1 short for API version 1?

Yes

tsaubergine commented 1 month ago

What is hub metadata? And how does it relate to metadata that gets recorded into the goby logs? As a user, I guess I would expect it to be the same or close to.

Same message. We don't currently send the bot metadata over XBee so this isn't available at runtime but we could do it over WiFi in the future.

jason-jaia commented 1 month ago

What's the syntax to activate one bot only?

jason-jaia commented 1 month ago

Would it be

!/usr/bin/env python3

import requests res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": False, "bots": [1]}, "command": { "type": "ACTIVATE"}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

?

tsaubergine commented 1 month ago

Would it be

!/usr/bin/env python3 import requests res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"all": False, "bots": [1]}, "command": { "type": "ACTIVATE"}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})

?

Exactly, though you can omit "all": False as that's the default in the protobuf if not included:

10.23.1.11:9092/jaia/v1/command/b1?type=ACTIVATE&api_key=4vS6s2jnulxVjrKSB-__tQ

or 

res = requests.post('http://10.23.1.11:9092/jaia/v1', json={"target": {"bots": [1]}, "command": { "type": "ACTIVATE"}, "api_key": "4vS6s2jnulxVjrKSB-__tQ"})
edsanville commented 1 month ago

I forgot to mention the 3 endpoint URLs mentioned in the JED app.

tsaubergine commented 1 month ago

Thanks for updating those!

ferrom-jaia commented 1 month ago

jaia doc rest_api is not working for me.


ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ sudo apt install jaiabot-doc
[sudo] password for ferrom:
Reading package lists... Done
Building dependency tree
Reading state information... Done
jaiabot-doc is already the newest version (1.11.0+46+gb3285e16-0~ubuntu20.04.1).
The following packages were automatically installed and are no longer required:
  docker-scan-plugin linux-headers-5.4.0-144 linux-headers-5.4.0-144-generic linux-headers-5.4.0-146 linux-headers-5.4.0-146-generic linux-headers-5.4.0-150
  linux-headers-5.4.0-150-generic linux-headers-5.4.0-155 linux-headers-5.4.0-155-generic linux-headers-5.4.0-156 linux-headers-5.4.0-156-generic linux-headers-5.4.0-163
  linux-headers-5.4.0-163-generic linux-headers-5.4.0-167 linux-headers-5.4.0-167-generic linux-headers-5.4.0-170 linux-headers-5.4.0-170-generic linux-headers-5.4.0-174
  linux-headers-5.4.0-174-generic linux-headers-5.4.0-176 linux-headers-5.4.0-176-generic linux-headers-5.4.0-177 linux-headers-5.4.0-177-generic linux-headers-5.4.0-182
  linux-headers-5.4.0-182-generic linux-tools-5.4.0-163 linux-tools-5.4.0-163-generic linux-tools-5.4.0-167 linux-tools-5.4.0-167-generic linux-tools-5.4.0-170
  linux-tools-5.4.0-170-generic linux-tools-5.4.0-174 linux-tools-5.4.0-174-generic linux-tools-5.4.0-176 linux-tools-5.4.0-176-generic linux-tools-5.4.0-177
  linux-tools-5.4.0-177-generic linux-tools-5.4.0-182 linux-tools-5.4.0-182-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)```
tsaubergine commented 1 month ago

jaia doc rest_api is not working for me.

ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ sudo apt install jaiabot-doc
[sudo] password for ferrom:
Reading package lists... Done
Building dependency tree
Reading state information... Done
jaiabot-doc is already the newest version (1.11.0+46+gb3285e16-0~ubuntu20.04.1).
The following packages were automatically installed and are no longer required:
  docker-scan-plugin linux-headers-5.4.0-144 linux-headers-5.4.0-144-generic linux-headers-5.4.0-146 linux-headers-5.4.0-146-generic linux-headers-5.4.0-150
  linux-headers-5.4.0-150-generic linux-headers-5.4.0-155 linux-headers-5.4.0-155-generic linux-headers-5.4.0-156 linux-headers-5.4.0-156-generic linux-headers-5.4.0-163
  linux-headers-5.4.0-163-generic linux-headers-5.4.0-167 linux-headers-5.4.0-167-generic linux-headers-5.4.0-170 linux-headers-5.4.0-170-generic linux-headers-5.4.0-174
  linux-headers-5.4.0-174-generic linux-headers-5.4.0-176 linux-headers-5.4.0-176-generic linux-headers-5.4.0-177 linux-headers-5.4.0-177-generic linux-headers-5.4.0-182
  linux-headers-5.4.0-182-generic linux-tools-5.4.0-163 linux-tools-5.4.0-163-generic linux-tools-5.4.0-167 linux-tools-5.4.0-167-generic linux-tools-5.4.0-170
  linux-tools-5.4.0-170-generic linux-tools-5.4.0-174 linux-tools-5.4.0-174-generic linux-tools-5.4.0-176 linux-tools-5.4.0-176-generic linux-tools-5.4.0-177
  linux-tools-5.4.0-177-generic linux-tools-5.4.0-182 linux-tools-5.4.0-182-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)```

Well you'll need jaiabot-doc from the test repo at the moment, or just use -Dbuild_doc=ON to cmake (then make) to show the docs from the local git clone.

ferrom-jaia commented 1 month ago

jaia doc rest_api is not working for me.

ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ sudo apt install jaiabot-doc
[sudo] password for ferrom:
Reading package lists... Done
Building dependency tree
Reading state information... Done
jaiabot-doc is already the newest version (1.11.0+46+gb3285e16-0~ubuntu20.04.1).
The following packages were automatically installed and are no longer required:
  docker-scan-plugin linux-headers-5.4.0-144 linux-headers-5.4.0-144-generic linux-headers-5.4.0-146 linux-headers-5.4.0-146-generic linux-headers-5.4.0-150
  linux-headers-5.4.0-150-generic linux-headers-5.4.0-155 linux-headers-5.4.0-155-generic linux-headers-5.4.0-156 linux-headers-5.4.0-156-generic linux-headers-5.4.0-163
  linux-headers-5.4.0-163-generic linux-headers-5.4.0-167 linux-headers-5.4.0-167-generic linux-headers-5.4.0-170 linux-headers-5.4.0-170-generic linux-headers-5.4.0-174
  linux-headers-5.4.0-174-generic linux-headers-5.4.0-176 linux-headers-5.4.0-176-generic linux-headers-5.4.0-177 linux-headers-5.4.0-177-generic linux-headers-5.4.0-182
  linux-headers-5.4.0-182-generic linux-tools-5.4.0-163 linux-tools-5.4.0-163-generic linux-tools-5.4.0-167 linux-tools-5.4.0-167-generic linux-tools-5.4.0-170
  linux-tools-5.4.0-170-generic linux-tools-5.4.0-174 linux-tools-5.4.0-174-generic linux-tools-5.4.0-176 linux-tools-5.4.0-176-generic linux-tools-5.4.0-177
  linux-tools-5.4.0-177-generic linux-tools-5.4.0-182 linux-tools-5.4.0-182-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
ferrom@LAPTOP-AH3AN14A:~/repos/jaiabot$ jaia doc rest_api
Could not find any markdown files - ensure that the jaiabot-doc package is installed (sudo apt install jaiabot-doc)```

Well you'll need jaiabot-doc from the test repo at the moment, or just use -Dbuild_doc=ON to cmake (then make) to show the docs from the local git clone.

Yes, thank you for pointing that out. I was able to get this to work with your directions.