AMI Data Management API

Overview

This project provides an API to push data (images and audio files) into an S3 server. The API is built with FastAPI and includes endpoints for managing AMI system deployments and their data.

Features

• Upload images and audio files to an S3 bucket.
• Manage deployment information.
• Automatically generated API documentation with Swagger UI and ReDoc.

Installation

Prerequisites

• Python 3.9+
• conda (optional, for creating a virtual environment)

Steps

1. Create a virtual environment:

   conda create -n ami-api python=3.9
   conda activate ami-api

2. Clone the repository:

   git clone https://github.com/AMI-system/ami-api.git
   cd ami-api

3. Install dependencies:

   pip install -e .

4. Create credentials.json: Create a file named credentials.json in the root folder with the following content:

   {
    "AWS_ACCESS_KEY_ID": "your_access_key_id",
    "AWS_SECRET_ACCESS_KEY": "your_secret_access_key",
    "AWS_REGION": "your_region",
    "AWS_URL_ENDPOINT": "your_endpoint"
   }

5. Add deployments_info.csv: Add the file named deployments_info.csv with the information about your AMI deployments.

Running the Application locally

Start the application using Uvicorn:

uvicorn main:app --port 8080 --reload

API Endpoints

Documentation

• Swagger UI: http://localhost:8080/docs
• ReDoc: http://localhost:8080/redoc

Data management

• Generate presigned url: Endpoint for creating unique urls for pushing images and audio files to the server.

  POST /generate-presigned-url/

  Form Data:

  • name: string
  • country: string
  • deployment: string
  • data_type: string
  • filename: string
  • file_type: string

• Upload Data: Endpoint for pushing images and audio files to the server. The maximum number of files allowed each time is 1,000, to avoid crashing the server's memory.

  POST /upload/

  Form Data:

  • name: string
  • country: string
  • deployment: string
  • data_type: string
  • files: files

• Ckeck if file exist: Endpoint for checking if file already exists in the bucket.

  POST /check-file-exist/

  Form Data:

  • name: string
  • country: string
  • deployment: string
  • data_type: string
  • filename: string

Deployments

• Get Deployments: Endpoint to retrieve all deployment information.

  GET /get-deployments/

• Create Deployment: Endpoint to create a new deployment.

  POST /create-deployment/

  Body (JSON):

  {
  "country": "Country Name",
  "country_code": "Country Code",
  "location_name": "Location Name",
  "lat": "Latitude",
  "lon": "Longitude",
  "camera_id": "Camera ID",
  "hardware_id": "Hardware ID",
  "status": "inactive"
  }

• Update Deployment: Endpoint to update a deployment information.

  PUT /update-deployment/

  Body (JSON):

  {
  "country": "Country Name",
  "country_code": "Country Code",
  "location_name": "Location Name",
  "lat": "Latitude",
  "lon": "Longitude",
  "location_id": "Location ID",
  "camera_id": "Camera ID",
  "system_id": "System ID",
  "hardware_id": "Hardware ID",
  "deployment_id": "Deployment ID",
  "status": "inactive"
  }

Other Operations

• List Data: Endpoint for retrieving the list of files available for a particular deployment.

  GET /list-data/

  Query Parameters:

  • country_location_name: string (format: "Country - Location Name")
  • data_type: string (one of "motion_images", "snapshot_images", "audible_recordings", "ultrasound_recordings")

• Get Logs: Endpoint for downloading the logs from a bucket in the S3 server. Everytime a user push some new data to the server, the log file is update with some information: date and time, username, country, deployment, data type and filename.

  GET /get-logs/

  Query Parameters:

  • country_location_name: string (format: "Country - Location Name")
  • data_type: string (one of "motion_images", "snapshot_images", "audible_recordings", "ultrasound_recordings")

• Create Bucket: Endpoint to create a new bucket in the S3 server. In our case, bucket are countries.

  POST /create-bucket/

  Body (JSON):

  {
  "bucket_name": "your_bucket_name"
  }

Contributing

Feel free to fork this repository and create a pull request. For major changes, please open an issue first to discuss what you would like to change.

License

This project is licensed under the Apache 2.0 License.

Contact

For more information, visit UKCEH AMI System or contact the team at ami-system@ceh.ac.uk.