search

EBISPOT / gwas-sumstats-service

Summary statistics service
2 stars 0 forks source link

readme

gwas-sumstats-service

Codacy Badge Build Status Codacy Badge

GWAS summary statistics service app

This handles the uploaded summary statistics files, validates them, reports errors to the deposition app and puts valid files in the queue for sumstats file harmonisation and HDF5 loading.

Local installation

Requirements

Installation

Run the tests

Run as a flask app

Run with Docker-compose

This section guides you through using Docker-compose to set up and run the gwas-sumstats-service with all necessary services, including Flask, RabbitMQ, Celery, and MongoDB.

Prerequisites

Steps to Run

  1. Replace the local Dockerfile and docker-compose file with Dockerfile and docker-compose.yaml, respectively.

  2. Build the Docker Containers

    Navigate to the cloned directory and build the Docker containers:

    docker-compose build

  3. Start the Docker Containers

    Spin up the Flask, RabbitMQ, Celery, and MongoDB containers:

    docker-compose up

Additional Configuration

Deploy with helm (kubernetes)

Testing

Testing with Postman

This section provides instructions on how to test the gwas-sumstats-service using Postman. The Postman collection for this service includes requests for submitting summary statistics and retrieving their validation status. Please find the collection here.

Pre-requisites

Testing Steps

  1. Submit Summary Statistics

    • Use the POST {{protocol}}://{{host}}:{{port}}/v1/sum-stats request to submit summary statistics.
    • Update the id field in the request body with a unique identifier. Example body for a valid file submission: 
      {
"requestEntries": [
 {
   "id": "{{callbackId}}",
   "filePath": "test_sumstats_file.tsv",
   "md5": "9b5f307016408b70cde2c9342648aa9b",
   "assembly": "GRCh38",
   "readme": "optional text",
   "entryUUID": "ABC1234",
   "minrows": "2"
 }
]
}
    • For an invalid file submission, modify the filePath and other relevant fields accordingly.
    • Note the returned callbackID from the response for the next step.

  2. Retrieve Validation Status

    • Use the GET {{protocol}}://{{host}}:{{port}}/v1/sum-stats/<callbackID> request to retrieve the status of your submission.
    • Replace <callbackID> with the ID obtained from the previous POST request.
    • The response will indicate the validation status of the submission.

Debugging Invalid Submissions

Postman Collection Details

Testing with Curl

Example POST method

curl -i -H "Content-Type: application/json" -X POST -d '{"requestEntries":[{"id":"abc123","filePath":"https://raw.githubusercontent.com/EBISPOT/gwas-sumstats-service/master/tests/test_sumstats_file.tsv","md5":"a1195761f082f8cbc2f5a560743077cc","assembly":"GRCh38", "readme":"optional text", "entryUUID": "globusdir"},{"id":"bcd234","filePath":"https://raw.githubusercontent.com/EBISPOT/gwas-sumstats-service/master/tests/test_sumstats_file.tsv","md5":"a1195761f082f8cbc","assembly":"GRCh38", "entryUUID": "globusdir"}]}' http://localhost:8000/v1/sum-stats

HTTP/1.0 201 CREATED
Content-Type: application/json
Content-Length: 26
Server: Werkzeug/0.15.4 Python/3.6.5
Date: Wed, 17 Jul 2019 15:15:23 GMT

{"callbackID": "TiQS2yxV"}

Example GET method (using callback id from above)

curl http://localhost:8000/v1/sum-stats/TiQS2yxV

{
  "callbackID": "TiQS2yxV",
  "completed": false,
  "statusList": [
    {
      "id": "abc123",
      "status": "VALID",
      "error": null
    },
    {
      "id": "bcd234",
      "status": "INVALID",
      "error": "md5sum did not match the one provided"
    }
  ]
}

Misc - Format and Lint

Installation

Follow these steps to set up FormatLint:

1. Create a Virtual Environment

Create a new virtual environment for the project to manage dependencies separately from your global Python setup:

python -m venv formatlint

Activate the virtual environment:

source formatlint/bin/activate

2. Install Dependencies

Install the required Python packages:

pip install -r requirements.dev.txt

3. Run FormatLint

Execute the formatting and linting script:

./format-lint