SidDar360 / solarWindsInternWork

Solarwinds repo for internship work
0 stars 0 forks source link

Create HTTP server called Ingester for receiving logs #5

Open peterinfra opened 7 months ago

peterinfra commented 7 months ago

This is a set of tasks that will result in having a HTTP server with a POST endpoint that can receive logs from a HTTP client and write to file on disk. This task is divided into several subtasks

  1. Create OpenAPI specification for the endpoints -> Think of what endpoints POST /logs GET ? PUT ? And what will be request payload, headers and response header, payload etc. -> Send PR for review before you write any code
  2. Create a directory and give it a good name. Create a Python project inside it with requirements.txt, Dockerfile ad a README for what the code does and how does one use it.
  3. After the OpenAPI spec is approved, implement the API specification using a feature branch . FastAPI is preferred but you can write it in any framework you wish. Send PR. First phase, receive the log lines (also called events) and write to a timestamped file on disk (e.g: UTC_timestamp_per_hour.log 20223-12-20:15:00.log)
  4. Write unit tests and Github Actions for the same. (Create a task for researching how to setup Github Actions and run on PR is approved for merge and after code is committed to main/master. As a part of the PR show unit test runs.
  5. Use flog to create a JSON templated log file and write it to disk.
  6. Create a Python program that reads the file and sends each line to the Ingester over HTTP using the API endpoint that was created. Ensure no errors in response from the Ingester server.
peterinfra commented 6 months ago

create github issues for each of the above bullet points so that you can close them one by one.