End to End Ingestion!

[jayceslesar]

Closes #18

This is a slightly beefy PR, but formalizes the API into authenticating, and validating the user is authenticated in all steps possible as well as enables users to stream demos to the db...

First we need to install.

Clone and CD into the repo and then install:

pdm sync

Ensure you have docker installed. Export the following environment variables: (The docker-compose.yml will pick up on these)

export PG_USER=demouser
export PG_PASS=test
docker compose up -d

Run database migrations, this will ensure that the DB in docker is up to date with the current revision:

pdm run alembic --name demos upgrade head

Run the litestar app and head to the provision

pdm run litestar --app-dir src/api run --debug

Then open it in your browser of choice and head to http://127.0.0.1:8000/provision, sign in with steam, and save your API key that it spits out. You will need this.

In a new file, paste in the following:

import requests

api_key = YOUR_API_KEY_HERE
session_id = ...

app_host = "127.0.0.1:8000"

response = requests.get(f"http://{app_host}/session_id", params={
    "api_key": api_key,
    "fake_ip": "123.45.6789",
    "map": "asdf"
})

print(response.json())

Paste the session_id in the response into your session_id variable above

Go ahead and connect to the database, I use pgcli but you can use whatever

 pgcli postgresql://${PG_USER}:${PG_PASS}@localhost:5432/demos

some sample queries:

select * from api_keys;

select * from demo_sessions;

Now, go ahead and either add to the existing python file you made or make a new one:

import websockets
import asyncio

async def send_demo_file():
    uri = f"ws://127.0.0.1:8000/demos?api_key={api_key}&session_id={session_id}"
    async with websockets.connect(uri) as socket:
        with open("some_demo_file.dem", "rb") as f:
            while True:
                chunk = f.read(1024)
                print(chunk)
                if not chunk:
                    break
                await socket.send(chunk)

async def test_demo_streaming() -> None:
    """Test that a demo is completely received by the API and sunk to a file."""
    await send_demo_file(session_id, api_key)

async def main():

    await test_demo_streaming()

# Run the event loop
if __name__ == "__main__":
    asyncio.run(main())

Once this is done, you will see your byte data in the demo field of the demo_sessions table!

I would eventually like to add tests back but anything at this point is a full integration test which is fine, but will just suck 2 hours of my time up to write...

[jayceslesar]

@carterwward ping for review

[carterwward]

So from my understanding this PR adds API authentication and a demo stream endpoint to the anti-cheat developer API, does that sound right @jayceslesar ? Wanting to make sure I understand what I am looking at and reading.

[jayceslesar]

So from my understanding this PR adds API authentication and a demo stream endpoint to the anti-cheat developer API, does that sound right @jayceslesar ? Wanting to make sure I understand what I am looking at and reading.

Yeah that is correct -- some auth existed but this formalizes into litestar guards, which are executed before any code actually described in the endpoint(s) they belong to. I would argue the same thing that flask offers in decorators but no funny explicit names needed

[carterwward]

So from my understanding this PR adds API authentication and a demo stream endpoint to the anti-cheat developer API, does that sound right @jayceslesar ? Wanting to make sure I understand what I am looking at and reading.

Yeah that is correct -- some auth existed but this formalizes into litestar guards, which are executed before any code actually described in the endpoint(s) they belong to. I would argue the same thing that flask offers in decorators but no funny explicit names needed

Makes sense, the litestar docs are much appreciated.