AIP-38 | Add Pool Summary stats to Dashboard

bbovenzi commented 1 month ago

Now that we have Get Pools in FastAPI and the UI Dashboard page. Let's add the Pool Slots Summary component:

We should aggregate information across all pools to have a combined total for each slot type. Each section can have a tooltip to say what slot type it is and across how many pools, if greater than one.

LefterisXefteris commented 1 month ago

Hello @bbovenzi ! I would like to work on this issue, if that's okay.

bbovenzi commented 1 month ago

@LefterisXefteris go for it

LefterisXefteris commented 1 month ago

Thank you very much!

LefterisXefteris commented 2 weeks ago

Hello @bbovenzi! I'm working on this issue and I can now dedicate time to complete it. I notice you mentioned a modern web application, but when I run the development servers (frontend with yarn dev in airflow/www/ and backend with airflow webserver), I'm only seeing the old web interface with the navigation bar at the top. Could you please guide me on how to access the modern web application interface? I want to ensure I'm working on the correct UI version for this implementation.

bbovenzi commented 2 weeks ago

The new UI code is in airflow/ui and you only have to run breeze start-airflow --dev-mode and the new UI will be at localhost:29091 with hot reload already working

bbovenzi commented 2 weeks ago

Make sure to pull down the latest from main. I just did some big changes to the UI on Friday

LefterisXefteris commented 2 weeks ago

Hello @bbovenzi ! I've set up the new development environment following the instructions in dev/breeze/README.md. Current status:

✅ Successfully implemented:

Breeze installation and configuration using the new UV-based workflow
Created UV virtual environment and synced dependencies from dev/breeze/uv.lock
Legacy UI accessible at :28080
Postgres backend running

❌ Issue: Receiving 404 Not Found when attempting to access the FastAPI UI at localhost:29091.

Current setup:


# Environment
breeze start-airflow --dev-mode OR
breeze start-airflow --dev-mode --python 3.10 --use-uv --backend postgres

# Running containers show proper port mapping
$ docker ps
CONTAINER ID   IMAGE                                              PORTS                                                                                              
d16ac27f695e   ghcr.io/apache/airflow/main/ci/python3.10:latest   0.0.0.0:29091->9091/tcp

bbovenzi commented 2 weeks ago

When you run breeze start-airflow the other services are running but not the FastAPI one. Is that right?

If so, can you try to if there are any errors in the terminal? You can also just run fastapi by running breeze and then inside of breeze running airflow fastapi-api

LefterisXefteris commented 2 weeks ago

When running Airflow in the Breeze development environment, there's an inconsistency between the webserver and FastAPI services accessibility:

Working

Legacy UI (webserver) is accessible at http://localhost:28080
Successfully authenticating with credentials
Webserver service starts correctly with airflow webserver command
- API documentation IS accessible at http://localhost:29091/docs`

Not Working

FastAPI endpoints return 404 responses when accessing through http://localhost:29091
Service starts without errors using airflow fastapi-api command

Environment Details

Using Breeze development environment
Python virtual environment configured
PostgreSQL backend
Default authentication configured

Questions

Are additional development dependencies required? Specifically:
- pip install -e ".[devel-all]"
- Specific provider packages
- Hatch build system dependencies

Attempted Steps

Started services via Breeze environment
Started FastAPI service with airflow fastapi-api

Expected Behavior

FastAPI endpoints should be accessible at `http://localhost:29091with proper authentication.

Logs

404 - not found

bbovenzi commented 2 weeks ago

Wait, have you navigated to http://localhost:29091/webapp or http://localhost:29091/docs?

LefterisXefteris commented 2 weeks ago

When I navigate to /docs, it works perfectly. However, when I try to access http://localhost:29091/webapp, I get an Internal Server Error.

bbovenzi commented 2 weeks ago

And then looking at your logs in the terminal running breeze? does it explain the internal error at all? Also, feel free to message me on the Airflow slack for faster back and forth debugging help

LefterisXefteris commented 2 weeks ago

this is what i get at the log:

172.18.0.1:57800 - "GET /webapp HTTP/1.1" 307 172.18.0.1:57800 - "GET /webapp/ HTTP/1.1" 500

Key Error: jinja2.exceptions.TemplateNotFound: /index.html

Location: File "/opt/airflow/airflow/api_fastapi/core_api/app.py", line 76, in webapp return templates.TemplateResponse("/index.html", {"request": request}, media_type="text/html")

bbovenzi commented 1 week ago

We did resolve this on slack. Manually building the UI once with pnpm install && pnpm build created the /dist directory necessary for fastapi to host the UI's html file.

apache / airflow