issues
search
VMois
/
miwaitway
Calculate average wait time on bus stops using GTFS real-time vehicle location and display it on a map
https://vmois.dev/miwaitway-average-wait-time-on-stop/
2
stars
0
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Consider ingesting data from MobilityData
#30
VMois
opened
6 days ago
0
Realtime ingestor improvements
#29
VMois
closed
1 month ago
0
Improve reliability of vehicle position ingestor
#28
VMois
closed
1 month ago
0
Rotate logs in Airflow, keep only last 30 days
#27
VMois
opened
1 month ago
0
Remove unnecessary stage table check in load_realtime_to_bq DAG
#26
VMois
opened
1 month ago
1
Calculate the average wait time at bus stops
#25
VMois
opened
1 month ago
4
Transform data from raw to prod
#24
VMois
closed
1 month ago
0
Me no likey big bills, protect bucket
#23
VMois
opened
1 month ago
0
If hash of vehicle positions data does not change after certain limit, emit a warning message
#22
VMois
opened
1 month ago
0
Transform raw data to prod to be used in features later
#21
VMois
closed
1 month ago
0
Sort vehicle positions CSV files by creation date to ingest the latest data
#20
VMois
opened
1 month ago
0
Resolving duplication issues in raw data
#19
VMois
closed
1 month ago
0
Resolving duplication issues in raw data
#18
VMois
closed
1 month ago
0
Set up Sentry for Airflow monitoring
#17
VMois
opened
2 months ago
0
Add realtime GTFS vehicle position ingestor
#16
VMois
closed
2 months ago
0
Revamp pipeline that loads raw realtime vehicle position CSV files from GCS to BQ
#15
VMois
closed
2 months ago
0
Add partitioning and partition expiry to raw realtime vehicle position based on timestamp of the event
#14
VMois
closed
2 months ago
1
Mount a path for a service JSON file to Airflow for GCP connection instead of providing a JSON file
#13
VMois
opened
2 months ago
0
Move SQL from Python files into a separate .sql file in load realtime DAG
#12
VMois
closed
2 months ago
0
Format raw static pipeline with Ruff and remove raw realtime vehicle position 24/7 ingestion DAG
#11
VMois
closed
2 months ago
0
Import raw static GTFS data from MiWay website
#10
VMois
closed
2 months ago
0
Set up Github CI pipeline for code quality
#9
VMois
opened
2 months ago
0
Move realtime gathering of data from Airflow to a separate small container that runs 24/7
#8
VMois
closed
2 months ago
0
Make raw static and real-time bus position data available in BigQuery
#7
VMois
closed
2 months ago
0
Deploy Airflow on VM
#6
VMois
closed
2 months ago
0
Enforce column types when fetch real-time vehicle location to insure it doesnt change accidentally due to Pandas conversion.
#5
VMois
opened
2 months ago
0
Using MERGE when importing real-time data doesn't make much sense because you cannot access historical data
#4
VMois
opened
2 months ago
1
Prepare for deploy
#3
VMois
closed
3 months ago
0
Deploy Airflow on Google Cloud Compute
#2
VMois
closed
3 months ago
0
Add basic DAG to ingest realtime data of vehicle location
#1
VMois
closed
3 months ago
0