Closed tayloramurphy closed 1 year ago
@tayloramurphy I added it to the list, following the Airflow UI command.
FYI @pnadolny13 moved to a post-2.0 effort. Not essential for the launch.
Also https://github.com/matsonj/nba-monte-carlo is an excellent example. Deployed on www.mdsinabox.com
I'm also working in https://github.com/archiewood/analytics_monorepo
gitlab ci
# The Docker image that will be used to build your app
pages:
stage: run
image:
name: $CI_REGISTRY_IMAGE:latest
entrypoint: [""]
variables:
MELTANO_ENVIRONMENT: prod
POSTGRES_DB: $TAP_POSTGRES_HOST
POSTGRES_USER: $TAP_POSTGRES_HOST #Evidence.dev
POSTGRES_PASSWORD: $TAP_POSTGRES_HOST #Evidence.dev
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_HOST: $TAP_POSTGRES_HOST #Evidence.dev
POSTGRES_DATABASE: $TAP_POSTGRES_HOST #Evidence.dev
DATABASE: $TAP_POSTGRES_HOST #Evidence.dev
POSTGRES_PORT: 5432 #Evidence.dev
# Functions that should be executed before the build script is run
script:
- cd /project/my-project
- meltano run dashboard
- apt update -y
- apt install curl -y
- curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
- apt install nodejs -y
- npm install
- npm run build
- cp -r build/ ${CI_PROJECT_DIR}/public
artifacts:
paths:
# The folder that contains the files to be exposed at the Page URL
- public
rules:
# This ensures that only pushes to the default branch will trigger
# a pages deploy
- if: ($CI_PIPELINE_SOURCE == "schedule" || $CI_PIPELINE_SOURCE == "web") && $mode == "dashboard"
services:
- postgres:12.2-alpine
I'm using Evidence.dev alongside Meltano but not as a plugin yet, just with the Vercel-GitHub integration: https://github.com/edgarrmondragon/meltano-dataops/
@edgarrmondragon and I also talked briefly about supporting Evidence via nodejs-pypi. This could be called directly in commands (I.e. declaring npm
as the executable) or alternatively we could make an Evidence plugin with the EDK that proxies Evidence commands 'natively' (allowing meltano invoke evidence run dev
rather than meltano invoke evidence:run-dev
) 🙂
👆had a stab at this: meltanolabs/evidence-ext. It's currently a working MVP. Would be good to figure out config injection from Meltano before adding to the Hub.
We should add Evidence.dev to the Hub as a reporting tool.
Evidence.dev is a BI/reporting tool that works very well in CI/CD environments.
Evidence uses Markdown as the reporting definition language, with jinja template support and specific extensions for rendering charts and visuals.
Related links:
A few example project using Evidence with Meltano: