search

videre-project / monorepo

The home of all things Videre Project.
https://videreproject.com
Apache License 2.0
1 stars 0 forks source link

Where is the data from? #1

Open j6e opened 1 week ago

j6e commented 1 week ago

Hi,

I came across your project from this website: https://observablehq.com/@qonfused/mtg-metagame

I was wondering where were you obtaining the data, since I couldn't found the ETLs for it.

Thanks!

j6e commented 1 week ago

(tagging @CodyJasonBennett and @Qonfused for visibility)

Qonfused commented 1 week ago

You can find the API under services/@videre-api, though the underlying event data is extracted from MTGO as they get broadcasted to the client.

All of the data for that notebook is queried from three API endpoints:

I have been working with the MTGO Daybreak team to try and open source the bot that extracts this data or at least parts of it as part of the MTGOSDK project, though for now I can't release the source code for it. Once the green light is lit, it'll be available under https://github.com/videre-project/MTGOBot.

There is however a shared database instance that is hosted on Neon -- the plan for this instance is to provide read-only replicas once cross-region replication is added to Neon sometime Q4-Q1. This is the same data queried by the API and used by the notebook, though it is only populated with events since 2023/12/23.