search

guidopetri / chess-pipeline

Pulling games from the Lichess API into a PostgreSQL database for data analysis.
GNU General Public License v3.0
20 stars 2 forks source link

Some table SQLs don't match actual tables #56

Closed guidopetri closed 3 years ago

guidopetri commented 3 years ago

e.g. promotions are different in the .sql and in the code. I should probably re-check the .sql to make sure it's consistent, as well as find some way of ensuring that it remains consistent.

guidopetri commented 3 years ago

It looks like the only table where this happens is chess_games. Not sure how this happened. It also looks like some columns should be not-null but are null? I'm guessing that was because there is information missing there that I can't set the null constraint, but I also don't want to set a default.

Is there any way of cross-referencing something like this? I have an .sql and a .py file that refer to the same thing but in different ways... I don't think there's a way of making this DRY. Maybe this should just be part of protocol somehow?

I can also see maybe moving the CopyGames task to its own file, and then somehow making sure that if that file is modified, so is a file in the table-sql/ folder.

For now I think I'm just going to fix this .sql. If this happens again, I definitely need to open a more encompassing issue to stop it from happening.

guidopetri commented 3 years ago

Fixed in a9d2841 .