search

rippinrobr / baseball-stats-db

Tools to help create and maintain databases based on the Baseball Databank Files
Apache License 2.0
10 stars 2 forks source link

Build in Docker Environment #31

Closed wesrice closed 5 years ago

wesrice commented 6 years ago

Hey Rob, great work on this!

I've been trying to make releases locally, but I'm having a few issues.

I'm new to Go, and had a bit of trouble of getting everything installed properly (go get ./internal/... ./cmd/databank-dbloader/... ./cmd/retrosched-dbloader/... ./cmd/retrogl-dbloader/... finally got me where I needed to be).

Additionally, I've had to alter -inputdir arguments in the Makefile to utilize environment variables so I can customize where data sources live on my machine.

I still haven't successfully compiled a release yet, but the further I get into this, I'm thinking that Dockerizing this tool might be useful for getting anyone up and running quickly. This would allow for a portable Go installation, -inputdir values could all be relative to installation paths within the Docker container, and db types could even have their own Docker image (sqlite, postgres, mysql, mongodb).

I'm willing to put some time into this and work on PR, but wanted to get your thoughts first.

rippinrobr commented 6 years ago

@wesrice dockerizing and using env vars sound like great ideas. I'm open to a PR for sure. Just out of curiosity what compiler errors are you receiving?

wesrice commented 6 years ago

I was actually able to compile the binaries, however, I'm stuck getting errors when running make release_all, starting with make sqlitedb.

Here is my working branch. I added directions in the README on how to get up and running with the Docker setup. After following those steps, you can run make sqlitedb to see the issue that I'm running into. Below is the output that I experience.

$ make sqlitedb
loading baseballdatabank db
./bin/databank-dbloader -dbtype sqlite -dbpath=baseball_databank_2017.sqlite3 -inputdir /opt/baseballdatabank/core
2018/06/10 04:48:21 People ==> Truncating
2018/06/10 04:48:21 truncate err: no such table: people
2018/06/10 04:48:21 People ==> Inserting 19478 Records
2018/06/10 04:48:21 Insert error: no such table: people
[followed by 19477 more "Insert error: no such table: people" errors]
rippinrobr commented 6 years ago

@wesrice before running the loader run this command to setup your schema

sqlite3 testing-for-wes.db < <path to the cloned repo>/baseball-stats-db/schemas/sqlite_databank_schema_2017.7.sql

That should fix that issue. I'm going to add an issue to create a nicer, more helpful error when this happens.

I'm going to pull down your fork and check out your docker work. Let me know if you have any other problems.

rippinrobr commented 6 years ago

@wesrice I tried to submit a PR on your fork but I was unable to do so. I've added a line to your Dockerfile that will load the Sqlite database schema before building the loader. Here's the line.

RUN sqlite3 ./baseball_databank_2017.sqlite3 < ./schemas/sqlite_databank_schema_2017.7.sql