search

codebuddies / backend

CodeBuddies back-end
https://codebuddies.org
GNU General Public License v3.0
20 stars 25 forks source link

Relocate the Postgres container volume so it is no longer versioned #52

Closed bengineerdavis closed 4 years ago

bengineerdavis commented 4 years ago

Actions

Notes

fatal: pathspec 'db_data' did not match any files

After eliminating local Postgres volume, current repo size seems primarily derived from these directories within the django project directory: ./resources ./users ./templates

Overview

Over on slack, we noticed that the current repo is about 73mb, which seems considerably larger than the amount of data we've put into the app.

Steps

  1. I first searched for venv in .git, which didn't return anything.

  2. Then I searched recursively for the largest sized directories/files in our repo and with sudo ls -lR | sudo du -ah | sort -n -r | head -n 30

The results:

964K ./cbv3_django_prototype 736K ./.git/objects/pack/pack-af3bbc530aded22c05b12ddc6bd8e00d44bef547.idx 680K ./cbv3_django_prototype/cbv3_django_prototype 608K ./db_data/base/16384/1255 608K ./db_data/base/13117/1255 608K ./db_data/base/13116/1255 608K ./db_data/base/1/1255 584K ./db_data/global 496K ./db_data/base/16384/1249 472K ./db_data/base/16384/2608 448K ./db_data/base/13117/2608 448K ./db_data/base/13116/2608 448K ./db_data/base/1/2608 408K ./db_data/base/16384/2838 408K ./db_data/base/13117/2838 408K ./db_data/base/13116/2838 408K ./db_data/base/1/2838 400K ./db_data/base/16384/2674 392K ./db_data/base/13117/1249 392K ./db_data/base/13116/1249 392K ./db_data/base/1/1249 368K ./db_data/base/13117/2674 368K ./db_data/base/13116/2674 368K ./db_data/base/1/2674 360K ./db_data/base/16384/2673 328K ./db_data/base/16384/2609 328K ./db_data/base/13117/2673 328K ./db_data/base/13117/2609 328K ./db_data/base/13116/2673 328K ./db_data/base/13116/2609

I had to run sudo because db_data is the local volume associated with our db Postgres container, which will always made modifications to files with privileged access. I spoke with @BethanyG and @angelocordon, and we agreed that this volume should never be in the git-versioned files.

Machine

Fedora 30 (Linux)

Options

.gitignore da_data

However, this is more of a bandaid, since the volume is still in the project directory.

Let docker-compose abstract the local volume setup

This will make access to the volume files trickier, but can remove a manual config step that we might not need to worry about anyway.

bengineerdavis commented 4 years ago

Currently, .git isn't tracking db_data volume. I'll have to dig some more to determine what is causing the repo size.

UPDATE:

After drilling down to django-concept/cbv3_django_prototype/cbv3_django_prototype, and then running sudo du -ah . | sort -n -r | head -n 20 produced the following result:

680K . 196K ./resources 140K ./users 116K ./templates 88K ./resources/migrations 80K ./userauth 72K ./contrib 68K ./templates/account 56K ./contrib/sites 52K ./static 48K ./resources/migrations/pycache 44K ./resources/fixtures 40K ./resources/fixtures/resources.json 40K ./contrib/sites/migrations 32K ./users/pycache 32K ./userauth/pycache 28K ./users/tests 28K ./resources/pycache 24K ./users/migrations 20K ./users/fixtures

We should then explore ./resources ./users ./templates.

billglover commented 4 years ago

This has puzzled me for a while. I couldn't find a copy of db_data within the repo. It did (for a while) exist in the project folder, but didn't make it into the project history.

When you run the following command (I removed sudo):

ls -lR | du -ah | sort -n -r | head -n 30

This returns the largest directories in the project folder. It doesn't take into account whether or not these files are part of the git repository.

The commands in issue #53 remove unnecessary files in the project history. Running them reveals that the large repository size appears to have come from the inadvertent inclusion of a python virtual environment in the project repository.

@bengineerdavis I propose closing this issue and using #53 to address the large repository size.

bengineerdavis commented 4 years ago

Thanks for closing this. And I realized that, too, but should've followed up here.

On Sun, Feb 2, 2020, 3:58 PM Bill Glover notifications@github.com wrote:

Closed #52 https://github.com/codebuddies/django-concept/issues/52.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/codebuddies/django-concept/issues/52?email_source=notifications&email_token=AMQU4TW552KUKR7IT5LJZNDRA4XWVA5CNFSM4KJ3BPVKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWLKOSUA#event-3000297808, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMQU4TTHH5OA24RBU6DBHWDRA4XWVANCNFSM4KJ3BPVA .