search

babybuddy / babybuddy

A :baby: buddy to help caregivers track sleep, feedings, diaper changes, tummy time and more to learn about and predict baby's needs without (as much) guess work.
http://docs.baby-buddy.net/
BSD 2-Clause "Simplified" License
2.1k stars 259 forks source link

500 Server Error on Weight Reports #729

Closed jschollenberger closed 11 months ago

jschollenberger commented 11 months ago

BabyBuddy is giving me a 500 Server Error when trying to view the weight report and the new percentile reports. 

OperationalError at /children/baby/reports/weight/weight/
no such table: core_weightpercentile
Request Method: GET
Request URL:    https://babybuddy.domain.com/children/baby/reports/weight/weight/
Django Version: 4.2.6
Exception Type: OperationalError
Exception Value:    
no such table: core_weightpercentile
Exception Location: /lsiopy/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py, line 328, in execute
Raised during:  reports.views.WeightChangeChildReport
Python Executable:  /lsiopy/bin/python3
Python Version: 3.11.6
Python Path:    
['/app/www/public',
 '/lsiopy/bin',
 '/usr/lib/python311.zip',
 '/usr/lib/python3.11',
 '/usr/lib/python3.11/lib-dynload',
 '/lsiopy/lib/python3.11/site-packages',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf',
 '/lsiopy/lib/python3.11/site-packages/odf']
Server time:    Sat, 21 Oct 2023 08:22:56 -0400

I see a migration file which seems to be responsible for creating this table "0030_weightpercentile_weightpercentile_unique_age_sex.py". Are you supposed to manually run migrations when upgrading to newer versions of babybuddy, or is this an indication something has gone wrong? I'm running inside a local docker container using the linuxserver image.

I see in a different issue, the following was recommended, although I'm not sure why a different migration was recommended. I will run it if this is the appropriate fix:

python manage.py migrate core 0029_alter_pumping_options_remove_pumping_time_and_more
python manage.py migrate

Thanks!

cdubz commented 11 months ago

I think the LSIO image runs migrations automatically but I'm not actually sure.

Couple of questions:

  1. Do you recall what version you were updating from?
  2. Could you provide the output of python manage.py migrate --check?
  3. Could you provide the output of python manage.py showmigrations?

I see in a different issue, the following was recommended, although I'm not sure why a different migration was recommended.

This is over at https://github.com/babybuddy/babybuddy/issues/722#issuecomment-1764409732 where the reporting user had an issue with data units and I found what was apparently a separate issue with the demo thinking it had run the migration but somehow missing it. I was hoping it was just some weird one-off issue with the demo but maybe not...

What those commands are doing is resetting the database back to the previous migration (0029_alter_pumping_options_remove_pumping_time_and_more) and then running migrations again. This worked for the demo and should also work for you here but I'll appreciate if you can answer the Qs above before doing that so I can see if I can get this to be reproducible.

jschollenberger commented 11 months ago

Looking at the logs from the container, I think this is the issue:

2023/10/20 08:52:56 stdout  ModuleNotFoundError: No module named 'dotenv'
2023/10/20 08:52:56 stdout      from dotenv import load_dotenv, find_dotenv
2023/10/20 08:52:56 stdout    File "/app/www/public/manage.py", line 4, in <module>
2023/10/20 08:52:56 stdout  Traceback (most recent call last):
2023/10/20 08:52:56 stdout  ModuleNotFoundError: No module named 'dotenv'
2023/10/20 08:52:56 stdout      from dotenv import load_dotenv, find_dotenv
2023/10/20 08:52:56 stdout    File "/app/www/public/manage.py", line 4, in <module>
2023/10/20 08:52:56 stdout  Traceback (most recent call last):

Presumably this is the app trying to perform the migration, and failing. I see the same error when running it manually.

I upgraded from version v2.0.4

cdubz commented 11 months ago

Well that's curious... what output do you get from pip list?

jschollenberger commented 11 months ago 
root@linuxserver-babybuddy:/# pip list
Package              Version
-------------------- ------------
annotated-types      0.6.0
asgiref              3.7.2
boto3                1.28.62
botocore             1.31.62
defusedxml           0.7.1
diff-match-patch     20230430
dj-database-url      2.1.0
Django               4.2.6
django-appconf       1.0.5
django-axes          6.1.1
django-dbsettings    1.3.0
django-filter        23.3
django-imagekit      5.0.0
django-import-export 3.3.1
django-qr-code       3.1.1
django-storages      1.14.1
django-taggit        4.0.0
django-widget-tweaks 1.5.0
djangorestframework  3.14.0
et-xmlfile           1.1.0
Faker                19.6.2
gunicorn             21.2.0
jmespath             1.0.1
MarkupPy             1.14
mysqlclient          2.2.0
odfpy                1.4.1
openpyxl             3.1.2
packaging            23.2
pilkit               3.0
Pillow               10.0.1
pip                  23.3
plotly               5.17.0
psycopg2-binary      2.9.9
pydantic             2.4.2
pydantic_core        2.10.1
python-dateutil      2.8.2
python-dotenv        1.0.0
pytz                 2023.3.post1
PyYAML               6.0.1
s3transfer           0.7.0
segno                1.5.2
setuptools           68.2.2
six                  1.16.0
sqlparse             0.4.4
tablib               3.5.0
tenacity             8.2.3
typing_extensions    4.8.0
uritemplate          4.1.1
urllib3              2.0.6
wheel                0.41.2
whitenoise           6.5.0
xlrd                 2.0.1
xlwt                 1.3.0

[notice] A new release of pip is available: 23.3 -> 23.3.1
[notice] To update, run: python3 -m pip install --upgrade pip
cdubz commented 11 months ago

Hm yeah that all looks normal... are you able to run the other commands I asked about in https://github.com/babybuddy/babybuddy/issues/729#issuecomment-1773787816? Or were you saying those logs were from trying to run those commands?

jschollenberger commented 11 months ago

The logs were from stdout in the container manager (Synology), but I get the same output when run manually:

root@linuxserver-babybuddy:/app/www/public# python manage.py migrate --check
Traceback (most recent call last):
  File "/app/www/public/manage.py", line 4, in <module>
    from dotenv import load_dotenv, find_dotenv
ModuleNotFoundError: No module named 'dotenv'
cdubz commented 11 months ago

Ah that may clue in to the actual issue... I keep forgetting to document this but when running the commands in the container you need to set up some env vars. Add these and then try the commands again:

export DJANGO_SETTINGS_MODULE="babybuddy.settings.base"
export SECRET_KEY="$(cat /config/.secretkey)"
cdubz commented 11 months ago

And another question -- what procedure did you use when updating? I'm not super familiar with Docker or container management but when testing with a basic Docker Compose config I was able to update from the LSIO 2.0.4 to 2.1.2 tag and get the correct migrations:

chris@hedonismbot  ~/lsio  docker-compose up     
Creating network "lsio_default" with the default driver
Pulling babybuddy (lscr.io/linuxserver/babybuddy:2.0.4)...
2.0.4: Pulling from linuxserver/babybuddy
f651f4c6bcbf: Pull complete
07a0e16f7be1: Pull complete
b59506af66ce: Pull complete
b6d2984da964: Pull complete
b43394b42ea8: Pull complete
e18de8c366cf: Pull complete
dc144fcf22d9: Pull complete
cc3e310c9f0a: Pull complete
15cae78a7c22: Pull complete
Digest: sha256:dabb84129e403d8cf010b3295d7b697227f2823281d900231fa1cd10c666ff48
Status: Downloaded newer image for lscr.io/linuxserver/babybuddy:2.0.4
Creating babybuddy ... done
Attaching to babybuddy
babybuddy    | [migrations] started
babybuddy    | [migrations] 01-nginx-site-confs-default: executing...
babybuddy    | [migrations] 01-nginx-site-confs-default: succeeded
babybuddy    | [migrations] 02-default-location: executing...
babybuddy    | [migrations] 02-default-location: succeeded
babybuddy    | [migrations] done
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    |       ██╗     ███████╗██╗ ██████╗ 
babybuddy    |       ██║     ██╔════╝██║██╔═══██╗
babybuddy    |       ██║     ███████╗██║██║   ██║
babybuddy    |       ██║     ╚════██║██║██║   ██║
babybuddy    |       ███████╗███████║██║╚██████╔╝
babybuddy    |       ╚══════╝╚══════╝╚═╝ ╚═════╝ 
babybuddy    | 
babybuddy    |    Brought to you by linuxserver.io
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | To support the app dev(s) visit:
babybuddy    | BabyBuddy: https://github.com/sponsors/cdubz
babybuddy    | 
babybuddy    | To support LSIO projects visit:
babybuddy    | https://www.linuxserver.io/donate/
babybuddy    | 
babybuddy    | ───────────────────────────────────────
babybuddy    | GID/UID
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | User UID:    1000
babybuddy    | User GID:    1000
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | Setting resolver to  127.0.0.11
babybuddy    | Setting worker_processes to 4
babybuddy    | generating self-signed keys in /config/keys, you can replace these with your own keys if required
babybuddy    | ...+.+..+.............+..+.+++++++++++++++++++++++++++++++++++++++*...+.+..+.+++++++++++++++++++++++++++++++++++++++*.+....+.....+..........+...+..............++++++
babybuddy    | ...+.....+.........+....+.....+......+....+..+.+...+.....+......+.+..+++++++++++++++++++++++++++++++++++++++*.+.......+.....+...+......+......+.......+........+.......+.....+.+.....+....+.....+......+.+.....+....+.....+.+.........+..+.........+.+..+..........+......+...........+.+..+.......+...+.....+...+...+....+++++++++++++++++++++++++++++++++++++++*.........+....+........................+......+.....+..........+......+.........++++++
babybuddy    | -----
babybuddy    | **** No secret key found, generating one ****
babybuddy    | Operations to perform:
babybuddy    |   Apply all migrations: admin, auth, authtoken, axes, babybuddy, contenttypes, core, dbsettings, sessions
babybuddy    | Running migrations:
babybuddy    |   Applying contenttypes.0001_initial... OK
babybuddy    |   Applying auth.0001_initial... OK
babybuddy    |   Applying admin.0001_initial... OK
babybuddy    |   Applying admin.0002_logentry_remove_auto_add... OK
babybuddy    |   Applying admin.0003_logentry_add_action_flag_choices... OK
babybuddy    |   Applying contenttypes.0002_remove_content_type_name... OK
babybuddy    |   Applying auth.0002_alter_permission_name_max_length... OK
babybuddy    |   Applying auth.0003_alter_user_email_max_length... OK
babybuddy    |   Applying auth.0004_alter_user_username_opts... OK
babybuddy    |   Applying auth.0005_alter_user_last_login_null... OK
babybuddy    |   Applying auth.0006_require_contenttypes_0002... OK
babybuddy    |   Applying auth.0007_alter_validators_add_error_messages... OK
babybuddy    |   Applying auth.0008_alter_user_username_max_length... OK
babybuddy    |   Applying auth.0009_alter_user_last_name_max_length... OK
babybuddy    |   Applying auth.0010_alter_group_name_max_length... OK
babybuddy    |   Applying auth.0011_update_proxy_permissions... OK
babybuddy    |   Applying auth.0012_alter_user_first_name_max_length... OK
babybuddy    |   Applying authtoken.0001_initial... OK
babybuddy    |   Applying authtoken.0002_auto_20160226_1747... OK
babybuddy    |   Applying authtoken.0003_tokenproxy... OK
babybuddy    |   Applying axes.0001_initial... OK
babybuddy    |   Applying axes.0002_auto_20151217_2044... OK
babybuddy    |   Applying axes.0003_auto_20160322_0929... OK
babybuddy    |   Applying axes.0004_auto_20181024_1538... OK
babybuddy    |   Applying axes.0005_remove_accessattempt_trusted... OK
babybuddy    |   Applying axes.0006_remove_accesslog_trusted... OK
babybuddy    |   Applying axes.0007_alter_accessattempt_unique_together... OK
babybuddy    |   Applying axes.0008_accessfailurelog... OK
babybuddy    |   Applying babybuddy.0001_initial... OK
babybuddy    |   Applying babybuddy.0002_add_settings... OK
babybuddy    |   Applying babybuddy.0003_add_refresh_help_text... OK
babybuddy    |   Applying babybuddy.0004_settings_language... OK
babybuddy    |   Applying babybuddy.0005_auto_20190502_1701... OK
babybuddy    |   Applying babybuddy.0006_auto_20190502_1744... OK
babybuddy    |   Applying babybuddy.0007_auto_20190607_1422... OK
babybuddy    |   Applying babybuddy.0008_auto_20200120_0622... OK
babybuddy    |   Applying babybuddy.0009_settings_timezone... OK
babybuddy    |   Applying babybuddy.0010_auto_20200609_0649... OK
babybuddy    |   Applying babybuddy.0011_auto_20200813_0238... OK
babybuddy    |   Applying babybuddy.0012_auto_20201024_1847... OK
babybuddy    |   Applying babybuddy.0013_auto_20210411_1241... OK
babybuddy    |   Applying babybuddy.0014_settings_hide_empty... OK
babybuddy    |   Applying babybuddy.0015_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0016_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0017_settings_hide_age... OK
babybuddy    |   Applying babybuddy.0018_auto_20211017_2136... OK
babybuddy    |   Applying babybuddy.0019_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0020_update_language_en_to_en_us... OK
babybuddy    |   Applying babybuddy.0021_alter_settings_language... OK
babybuddy    |   Applying babybuddy.0022_alter_settings_language... OK
babybuddy    |   Applying babybuddy.0023_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0025_auto_20221023_0821... OK
babybuddy    |   Applying babybuddy.0026_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0027_remove_standard_group... OK
babybuddy    |   Applying babybuddy.0028_alter_settings_language_alter_settings_timezone... OK
babybuddy    |   Applying babybuddy.0029_create_cache_table... OK
babybuddy    |   Applying dbsettings.0001_initial... OK
babybuddy    |   Applying core.0001_initial... OK
babybuddy    |   Applying core.0002_auto_20171028_1257... OK
babybuddy    |   Applying core.0003_weight... OK
babybuddy    |   Applying core.0004_child_picture... OK
babybuddy    |   Applying core.0005_auto_20190416_2048... OK
babybuddy    |   Applying core.0006_auto_20190502_1701... OK
babybuddy    |   Applying core.0007_temperature... OK
babybuddy    |   Applying core.0008_auto_20190607_1422... OK
babybuddy    |   Applying core.0009_diaperchange_amount... OK
babybuddy    |   Applying core.0010_timer_child... OK
babybuddy    |   Applying core.0011_auto_20200214_1939... OK
babybuddy    |   Applying core.0012_auto_20200813_0238... OK
babybuddy    |   Applying core.0013_auto_20210415_0528... OK
babybuddy    |   Applying core.0014_alter_child_slug... OK
babybuddy    |   Applying core.0015_add_nap_field_for_sleep... OK
babybuddy    |   Applying core.0016_alter_sleep_napping... OK
babybuddy    |   Applying core.0017_alter_child_last_name... OK
babybuddy    |   Applying core.0018_bmi_headcircumference_height... OK
babybuddy    |   Applying core.0019_tag_tagged_note_tags... OK
babybuddy    |   Applying core.0020_bmi_tags_diaperchange_tags_feeding_tags_and_more... OK
babybuddy    |   Applying core.0021_pumping... OK
babybuddy    |   Applying core.0022_alter_default_date_and_time... OK
babybuddy    |   Applying core.0023_alter_tag_options_alter_bmi_tags_and_more... OK
babybuddy    |   Applying core.0024_alter_tag_slug... OK
babybuddy    |   Applying core.0025_pumping_tags... OK
babybuddy    |   Applying core.0026_alter_feeding_end_alter_feeding_start_and_more... OK
babybuddy    |   Applying core.0027_alter_timer_options_remove_timer_duration_and_more... OK
babybuddy    |   Applying core.0028_alter_sleep_options_remove_sleep_napping_sleep_nap... OK
babybuddy    |   Applying core.0029_alter_pumping_options_remove_pumping_time_and_more... OK
babybuddy    |   Applying sessions.0001_initial... OK
babybuddy    | Cache table 'cache_default' already exists.
babybuddy    | [custom-init] No custom files found, skipping...
babybuddy    | [2023-10-21 13:36:36 +0000] [254] [INFO] Starting gunicorn 21.2.0
babybuddy    | [2023-10-21 13:36:36 +0000] [254] [INFO] Listening at: http://127.0.0.1:3000 (254)
babybuddy    | [2023-10-21 13:36:36 +0000] [254] [INFO] Using worker: gthread
babybuddy    | [2023-10-21 13:36:36 +0000] [269] [INFO] Booting worker with pid: 269
babybuddy    | [2023-10-21 13:36:36 +0000] [270] [INFO] Booting worker with pid: 270
babybuddy    | [ls.io-init] done.
^CGracefully stopping... (press Ctrl+C again to force)
Stopping babybuddy ... done
 chris@hedonismbot  ~/lsio  vim docker-compose.yml 
 chris@hedonismbot  ~/lsio  docker-compose up                  
Pulling babybuddy (lscr.io/linuxserver/babybuddy:2.1.2)...
2.1.2: Pulling from linuxserver/babybuddy
95e5233d28e2: Pull complete
07a0e16f7be1: Pull complete
0884370a21fb: Pull complete
829fd089b86c: Pull complete
dd4973dd380b: Pull complete
2be8bcb68f9c: Pull complete
395670d94ef1: Pull complete
d239e312faf8: Pull complete
6d2d8bb11d2d: Pull complete
Digest: sha256:286cb357dda6520b51a4a2be1364faa0a17c182fc32a404079afa3fb640db123
Status: Downloaded newer image for lscr.io/linuxserver/babybuddy:2.1.2
Recreating babybuddy ... done
Attaching to babybuddy
babybuddy    | [migrations] started
babybuddy    | [migrations] 01-nginx-site-confs-default: skipped
babybuddy    | [migrations] 02-default-location: skipped
babybuddy    | [migrations] done
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    |       ██╗     ███████╗██╗ ██████╗ 
babybuddy    |       ██║     ██╔════╝██║██╔═══██╗
babybuddy    |       ██║     ███████╗██║██║   ██║
babybuddy    |       ██║     ╚════██║██║██║   ██║
babybuddy    |       ███████╗███████║██║╚██████╔╝
babybuddy    |       ╚══════╝╚══════╝╚═╝ ╚═════╝ 
babybuddy    | 
babybuddy    |    Brought to you by linuxserver.io
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | To support the app dev(s) visit:
babybuddy    | BabyBuddy: https://github.com/sponsors/cdubz
babybuddy    | 
babybuddy    | To support LSIO projects visit:
babybuddy    | https://www.linuxserver.io/donate/
babybuddy    | 
babybuddy    | ───────────────────────────────────────
babybuddy    | GID/UID
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | User UID:    1000
babybuddy    | User GID:    1000
babybuddy    | ───────────────────────────────────────
babybuddy    | 
babybuddy    | using keys found in /config/keys
babybuddy    | Operations to perform:
babybuddy    |   Apply all migrations: admin, auth, authtoken, axes, babybuddy, contenttypes, core, dbsettings, sessions
babybuddy    | Running migrations:
babybuddy    |   Applying core.0030_weightpercentile_weightpercentile_unique_age_sex... OK
babybuddy    | Cache table 'cache_default' already exists.
babybuddy    | [custom-init] No custom files found, skipping...
babybuddy    | [2023-10-21 13:44:55 +0000] [235] [INFO] Starting gunicorn 21.2.0
babybuddy    | [2023-10-21 13:44:55 +0000] [235] [INFO] Listening at: http://127.0.0.1:3000 (235)
babybuddy    | [2023-10-21 13:44:55 +0000] [235] [INFO] Using worker: gthread
babybuddy    | [2023-10-21 13:44:55 +0000] [250] [INFO] Booting worker with pid: 250
babybuddy    | [2023-10-21 13:44:55 +0000] [251] [INFO] Booting worker with pid: 251
babybuddy    | [ls.io-init] done.
jschollenberger commented 11 months ago

I exported, deleted the container entirely, then recreated it from the latest image. It seems to have unwedged whatever went wrong with dotenv because it's working now. Not a great solution, but I couldn't seem to fix things from inside the container. So, I guess this is solved.

I'm not sure what Container Manager is doing behind the scenes for the upgrade, but I'm guessing it just pulls the latest image then resets the container. I'm not sure why this would be any different than me deleting it like I did above, but that was what it took to get it going.

Appreciate you talking it through and your efforts on BabyBuddy!

cdubz commented 11 months ago

Glad ya got it working. And as a bonus I finally documented how to run admin commands in the container.

barakplasma commented 11 months ago

Looks like dotenv wasn't installed somehow based on the pip output. Not sure how that would happen.

jschollenberger commented 11 months ago

Isn't that python-dotenv 1.0.0? It's on there.

cdubz commented 11 months ago

We didn't establish where that output came from. I think that can happen if the right ENV variables are not set but I can't remember for sure.

jschollenberger commented 11 months ago

I did actually set those environment variables as part of trying to recover, before nuking the whole thing. There was no change.