search

Aleksi44 / wagtailsvg

Wagtail + SVG
https://pypi.org/project/wagtailsvg/
GNU General Public License v3.0
35 stars 24 forks source link

Missing migration file for wagtailsvg', '0005_alter_svg_file ? #20

Closed ajfriesen closed 1 year ago

ajfriesen commented 2 years ago

Today I tried to get up my django app on a server.

When I hit the migrations I see the following error:

curated-app       | Traceback (most recent call last):
curated-app       |   File "/code/manage.py", line 22, in <module>
curated-app       |     main()
curated-app       |   File "/code/manage.py", line 18, in main
curated-app       |     execute_from_command_line(sys.argv)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
curated-app       |     utility.execute()
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
curated-app       |     self.fetch_command(subcommand).run_from_argv(self.argv)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
curated-app       |     self.execute(*args, **cmd_options)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
curated-app       |     output = self.handle(*args, **options)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 98, in wrapped
curated-app       |     res = handle_func(*args, **kwargs)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 108, in handle
curated-app       |     executor = MigrationExecutor(connection, self.migration_progress_callback)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 18, in __init__
curated-app       |     self.loader = MigrationLoader(self.connection)
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 58, in __init__
curated-app       |     self.build_graph()
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 276, in build_graph
curated-app       |     self.graph.validate_consistency()
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 198, in validate_consistency
curated-app       |     [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 198, in <listcomp>
curated-app       |     [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
curated-app       |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/graph.py", line 60, in raise_error
curated-app       |     raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
curated-app       | django.db.migrations.exceptions.NodeNotFoundError: Migration self_hosted_apps.0001_initial dependencies reference nonexistent parent node ('wagtailsvg', '0005_alter_svg_file')

I thought maybe I can find this migration in your repo, but it is missing.

Is this something in my project or something from wagtailsvg?

Odd thing: On my local setup It works but that database is running for a few weeks now.

ajfriesen commented 2 years ago

I tried again and when setting up a fresh local environment on my laptop the migration will work. I then checked the possible missing migration which is located here:

~/.pyenv/versions/curated/lib/python3.10/site-packages/wagtailsvg/migrations/0005_alter_svg_file.py 

# Generated by Django 4.0.6 on 2022-08-02 06:19

from django.db import migrations, models

class Migration(migrations.Migration):

    dependencies = [
        ('wagtailsvg', '0004_remove_svg_edit_code'),
    ]

    operations = [
        migrations.AlterField(
            model_name='svg',
            name='file',
            field=models.FileField(upload_to='svg', verbose_name='file'),
        ),
    ]

The difference between local and production is that I am using a docker container for production. It looks like when I am building that container the file is not part of the container since it is generated on the fly.

Should this be part of the package?

Aleksi44 commented 2 years ago

Hi @ajfriesen

I think there is a migration missing in the wagtailsvg. In the meantime, you can modify your local migration file and remove the depandancy "0005_alter_svg_file".

ajfriesen commented 2 years ago

Mh, I am not sure how I can just ignore them.

For local development I can comment the depenceny, make migrations and the missing migration will be created for this package.

But when building my docker image I start my docker container by applying migrations. This does not work because of the missing on in the package.

Is there any workaround where I can first create the missing migration file for wagtailsvg and then migrate my apps?

Aleksi44 commented 2 years ago

@ajfriesen

In your Docker, you have to copy your migration files. I think you are not supposed to do a "makemigrations" but only a "migrate".

ajfriesen commented 2 years ago

@ajfriesen

In your Docker, you have to copy your migration files. I think you are not supposed to do a "makemigrations" but only a "migrate".

In docker I only do migrate because otherwise, I might miss migrations in my repo.

But I am still not sure how to implement a workaround for the missign migration.

Aleksi44 commented 2 years ago

@ajfriesen

It's weird because when I do a makemigration on wagtailsvg, I have no migration generated. I don't understand why you have one generated. Do you have an idea?

Aleksi44 commented 2 years ago

@ajfriesen

Can you show me this file : self_hosted_apps.0001_initial ?

ajfriesen commented 2 years ago

I think I get the migration because I add the field with type of wagtailsvg to my model. But this is just a guess.

Here my repo and the migrations:

https://github.com/ajfriesen/curated/tree/main/self_hosted_apps/migrations

docker compose up
python manage.py runserver --settings curated.settings.local

I am trying to understand what is happening. Maybe django MIGRATTION_MODULES is something which could help here. But not sure on that.

https://stackoverflow.com/questions/47153776/how-to-store-third-party-apps-migrations-in-django

Aleksi44 commented 2 years ago

I see in your migrations files, there are 2 calls of a wagtailsvg migration that does not exist :

If you remove these lines, I think it could work but I'm not sure because your backtrace seem to say that you generate new migrations files (because the error is in 0001 migration file) :

django.db.migrations.exceptions.NodeNotFoundError: Migration self_hosted_apps.0001_initial dependencies reference nonexistent parent node ('wagtailsvg', '0005_alter_svg_file')

I can check later in detail. Don't hesitate to tell me if you find the problem.

ajfriesen commented 2 years ago

sorry the backtrace was from a test when I testeds with a single migration file. That is misleading.

Nonetheless, I tested again:

I have deleted my postgres db and removed all my migrations. I then start up the postgres again. Then I start the python process. Create all migrations

python manage.py makemigrations           
Migrations for 'wagtailsvg':
  /home/andrej/.pyenv/versions/curated/lib/python3.10/site-packages/wagtailsvg/migrations/0005_alter_svg_file.py
    - Alter field file on svg
Migrations for 'self_hosted_apps':
  self_hosted_apps/migrations/0001_initial.py
    - Create model AppListPage
    - Create model AppPage

As you can see this will create a migration file in the package layer.

When I migrate everything works fine.

python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, django_celery_results, generic_page, home, self_hosted_apps, sessions, taggit, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailredirects, wagtailsearch, wagtailsvg, wagtailusers
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying django_celery_results.0001_initial... OK
  Applying django_celery_results.0002_add_task_name_args_kwargs... OK
  Applying django_celery_results.0003_auto_20181106_1101... OK
  Applying django_celery_results.0004_auto_20190516_0412... OK
  Applying django_celery_results.0005_taskresult_worker... OK
  Applying django_celery_results.0006_taskresult_date_created... OK
  Applying django_celery_results.0007_remove_taskresult_hidden... OK
  Applying django_celery_results.0008_chordcounter... OK
  Applying django_celery_results.0009_groupresult... OK
  Applying django_celery_results.0010_remove_duplicate_indices... OK
  Applying django_celery_results.0011_taskresult_periodic_task_name... OK
  Applying wagtailcore.0001_squashed_0016_change_page_url_path_to_text_field... OK
  Applying wagtailcore.0017_change_edit_page_permission_description... OK
  Applying wagtailcore.0018_pagerevision_submitted_for_moderation_index... OK
  Applying wagtailcore.0019_verbose_names_cleanup... OK
  Applying wagtailcore.0020_add_index_on_page_first_published_at... OK
  Applying wagtailcore.0021_capitalizeverbose... OK
  Applying wagtailcore.0022_add_site_name... OK
  Applying wagtailcore.0023_alter_page_revision_on_delete_behaviour... OK
  Applying wagtailcore.0024_collection... OK
  Applying wagtailcore.0025_collection_initial_data... OK
  Applying wagtailcore.0026_group_collection_permission... OK
  Applying wagtailcore.0027_fix_collection_path_collation... OK
  Applying wagtailcore.0024_alter_page_content_type_on_delete_behaviour... OK
  Applying wagtailcore.0028_merge... OK
  Applying wagtailcore.0029_unicode_slugfield_dj19... OK
  Applying wagtailcore.0030_index_on_pagerevision_created_at... OK
  Applying wagtailcore.0031_add_page_view_restriction_types... OK
  Applying wagtailcore.0032_add_bulk_delete_page_permission... OK
  Applying wagtailcore.0033_remove_golive_expiry_help_text... OK
  Applying wagtailcore.0034_page_live_revision... OK
  Applying wagtailcore.0035_page_last_published_at... OK
  Applying wagtailcore.0036_populate_page_last_published_at... OK
  Applying wagtailcore.0037_set_page_owner_editable... OK
  Applying wagtailcore.0038_make_first_published_at_editable... OK
  Applying wagtailcore.0039_collectionviewrestriction... OK
  Applying wagtailcore.0040_page_draft_title... OK
  Applying wagtailcore.0041_group_collection_permissions_verbose_name_plural... OK
  Applying wagtailcore.0042_index_on_pagerevision_approved_go_live_at... OK
  Applying wagtailcore.0043_lock_fields... OK
  Applying wagtailcore.0044_add_unlock_grouppagepermission... OK
  Applying wagtailcore.0045_assign_unlock_grouppagepermission... OK
  Applying wagtailcore.0046_site_name_remove_null... OK
  Applying wagtailcore.0047_add_workflow_models... OK
  Applying wagtailcore.0048_add_default_workflows... OK
  Applying wagtailcore.0049_taskstate_finished_by... OK
  Applying wagtailcore.0050_workflow_rejected_to_needs_changes... OK
  Applying wagtailcore.0051_taskstate_comment... OK
  Applying wagtailcore.0052_pagelogentry... OK
  Applying wagtailcore.0053_locale_model... OK
  Applying wagtailcore.0054_initial_locale... OK
  Applying wagtailcore.0055_page_locale_fields... OK
  Applying wagtailcore.0056_page_locale_fields_populate... OK
  Applying wagtailcore.0057_page_locale_fields_notnull... OK
  Applying wagtailcore.0058_page_alias_of... OK
  Applying wagtailcore.0059_apply_collection_ordering... OK
  Applying wagtailcore.0060_fix_workflow_unique_constraint... OK
  Applying wagtailcore.0061_change_promote_tab_helpt_text_and_verbose_names... OK
  Applying wagtailcore.0062_comment_models_and_pagesubscription... OK
  Applying wagtailcore.0063_modellogentry... OK
  Applying wagtailcore.0064_log_timestamp_indexes... OK
  Applying wagtailcore.0065_log_entry_uuid... OK
  Applying wagtailcore.0066_collection_management_permissions... OK
  Applying wagtailcore.0067_alter_pagerevision_content_json... OK
  Applying wagtailcore.0068_log_entry_empty_object... OK
  Applying wagtailcore.0069_log_entry_jsonfield... OK
  Applying generic_page.0001_initial... OK
  Applying home.0001_initial... OK
  Applying home.0002_remove_homepage_body... OK
  Applying home.0003_homepage_body... OK
  Applying taggit.0001_initial... OK
  Applying taggit.0002_auto_20150616_2121... OK
  Applying taggit.0003_taggeditem_add_unique_index... OK
  Applying wagtailsvg.0001_initial... OK
  Applying wagtailsvg.0002_svg_edit_code... OK
  Applying wagtailsvg.0003_alter_svg_options... OK
  Applying wagtailsvg.0004_remove_svg_edit_code... OK
  Applying wagtailsvg.0005_alter_svg_file... OK
  Applying wagtailimages.0001_squashed_0021... OK
  Applying wagtailimages.0022_uploadedimage... OK
  Applying wagtailimages.0023_add_choose_permissions... OK
  Applying wagtailimages.0024_index_image_file_hash... OK
  Applying self_hosted_apps.0001_initial... OK
  Applying sessions.0001_initial... OK
  Applying taggit.0004_alter_taggeditem_content_type_alter_taggeditem_tag... OK
  Applying wagtailadmin.0001_create_admin_access_permissions... OK
  Applying wagtailadmin.0002_admin... OK
  Applying wagtailadmin.0003_admin_managed... OK
  Applying wagtaildocs.0001_initial... OK
  Applying wagtaildocs.0002_initial_data... OK
  Applying wagtaildocs.0003_add_verbose_names... OK
  Applying wagtaildocs.0004_capitalizeverbose... OK
  Applying wagtaildocs.0005_document_collection... OK
  Applying wagtaildocs.0006_copy_document_permissions_to_collections... OK
  Applying wagtaildocs.0005_alter_uploaded_by_user_on_delete_action... OK
  Applying wagtaildocs.0007_merge... OK
  Applying wagtaildocs.0008_document_file_size... OK
  Applying wagtaildocs.0009_document_verbose_name_plural... OK
  Applying wagtaildocs.0010_document_file_hash... OK
  Applying wagtaildocs.0011_add_choose_permissions... OK
  Applying wagtaildocs.0012_uploadeddocument... OK
  Applying wagtailembeds.0001_initial... OK
  Applying wagtailembeds.0002_add_verbose_names... OK
  Applying wagtailembeds.0003_capitalizeverbose... OK
  Applying wagtailembeds.0004_embed_verbose_name_plural... OK
  Applying wagtailembeds.0005_specify_thumbnail_url_max_length... OK
  Applying wagtailembeds.0006_add_embed_hash... OK
  Applying wagtailembeds.0007_populate_hash... OK
  Applying wagtailembeds.0008_allow_long_urls... OK
  Applying wagtailembeds.0009_embed_cache_until... OK
  Applying wagtailforms.0001_initial... OK
  Applying wagtailforms.0002_add_verbose_names... OK
  Applying wagtailforms.0003_capitalizeverbose... OK
  Applying wagtailforms.0004_add_verbose_name_plural... OK
  Applying wagtailforms.0005_alter_formsubmission_form_data... OK
  Applying wagtailredirects.0001_initial... OK
  Applying wagtailredirects.0002_add_verbose_names... OK
  Applying wagtailredirects.0003_make_site_field_editable... OK
  Applying wagtailredirects.0004_set_unique_on_path_and_site... OK
  Applying wagtailredirects.0005_capitalizeverbose... OK
  Applying wagtailredirects.0006_redirect_increase_max_length... OK
  Applying wagtailredirects.0007_add_autocreate_fields... OK
  Applying wagtailredirects.0008_add_verbose_name_plural... OK
  Applying wagtailsearch.0001_initial... OK
  Applying wagtailsearch.0002_add_verbose_names... OK
  Applying wagtailsearch.0003_remove_editors_pick... OK
  Applying wagtailsearch.0004_querydailyhits_verbose_name_plural... OK
  Applying wagtailsearch.0005_create_indexentry... OK
  Applying wagtailsearch.0006_customise_indexentry... OK
  Applying wagtailusers.0001_initial... OK
  Applying wagtailusers.0002_add_verbose_name_on_userprofile... OK
  Applying wagtailusers.0003_add_verbose_names... OK
  Applying wagtailusers.0004_capitalizeverbose... OK
  Applying wagtailusers.0005_make_related_name_wagtail_specific... OK
  Applying wagtailusers.0006_userprofile_prefered_language... OK
  Applying wagtailusers.0007_userprofile_current_time_zone... OK
  Applying wagtailusers.0008_userprofile_avatar... OK
  Applying wagtailusers.0009_userprofile_verbose_name_plural... OK
  Applying wagtailusers.0010_userprofile_updated_comments_notifications... OK

The problem is, that I don't know how to package that package migration in my docker container. Unless I do a makemigration when I build the container. This seems like the wrong way. But I could be mistaken on that opinion.

ajfriesen commented 2 years ago

I had some help in the django forum: https://forum.djangoproject.com/t/possible-migration-file-missing-from-third-party-package/15170/7

It looks like this migration will only be created when you set the variable WAGTAILSVG_UPLOAD_FOLDER = 'svg'. I got this from the readme.

When I do not use this variable and just keep the default there will be no migration created on package level. Only on my application level.

Aleksi44 commented 2 years ago

Indeed, thank you for finding the problem. Do you know how to fix it from wagtailsvg ? I can apply PR so you won't have the problem anymore.

ajfriesen commented 2 years ago

I am not sure to handle this.

It looks like others do this with the MIGRATION_MODULES which could be an approach when handling migrations for third-party apps.

Did not tackle that though.

https://stackoverflow.com/questions/47153776/how-to-store-third-party-apps-migrations-in-django

ekersten commented 2 years ago

@Aleksi44 you could maybe try something like tihis https://medium.com/@nurettinabaci/dynamic-upload-to-a9935995be33 and check if that method doesn't create a new migration when the folder changes

ekersten commented 1 year ago

the issue was fixed on PR #22 but it's still not merged.

Aleksi44 commented 1 year ago

@ekersten

Thanks, I'll take care of it tonight.