Version 3.0.0 has conflicts with previous versions, no migrations

[SalahAdDin]

I decided install and test the new version and i have the next problem:

 salahaddin@TulipanROG  ~/Proyectos/Works/recipes/cms/cms   master ↑1  cms                                                                                                                                                                ✔  2293  15:29:02./manage.py migrate           
/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Operations to perform:
  Apply all migrations: admin, auth, axes, contenttypes, core, home, robots, sessions, sites, taggit, wagtail_embed_videos, wagtail_feeds, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailmenus, wagtailmetadata, wagtailredirects, wagtailsearch, wagtailsearchpromotions, wagtailstreamforms, wagtailusers
Running migrations:
  No migrations to apply.
 salahaddin@TulipanROG  ~/Proyectos/Works/recipes/cms/cms   master ↑1  cms                                                                                                                                                                ✔  2294  15:29:10./manage.py runserver         
/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Performing system checks...

System check identified no issues (0 silenced).
May 19, 2018 - 15:29:21
Django version 2.0.5, using settings 'cms.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[19/May/2018 15:29:39] "GET /admin/settings/core/globalsettings/2/ HTTP/1.1" 200 5640
[19/May/2018 15:29:43] "GET /admin/wagtailmenus/flatmenu/ HTTP/1.1" 200 4239
[19/May/2018 15:29:43] "GET /static/wagtailmodeladmin/css/index.css HTTP/1.1" 200 2089
Internal Server Error: /admin/wagtailstreamforms/form/
Traceback (most recent call last):
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: no existe la relación «wagtailstreamforms_form»
LINE 1: SELECT COUNT(*) AS "__count" FROM "wagtailstreamforms_form"
                                          ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/views/decorators/cache.py", line 31, in _cache_controlled
    response = viewfunc(request, *args, **kw)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/admin/urls/__init__.py", line 102, in wrapper
    return view_func(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/admin/decorators.py", line 34, in decorated_view
    return view_func(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/contrib/modeladmin/options.py", line 315, in index_view
    return view_class.as_view(**kwargs)(request)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/utils/decorators.py", line 62, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/utils/decorators.py", line 58, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/contrib/modeladmin/views.py", line 248, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/utils/decorators.py", line 62, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/utils/decorators.py", line 58, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/contrib/modeladmin/views.py", line 68, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/views/generic/base.py", line 89, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/views/generic/base.py", line 150, in get
    context = self.get_context_data(**kwargs)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/wagtail/contrib/modeladmin/views.py", line 616, in get_context_data
    all_count = self.get_base_queryset().count()
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/models/query.py", line 387, in count
    return self.query.get_count(using=self.db)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/models/sql/query.py", line 491, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/models/sql/query.py", line 476, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
    cursor.execute(sql, params)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/debug_toolbar/panels/sql/tracking.py", line 164, in execute
    return self._record(self.cursor.execute, sql, params)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/debug_toolbar/panels/sql/tracking.py", line 106, in _record
    return method(sql, params)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: no existe la relación «wagtailstreamforms_form»
LINE 1: SELECT COUNT(*) AS "__count" FROM "wagtailstreamforms_form"
                                          ^

[19/May/2018 15:29:45] "GET /admin/wagtailstreamforms/form/ HTTP/1.1" 500 23235
^C%                                                                                                                                                                                                                                                                   salahaddin@TulipanROG  ~/Proyectos/Works/recipes/cms/cms   master ↑1  cms                                                                                                                                                                ✔  2295  15:29:54./manage.py makemigrations
/home/salahaddin/Proyectos/Works/recipes/cms/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
No changes detected

As you can see there is no changes or new migrations in this version, but i get a problem with tables, that's because i used the previous version before.

There is any solution for this problem?

[stuartaccent]

Hi @SalahAdDin,

Hope you are well, you are right v3 will not install over an old version as the migrations were all binned due to it being re-written. None of the old tables now exist with the exception of the FormSubmission.

The only way currently to get this on an existing install is to first loose the old one, db tables, migration rows in the django_migrations table also, then go from there.

http://wagtailstreamforms.readthedocs.io/en/latest/#backwards-compatibility

That being said I may actually look at this tonight and see if it can be fixed, though I dont know if its a priority as im not sure if we are putting it on pypi or just leaving it in the repo

Cheers Stu

[stuartaccent]

Further more the reason u see No changes detected is that v3 only has 0001_initial.py as a migration and that already exists as part of the old one, if you delete the tables and that row in django_migrations it should go on ok.

[SalahAdDin]

@stuartaccent But if we delete all tables will lose all data content, that's why we need migrations.

[stuartaccent]

@SalahAdDin

I appreciate that totally and it has been on my mind since we started this, if we do release it on pypi we will probably have to look at this though there will still be things to overcome.

The old package allowed additional form models to be created that we would not know the additional model fields that were created in these only what we expect the models to have for the form only. We would be not be able to handled these as there is no where to put them. Nor would we be able to see them without rewinding the code base to deal with these before all the code was removed.

Things like the email sending is now a hook and its expected that this functionality is in the installing apps code base via the hooks, so we could not do anything with this data either.

Fields would need to be converted to the StreamField StructValue, doable i reckon.

The recaptcha part requires a field to be created also which is on a app by app basis.

If we try to create the migrations to upgrade the existing tables i'm not sure then we could do it in a way where it wouldn't force the loss of data. Which is why we decided in the first place 2 would not be upgradable to 3. If anyone has got any ideas i'm all ears :)

Short of making this a separate package entirely and allowing the install side by side I don't know what we can do as it's model layout is totally different to the original spec :(

Stu

[SalahAdDin]

@stuartaccent It's a very complex situation, by my lucky i haven't data, so, i can change it easily, but...

[stuartaccent]

@SalahAdDin will see what i can come up with but will be far from ideal and i really dont want people to just upgrade then realise things have been lost :(

[SalahAdDin]

Yes, i thing the same.

[SalahAdDin]

@stuartaccent what happened with email forms, and regex validators?

[stuartaccent]

Hi @SalahAdDin. Am on my phone at the minute but when I get home will write up in an issue what our thoughts and plans are for this and the changes. This should help answer any questions like this in one place.

Cheers. Stu

[stuartaccent]

Hi @SalahAdDin hope #90 explains things :)

[SalahAdDin]

@stuartaccent, i think many people came to this repo because they did want no make form by herself, that's why built-in form were greatful; at least you can provide a email form and regex validators example.

[stuartaccent]

@SalahAdDin, the example hook in the docs is the email one http://wagtailstreamforms.readthedocs.io/en/v3.0.0/hooks.html#create-your-own-hook.

I will add the validator field example too when i get a sec :)

[SalahAdDin]

We need to extend the new Form model, right?

[stuartaccent]

Nope you just need to register a new field like in the example pr #91 , docs for adding new fields are here

[SalahAdDin]

I mean, if we want create a new email form, we have to extend the form model, right?

[stuartaccent]

Extending the models was removed in favour of providing hooks to add the required functionality. As I linked above http://wagtailstreamforms.readthedocs.io/en/v3.0.0/hooks.html#create-your-own-hook has an example with how to create a hook to do this.

While we also appreciate removing functionality is not good, v3 was created only for our own benefit which is why we are not publishing it as outlined in #90.

Cheers, Stu

[SalahAdDin]

Email form and news letter form is interesting feature.

[stuartaccent]

Hi, Email is def something we are gonna look at. We always had the idea of being able to define templates in the ui or integration with a third party package. but originally we had no time for it. Now we have the time we are gonna look at it in more detail. I am keen not to just replicate what we did originally as it brings alot of what seemed like pointless code for what was just an email and want to make sure the end user can define the email in its enterity. Am also keen keep the form model to just handle the form if you know what i mean.

If you have any idea or good packages to either get ideas or integrate shout.

Ty :)

[SalahAdDin]

No, i didn't understand, sorry. XD