search

mozilla-services / syncserver

Run-Your-Own Firefox Sync Server
Mozilla Public License 2.0
1.87k stars 145 forks source link

Specified key was too long; max key length is 767 bytes #121

Closed timakro closed 6 years ago

timakro commented 6 years ago 
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1071, 'Specified key was too long; max key length is 767 bytes') [SQL: u'CREATE UNIQUE INDEX clientstate_idx ON users (email, service, client_state)']

I get this error with SQLAlchemy's default MySQL driver on MariaDB 15.1

timakro commented 6 years ago

It seems like in MySQL the maximum length for utf8mb4 encoded VARCHAR fields is 191 according to https://stackoverflow.com/a/1814594/3235192.

In staticnode.py line 37 its set to 255:

 Column("email", String(255), nullable=False),
rfk commented 6 years ago

@timakro to clarify, do you get this error when trying to start the server for the first time, or after the server is up and running and you try to use it?

rfk commented 6 years ago

Ah, it's in CREATE UNIQUE INDEX clientstate_idx so I guess it's occurring when trying to initialize the DB. If you change the size of the email column declaration that you linked above, does it fix the issue?

timakro commented 6 years ago

Yes, changing the size of the email column from 255 to 191 fixes this for me.

EDIT: And yes, it seems to be because of the unique index 255 doesn't work.

rfk commented 6 years ago

I sat down today to see about landing a fix for this, and realized that the troublesome "clientstate_idx" index has been removed:

https://github.com/mozilla-services/tokenserver/blob/master/tokenserver/assignment/sqlnode/migrations/versions/3d5af3924466_drop_clientstate_idx.py

So I'm not entirely sure why this was failing when trying to create it. @timakro how were you installing/running the package to receive this error?

timakro commented 6 years ago

I followed the official tutorial and I'm using uWSGI. I think SQLAlchemy tries to create the table like in the staticnode.py file before any alembic migrations are applied.

The tokenserver is another repo, right? What is that anyways?

rfk commented 6 years ago

Aha, thanks, I forgot that syncserver has its own implementation of this rather than using the one from the tokenserver repo.

The tokenserver is another repo, right? What is that anyways?

Right. We don't actually deploy this "syncserver" repo in production, we run separate servers for the tokenserver part and for the storage part, basically because we need to run a huge number of storage nodes.

rfk commented 6 years ago

Proposed fix in https://github.com/mozilla-services/syncserver/pull/123 by just dropping the unique constraint.