Encryption

[uprjfc]

Sadly it seems we will have to stop using Formtools because of a new European law which means we have to hold data in an encrypted form on the server. This is now any personal data not just financial data.

I know this has been asked in the past but is there any 'easy' way to add 2 way encryption to the system so that it can be used as it is but with the data encrypted within the database?

Thanks

[nelsondcosta23]

Sadly its true, that will be a pain in the *** for us :(

[benkeen]

Hey @uprjfc, I'm so sorry for not getting back to you sooner - you posted right when I'd left town for a few days.

Nelson's quite right, unfortunately it's not a trivial change. Encoding/decoding for DB queries wouldn't be too tricky to add but there are lot of scenarios that need to be accounted for, e.g. ensuring database field sizes are all set to specific sizes to store the (usually longer) encrypted form of the data, plus allowing the encryption feature to be toggled on an off.

Do you have any info on the law? I'd very much like to read up on it to see just how broad it is and how likely it is that this will affect other users. I'm absolutely game to add in this feature in a future release, but it'll take time to get right.

[nelsondcosta23]

Hi Ben, how are you? So the best short version I found is: (read the resume, maybe first :P) https://www.fellowshipproductions.co.uk/make-your-website-gdpr-compliant/

Resume: I think the idea for FT works with GDPR, and I think its very important because this will expand to USA and ASIA. IN some way, when we create a from adn we creat the fields on FT we can have an option to encrypt or not each field. So this way, the user/admin is the one who have to make sure GDPR laws are working. And make sure the DB can hold the encryption. I think Ir might be a easy way and the better way. Did I exaplin myself?! Yap im bad at this XD

[benkeen]

Excellent, thanks for the link Nelson. I'll give it a good read over this weekend, but it looks like this is going to be necessary to add - and sooner rather than later.

[uprjfc]

Hi Ben I am not very good with github as I replied by email 10 days ago but it is not here.I have copied my message below

Hi

It is the usual sledgehammer to crack a nut legislation from the European Union.

It is designed to punish companies that have their data hacked but it is so wide it includes paper records and every organisation even little sports clubs like ours.

This is the official page https://www.eugdpr.org/

Advice given to us says that data should be encrypted 'in transit' and 'at rest'. It is not mandatory but given they can levy almost unlimited fines who wants to take a chance?

I know you have said that if the key is stored on the server it can be hacked but there is also an element of complying at a database level to be able to say 'that is the best we can do'. Commercial services offer encryption but all output functionality such as emails, reports etc do not work which is not a workable solution.

So any solution would be good even if not perfect.

Thanks

[nelsondcosta23]

I'm actually right now in a gdpr workshop, well, encryption on name, ID numbers, Geo/data location, and electronics IDs. So, Ben, do you think there is a way to make a modulo to say witch fields we want or not encrypted?

[benkeen]

Hey, sorry I didn't respond to this earlier. My family descended on me the last couple of weekends so I've gotten very little done.

So I did end up looking into this but found there weren't any particular good solutions, just a lot of problems. Other than the inevitable performance issues, the biggest problem with encrypting the data in the database is that is effectively nullifies database searches. Form Tools includes numerous content searches, such as the main submission search, view filters, and more. They rely on being able to search actual strings in the database - but when they're encrypted, those features can no longer work. That's pretty bad: this is key functionality.

The more I read about it it seems that encrypting the data before and after insertion and retrieval from the database is really no more than a hack. What's needed is a proper, transparent database-level encryption so the data is encrypted at rest but trusted applications can query that data and have the unencrypted results returns for display and manipulation. This is offered by various commercial scripts/services, like MySQL Enterprise https://www.mysql.com/products/enterprise/tde.html and others - but I haven't encountered an open source option yet. Certainly not one that would be a good fit for Form Tools.

I'm afraid this is an extremely thorny issue that I don't have a good answer for.

[nelsondcosta23]

Hi Ben, MARIADB 10.1 I think it may have something like that.

https://www.v3.co.uk/v3-uk/news/2426170/mariadb-101-adds-database-encryption-ready-for-new-eu-data-protection-laws

[uprjfc]

Hi

That would work for us if Formtools is not affected by it.

We can't afford a commercial solution at the moment.

[uprjfc]

Sorry closed by mistake

[benkeen]

Thanks for the link, Nelson. That's exactly what I had in mind. I daresay with the laws changing as they are, this will become a defacto standard with all database engines.

[nelsondcosta23]

So this is a hope? :P If I understood, we create a key in the DB server and give the key to the FT and the data travel encrypted inside the key connection?

[streich]

Has anyone successfully used the built-in data-at-rest encryption available in MySQL? [https://dev.mysql.com/doc/refman/8.0/en/innodb-data-encryption.html]