globaleaks / globaleaks-whistleblowing-software

GlobaLeaks is free, open-source whistleblowing software enabling anyone to easily set up and maintain a secure reporting platform.
https://www.globaleaks.org
Other
1.23k stars 270 forks source link

GDPR Compliance: Make it possible to mask (hide) parts of reports (such as unnecessary personal info) #3420

Open danielvaknine opened 1 year ago

danielvaknine commented 1 year ago

Proposal

We suggest that recipients can be enabled the ability to redact parts of reports or chats with the whistleblower (like they can be enabled the possibility to delete the cases as a whole). This is important when it comes to e.g. unnecessary personal information or other sensitive data that the receiving party has no right to process and/or keep.

The function would not straight-off delete the text, but only redact it, as seen below

Screenshot 2023-04-12 at 21 26 28

The original info can still be shown to the whistleblower.

More advanced implementations could also be evaluated, such as that another recipient must approve the redacting, that a comment must be made to the redaction or that the redacted text could be retrieved. However, what's most important is that recipients can have the ability to, if needed, hide or delete certain unnecessary parts of reports.

Motivation and context

This function would be needed to make the platform fully compliant with the GDPR by enabling the redaction of sensitive personal information not needed for the investigation. It could also be used to e.g. hide the whistleblower's identity before granting access to other recipients (that the whistleblower may not know would get access to the report, hence their identity should be hidden) and many other important aspects of a fully-compliant whistleblowing channel with high levels of integrity and whistleblower protection.

Related tickets: https://github.com/globaleaks/GlobaLeaks/issues/2541 https://github.com/globaleaks/GlobaLeaks/issues/3429

evilaliv3 commented 1 year ago

Thank you @danielvaknine for your valuable feedback.

This proposal is something we are considering not only in relation to the general principles of the GDPR but as well in relation to Article 17 of the European Directive on Whistleblowing that is built on it an states:

Screenshot from 2023-04-12 20-55-44

Such an article appear to us not so trivial to be implemented and we are hoping that the legislator will provide advice especially that Article 17, competes with the principles of Article 12:

Screenshot from 2023-04-12 21-26-18

This article primary requires that reporting channels "are designed, established and operated in a manner that ensures the completeness, integrity and confidentiality of the information and prevents access thereto by non-authorized staff members of the competent authority". This requirements is stated just for external reporting channels but as the principle is very general we consider it had probably to apply to both the two types of channels

For this reason it is not clear if:

We could definitely start addressing the possibility to redact information and produce a copy of the report that do not export redacted information but it is very unclear what to do in relation to actual ultimate deletion of such information.

gianlucagilardi commented 1 year ago

@danielvaknine redaction, at least as far as GDPR goes, is not deletion thus redaction would unlikely lead to art. 17 compliance.

@evilaliv3 food for thought: is voluntary deletion of a record in a DB (more so if this deletion is mandated by law) an integrity violation? If the answer is yes then any deletion of a record containing personal data (even by request of the data subject) should be considered a data breach, with all the relevant GDPR consequences.

danielvaknine commented 1 year ago

Thanks @gianlucagilardi for your feedback! We meant redaction as deletion, but where it is still visible what text was deleted, as the image in the ticket above.

@evilaliv3 , I completely understand the difficult situations/questions you pose and respect that they are no "easy fix". However, we believe that enabling the redaction of data is important to let the user handle reports in the way they want to. We don't believe that the GlobaLeaks platform should tell users how to work in relation to the GDPR and other data processing, but rather enable them to work with it according to their own interpretations.

In this way, those who don't want to enable redaction can keep it that way, while those who do want to enable it can. Many law firms put redaction as a requirement for whistleblowing platforms to be fully compliant, and we want to enable organisations to follow the interpretation they want (while still making sure status quo can be used).

Should we really decide for the organisations how to interpret these complex regulations?

gianlucagilardi commented 1 year ago

Thanks @gianlucagilardi for your feedback! We meant redaction as deletion, but where it is still visible what text was deleted, as the image in the ticket above.

@evilaliv3 , I completely understand the difficult situations/questions you pose and respect that they are no "easy fix". However, we believe that enabling the redaction of data is important to let the user handle reports in the way they want to. We don't believe that the GlobaLeaks platform should tell users how to work in relation to the GDPR and other data processing, but rather enable them to work with it according to their own interpretations.

In this way, those who don't want to enable redaction can keep it that way, while those who do want to enable it can. Many law firms put redaction as a requirement for whistleblowing platforms to be fully compliant

I myself am having a quite a hard time arguing art. 17 compliance of the platform at the moment and I can definitely relate to the struggle of lawyers on the issue at hand.

evilaliv3 commented 1 year ago

Thank you both for your feedback and i understand your points.

There are many points of which i think we have consensus here and we could:

The part on which instead at the moment different opinions exist is if the original should be preserved to preserve the integrity of the forensic evidence or if it is acceptable to invalidate the evidence to pursue article 17.

Regarding the question:

Should we really decide for the organizations how to interpret these complex regulations?

I do not have an answer that is probably representative for anyone but i can give my answer to this question that is "definitely yes" and i consider this is the vision at the base of the creation of globaleaks project and shared by most of the contributors.

It is great that the software complies for the most with the existing regulations and we will try to continue to guarantee this, but let me say that this is just a state of facts and it was not the original goal.

GlobaLeaks is a project started 13 years ago to support whistleblowers before any regulations existed and has at it's base the will to support the creation of a community of whistleblowers protection activists and stimulate the creation on a best practice on the matter. The configurations and the functionalities implemented are the results of continuous improvements thanks to the research on the matter by the early adopters that worked and made lobbies together to try to get whistleblowing protection laws. The fact that an European, powerful, directive exists is such a noticeable result but the fact that the transpositions appear to be for the most poor translations enabling every possible interpretation does not mean our work is ended and all the research previously performed should vanish.

evilaliv3 commented 1 year ago

I removed one of my previous argument because incorrect in relation to the usage of "shall" that should in fact be read as a "must".

danielvaknine commented 1 year ago

Thanks @evilaliv3 for your comments! I agree regarding where we have a consensus and I believe these bullet points are key functions. Then, later on (if that's what's agreed then), it's a smaller step to go from "redaction" to "deletion", if even necessary.

Adding one potential proposal (that could potentially replace your third point.

I understand your explanations on GlobaLeaks stand on helping organisations comply. Let's keep it that way!

Perhaps we should try involving more parties to get their thought on the matter?

What more can we provide to help begin/before beginning the development?

evilaliv3 commented 1 year ago

Thank you @danielvaknine for your feedback that was very fruitful.

Based on your feedback in fact i consider we just need a single privilege to enable a user to redact (and unredact) information; This privileged could be probably specified: "Give this recipient the ability to redact reports." Based on such extension we could give all users that have the privilege the possibility to see the original, redact new parts or underacts part of the report redacted by others. All the users without the privilege instead will have only access to the current status of the report without the possibility to read its redactions.

This said regarding "how could we support development?

There are many activities actually that even non developers could support and that are part of the research and development of every feature.

From a development standpoint I consider the main challenge will be identifying technically how to make it possible to redact part of the text without enabling users to hand craft an arbitrary message fully replacing what has been reported; this would in fact result in a significant vulnerability of the whistleblowing process especially if combined in the possibility to permanently save the result of the redaction activity removing the original answer. This in the context of a web application is actually not trivial.

I agree that Its worth to continue including more users in these evaluations to get proper understanding that could the be translated in the development. This would help and stimulate developers interested in developing this feature to possibly move forward this implementation.

Probably at this stage other users that cold participate in the feature design are: @pablovigo @maxmois @elbill @giorgiofraschini and @msmannan02

evilaliv3 commented 1 year ago

@danielvaknine: as an example of the activity we should try to do together i've created this prototype: https://jsfiddle.net/uv3nLoc6/

I've produced a small prototype where i'm simulating few questions and an interface enabling to redact the information. If you would like give it a try and provide feedback.

At the moment i consider the interface would result very ackward due to the redundancy of buttons. while developing a good features we should consider in fact these aspects of usability. We should consider for example how to implement a proper compact interface and use it in the whole application in relation for example to free text answers and to messages.

Screenshot from 2023-04-15 19-39-39

In this demo implementation, instead of letting the user edit the text directly i'm just collecting the coordinates of the texts that should be redacted.

danielvaknine commented 1 year ago

Really like your sketch @evilaliv3 !

As you mentioned, the problem is the redundancy of buttons. I therefore suggest having an erase-button like in this (very simple) Figma sketch: https://www.figma.com/proto/J0qiKkDez2aGfGgKAD3949/Untitled?node-id=1-22&scaling=min-zoom&page-id=0%3A1&starting-point-node-id=1%3A22

Basically having a very discrete erase-button, which then (when you click the erase-button) show the redact/unredact buttons. There could probably be just one erase-button for the whole questionnaire with the redact/unredact buttons in the end, and then one set of these for every whistleblower message.

It would be much more discrete and not interrupting or distracting the user by not spotlighting the redact function, but rather enabling it when it is needed

gianlucagilardi commented 1 year ago

As far as permanent deletion of information is concerned, please also see https://edps.europa.eu/sites/default/files/publication/19-12-17_whisteblowing_guidelines_en.pdf, especially chapter 4 § 15: "EUIs may sometimes come into possession of personal information, which is clearly of no interest or relevance to the allegations. Any such information should not be further processed. This is particularly important for special categories of information. All investigators should be made aware of this rule." And by now we all know that the fact that storing an information is "processing" according to the GDPR :) I appreciate the concerns that the provision of art. 17 introduces some new vulnerability to the WB process, but I honestly fail to see how we can maintain compliance while not providing (some) recipients the ability to permanently remove information.

On the construens side let me point out that the whole WB system as provided for by the directive does provide some "failsafes" for investigation tampering (e.g., by deleting data) since the whistleblower can escalate to external channels or even to the media in the event the internal channel fails to address appropriately the facts (s)he is providing notice of.

danielvaknine commented 1 year ago

@gianlucagilardi A very interesting point that we would need to investigate further. But I personally agree with you.

However interpretation, development of the function could proceed since it would either be a) unredactable or b) permanently deleted.

There could also be an alternative c) where admin can assign users ability to either "temporarily" redact or permanently delete, i.e. 2 different levels. The issue could however be more development time and a bit more complex for admins to interprete.

evilaliv3 commented 1 year ago

Really like your sketch @evilaliv3 !

As you mentioned, the problem is the redundancy of buttons. I therefore suggest having an erase-button like in this (very simple) Figma sketch: https://www.figma.com/proto/J0qiKkDez2aGfGgKAD3949/Untitled?node-id=1-22&scaling=min-zoom&page-id=0%3A1&starting-point-node-id=1%3A22

Basically having a very discrete erase-button, which then (when you click the erase-button) show the redact/unredact buttons. There could probably be just one erase-button for the whole questionnaire with the redact/unredact buttons in the end, and then one set of these for every whistleblower message.

It would be much more discrete and not interrupting or distracting the user by not spotlighting the redact function, but rather enabling it when it is needed

Compared to my small prototype of few hours of work, your suggestion is definitely what we should look at and very interesting from an end user perspective. We should try to understand how to onboard some developers on this matter and we may try to coordinate them on the development.

evilaliv3 commented 1 year ago

As far as permanent deletion of information is concerned, please also see https://edps.europa.eu/sites/default/files/publication/19-12-17_whisteblowing_guidelines_en.pdf, especially chapter 4 § 15: "EUIs may sometimes come into possession of personal information, which is clearly of no interest or relevance to the allegations. Any such information should not be further processed. This is particularly important for special categories of information. All investigators should be made aware of this rule." And by now we all know that the fact that storing an information is "processing" according to the GDPR :) I appreciate the concerns that the provision of art. 17 introduces some new vulnerability to the WB process, but I honestly fail to see how we can maintain compliance while not providing (some) recipients the ability to permanently remove information.

On the construens side let me point out that the whole WB system as provided for by the directive does provide some "failsafes" for investigation tampering (e.g., by deleting data) since the whistleblower can escalate to external channels or even to the media in the event the internal channel fails to address appropriately the facts (s)he is providing notice of.

Your reply is definitely on point.

With this simplistic read of article 17 we are definitely not compliant here and we should be trying to reach out for advice to the regulators explaining our concerns.

JaviAlama commented 1 year ago

Good morning, I would like to offer my opinion on this issue.

Based on our experience:

1- All information sent to us by a whistleblower must always be kept in the same format and with the same content unaltered, integrity depends on this. If a complaint ends up in court we must be able to present all the evidence as we receive it. 2- Globaleaks, in our case, is used as a complaint mailbox and as a way of communicating with the whistleblower, but not as a tool for processing complaints. In the processing tool, if necessary, copies of the documentation and information provided through the mailbox are incorporated, and all information that is not necessary is anonymised or censored in this tool. 3-About art. 17 of the Directive, it talks about the information collected during processing. This does not include information included in the complaint itself.

In summary, I believe that an original, complete and immutable version of the documentation and information received should always be kept, and that if any part of it can be censored, it should always be from a copy.

gianlucagilardi commented 1 year ago

3-About art. 17 of the Directive, it talks about the information collected during processing. This does not include information included in the complaint itself.

I am afraid the EDPS does not agree with your interpretation (https://edps.europa.eu/sites/edp/files/publication/19-12-17_whisteblowing_guidelines_en.pdf, page 6)

"4. AVOID PROCESSING EXCESSIVE PERSONAL INFORMATION 15 EUIs may sometimes come into possession of personal information, which is clearly of no interest or relevance to the allegations. Any such information should not be further processed. This is particularly important for special categories of information. All investigators should be made aware of this rule. Example 2: A whistleblower reports that a colleague has committed fraud. Within his statement, the whistleblower happens to disclose information about his colleague’s health situation. It is clear to the institution that this information is completely irrelevant to the reported wrongdoing, and therefore it should not be further processed or returned to the sender."

JaviAlama commented 1 year ago

Are you sure? I think I am right.

If medical data or religious ideology of a denounced person appear in a complaint and they are not relevant to the facts denounced, what should be done is to ignore them in the management of the complaint, in other words, not to process them.

That is what I mean, one thing is the information that the whistleblower wants to provide us with and afterwards the information that we use during the processing of the complaint, which we normally do not do from the complaint mailbox itself but from another tool.

On the other hand, in our case, in the management of the complaints we base ourselves on the facts, not on personal information, we treat nominative complaints in the same way as anonymous ones, and other information not related to the facts is not transferred to the processing.

We apply "Data Minimisation" (GDPR Art 5c) in relation to their processing.

gianlucagilardi commented 1 year ago

Are you sure? I think I am right.

If medical data or religious ideology of a denounced person appear in a complaint and they are not relevant to the facts denounced, what should be done is to ignore them in the management of the complaint, in other words, not to process them.

Since we are discussing GDPR and data protection here, the mere fact of storing some information which can be construed as personal data is processing.

"‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organization, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction;" (GDPR, art. 4.2)

Thus, if I have to stop processing data i have to permanently delete them or make them anonymous (at least sufficiently so, following the Breyer principles)

danielvaknine commented 1 year ago

We agree with Gianluca's interpretation

On Tue, 25 Apr 2023, 09:33 Gianluca Gilardi, @.***> wrote:

Are you sure? I think I am right.

If medical data or religious ideology of a denounced person appear in a complaint and they are not relevant to the facts denounced, what should be done is to ignore them in the management of the complaint, in other words, not to process them.

Since we are discussing GDPR and data protection here, the mere fact of storing some information which can be construed as personal data is processing.

"‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organization, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction;" (GDPR, art. 4.2)

Thus, if I have to stop processing data i have to permanently delete them or make them anonymous (at least sufficiently so, following the Breyer principles)

— Reply to this email directly, view it on GitHub https://github.com/globaleaks/GlobaLeaks/issues/3420#issuecomment-1521292304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZXMC4MFCJU7IJGJVXPMNVDXC543RANCNFSM6AAAAAAW4BR5HE . You are receiving this because you were mentioned.Message ID: @.***>

JaviAlama commented 1 year ago

I insist, in our case we never modify the content of a complaint, as this could reduce trust with the whistleblower and with the system itself.

On the other hand, we always keep a full copy of the content of the report because, although we anonymise those parts that we think may not be relevant for its management, before a court of law we have been asked for the full evidence and it is up to the court to decide whether it is relevant or not and whether it can be omitted or not.

Taking these decisions to anonymise information without having the possibility to obtain the complaint as it was filed can lead to legal problems in the future.

Peace and love.

danielvaknine commented 1 year ago

I understand your point @JaviAlama and the dangers you describe, but the issue of processing irrelevant information still stands. The question should rather be: How can we implement redaction/hiding of certain parts of a report without risking the integrity?

For example, double "approval" of redaction and/or the ability to both redact/unredact for certain users.

Either way, we believe it should be up to the admin to set these permissions. It should therefore be possible for organisations to keep the status quo of the platform if they'd like, even if we believe that GlobaLeaks should recommend this feature to all organisations (at least within the EU).

JaviAlama commented 1 year ago

And who decides what information is irrelevant or not? Who is responsible for modifying the content of a complaint that may end up in court?

In many cases it will be a matter of subjective criteria as it is not information that we request but information that a whistleblower provides us in relation to his or her complaint.

I think this is where the problem resides. I see the functionality itself as useful but always keeping in mind that the report should always be available as it was submitted.

Perhaps what you indicate @danielvaknine is an option, that each organisation decides whether or not to give the possibility of modifying the content of a report submitted by a whistleblower.

danielvaknine commented 1 year ago

All in all, I think we must see this function like the function for permanent deletion:

  1. It needs to be an option for admins to enable certain users to do it (to be fully compliant)
  2. Not all users must be able to do it
  3. There are the same risks in redaction/deletion-access as permanent deletion-access: there is always a risk that the user does the wrong thing – here we need to trust the organisation, admin and users to handle their whistleblowing reports correctly. (For example, the report is permanently deleted also for the whistleblower when the user deletes the report – why is that less "risky" than enabling redaction/deletion of only certain parts of reports?)
elbill commented 1 year ago

Hello @evilaliv3 at all. Let me propose a different take that may make things simpler.

Hiding personal data should be reversible. users have two checkboxes in their settings Right to hide Right to unhide (or alternatively right to view original report) They could have both or just one privilege. Hiding and unhiding should be recorded in the audit log.

I believe the feature as proposed is a must have feature from a gdpr perspective.

danielvaknine commented 1 year ago

@elbill 's option is a good one that we're comfortable with

evilaliv3 commented 1 year ago

Thank you for your feedback.

Actually by directive and national laws unfortunately we have confirmation that we also need the possibility for a privilege for permanently delete redacted information.

I think that there is strong consensus here that:

With this feature we can implement 3 different users: -- 1. users that can redact and unredact information -- 2. users that can only permanently delete redacted information by others -- 3. users that can do 1 and 2 -- 4. users that should see only the current status of the redacted report

What do you think?

danielvaknine commented 1 year ago

This sounds like an excellent idea, both enabling permanent deletion while still e.g. letting organisations have a workflow of "one person redacts, the other one 'approves' by deleting"

elbill commented 1 year ago

One addition. Any kind of permanent deletion or removal should follow the 4 eyes priciple. So you should have a consensus of 2 users. What happens if a user delibately or mistakenly removes some info? I'm getting a lot of questions from users regarding this (for the deletion of reports). That complicates the feature a little bit.

elbill commented 1 year ago

An alternative that a commercial platform (cant remember which one) is using is that they duplicate the report information and one copy is editable (remove info), while the other is archived or then deleted. Don't know if they use some workflow or kind of a consensus for the deletion. That would require extensive database modification. Just mentioning this in case it helps.

danielvaknine commented 1 year ago

I think the 4 eyes principle could be followed by having one recipient that can redact and then a different one that can delete.

It shouldn't be required to have a consensus of 2 users but this should in our opinion be up to the organisation in question. Just the possibility to make it as I described here should be enough.

JaviAlama commented 1 year ago

To ensure the integrity of the information in the event of a possible lawsuit arising from the submitted complaint, the information submitted by the complainant should never be changed. An immutable copy of the information submitted should always be maintained.

Please keep this in mind. Any changes made to the initial information submitted may mean that the evidence is altered and therefore invalid in a court.

gianlucagilardi commented 1 year ago

To ensure the integrity of the information in the event of a possible lawsuit arising from the submitted complaint, the information submitted by the complainant should never be changed. An immutable copy of the information submitted should always be maintained.

Please keep this in mind. Any changes made to the initial information submitted may mean that the evidence is altered and therefore invalid in a court.

IMU this is going to be an option so that the platform admin may choose whether being WB directive/GDPR compliant or not. In any event the whistleblower is going to have access to an unredacted copy.

gianlucagilardi commented 1 year ago

I think the 4 eyes principle could be followed by having one recipient that can redact and then a different one that can delete.

It shouldn't be required to have a consensus of 2 users but this should in our opinion be up to the organisation in question. Just the possibility to make it as I described here should be enough.

I am inclined to think that there are not going to be lots of companies where the 4-eyes principle is going to be feasible. Heck, I would be extremely happy if many micro-companies would actually have 2 eyes on reports....

JaviAlama commented 1 year ago

It is not only about compliance, but also about the whistleblower's trust that his or her complaint will not be changed.

The whistleblower, and also the Receiving Entity, must always be able to access the original content of the complaint.

When a complaint ends up in the public prosecutor's office, at least in Spain, the first thing they ask for is all the documentation as it was presented, and in this case the GDPR is below the regulations on criminal investigation.

The GDPR is applied in the management of the complaint internally through pseudonymisation and externally through anonymisation, but always keeping the initial content unalterable and only accessible by those who legally must have access as data controllers.

gianlucagilardi commented 1 year ago

It is not only about compliance, but also about the whistleblower's trust that his or her complaint will not be changed.

The whistleblower, and also the Receiving Entity, must always be able to access the original content of the complaint.

When a complaint ends up in the public prosecutor's office, at least in Spain, the first thing they ask for is all the documentation as it was presented, and in this case the GDPR is below the regulations on criminal investigation.

The GDPR is applied in the management of the complaint internally through pseudonymisation and externally through anonymisation, but always keeping the initial content unalterable and only accessible by those who legally must have access as data controllers.

I guess we just have to agree to disagree :) My only suggestion is that you bring your point of view and interpretation along with the Directive and your local regulation (which, if I am not mistaken, is Ley 2/2023, de 20 de febrero, reguladora de la protección de las personas que informen sobre infracciones normativas y de lucha contra la corrupción especially as regards art. 29) to your DPO :)

JaviAlama commented 1 year ago

It is not only about compliance, but also about the whistleblower's trust that his or her complaint will not be changed. The whistleblower, and also the Receiving Entity, must always be able to access the original content of the complaint. When a complaint ends up in the public prosecutor's office, at least in Spain, the first thing they ask for is all the documentation as it was presented, and in this case the GDPR is below the regulations on criminal investigation. The GDPR is applied in the management of the complaint internally through pseudonymisation and externally through anonymisation, but always keeping the initial content unalterable and only accessible by those who legally must have access as data controllers.

I guess we just have to agree to disagree :) My only suggestion is that you bring your point of view and interpretation along with the Directive and your local regulation (which, if I am not mistaken, is Ley 2/2023, de 20 de febrero, reguladora de la protección de las personas que informen sobre infracciones normativas y de lucha contra la corrupción especially as regards art. 29) to your DPO :)

As long as it is optional, there is no need to worry (at least in our case).

Regarding art 29, it authorises the treatment of personal data according to law 7/2021, and this is the law that authorises the treatment of personal data by competent authorities in the exercise of investigative tasks.

In short, in our case it is clear, it is not interpretable, we must always keep an immutable copy of the complaint as it is presented. And if, as I said, the mechanisms that are being considered are going to be of optional use, there is no problem.

gianlucagilardi commented 1 year ago

It is not only about compliance, but also about the whistleblower's trust that his or her complaint will not be changed. The whistleblower, and also the Receiving Entity, must always be able to access the original content of the complaint. When a complaint ends up in the public prosecutor's office, at least in Spain, the first thing they ask for is all the documentation as it was presented, and in this case the GDPR is below the regulations on criminal investigation. The GDPR is applied in the management of the complaint internally through pseudonymisation and externally through anonymisation, but always keeping the initial content unalterable and only accessible by those who legally must have access as data controllers.

I guess we just have to agree to disagree :) My only suggestion is that you bring your point of view and interpretation along with the Directive and your local regulation (which, if I am not mistaken, is Ley 2/2023, de 20 de febrero, reguladora de la protección de las personas que informen sobre infracciones normativas y de lucha contra la corrupción especially as regards art. 29) to your DPO :)

As long as it is optional, there is no need to worry (at least in our case).

Regarding art 29, it authorises the treatment of personal data according to law 7/2021, and this is the law that authorises the treatment of personal data by competent authorities in the exercise of investigative tasks.

I was referring to this :)

"No se recopilarán datos personales cuya pertinencia no resulte manifiesta para tratar una información específica o, si se recopilan por accidente, se eliminarán sin dilación indebida."

msmannan00 commented 1 year ago

I am attaching the proposed database changes for the community to give suggestions Masking.docx

msmannan00 commented 1 year ago

@evilaliv3

Masking Admin Side

Administrators will have the authority to grant recipients the privilege to mask information, the privilege to delete masks, or both. Database Schema

A new table called "masking" will be created, which will establish a one-to-many relationship with the "fieldoption," "comment," and "message" tables.

The "masking" table will contain the following fields:

unique_id: The unique identifier for each record.
content_id: The primary key for the other three tables (fieldoption, comment, and message).
temporary_ranges: Represents the temporary masking content text initial and last index. If a recipient masks a temporary range, the data will be stored in this column.
permanent_ranges: Represents the permanent mask initial and last index. If a recipient masks a permanent range, the data will be shifted from temporary_ranges to permanent_ranges against the respective unique_id.
mask_date: The date when the masking was applied.The word "ranges" represents the length of the masked text, indicating the number of characters or portions of the original text that are subtracted. The range can vary and may involve multiple segments.

Recipient Side

For the mathematical functionality of masking, the range of the masked text can be calculated by determining the first and last indices of the masked portion. Multiple ranges can be computed, which will differ from each other and be in sequential order. The final result of the masking calculation will be in the form of ranges (e.g., "8-9, 10-22, ...").

Once the masking calculation is done, the resulting ranges will be stored (in JSON format) in the database.

There are three cases that will be implemented: temporary masking, permanent masking, and both. Temporary Masking

  1. In this case, recipients have the right to mask information. The masked text will be displayed in edit mode, while the original text will be displayed in the report. Recipients do not have the right to unmask or reveal the original text. Only one request will be processed, which will insert the ranges (in JSON format) into the temporary_ranges column in the masking table. Permanent Masking

  2. In this case, recipients have the right to delete the masked information, but they cannot mask the text themselves. Recipients can unmask or reveal the permanently masked text data. The permanently masked text data will be displayed in the original report text and edit mode. Both Temporary and Permanent Masking

  3. In this case, recipients have the right to both mask and unmask the text. To differentiate between the temporarily masked and permanently masked text, the permanently masked text will be displayed in red color, while temporary masking will be displayed in blue color.

The masking table data will be deleted in two scenarios:

  1. Report Expiration: The data will be automatically deleted based on the report's expiration date. When the expiration date is reached, any corresponding records in the masking table will be removed.
  2. Recipient Deletion: If a recipient chooses to delete the report, the data in the masking table associated with that report will also be deleted. This ensures that any masking information specific to that report is permanently removed from the database.

image

danielvaknine commented 1 year ago

I am attaching the proposed database changes for the community to give suggestions Masking.docx

Had a quick look and if I understood it correctly, a recipient with "temporary masking"-permission won't be able to unmask? The text there says: "Recipients do not have the right to unmask or reveal the original text."

And a recipient with "permanent masking"-permission will be able to see the "deleted" information? The text there says: "Recipients can unmask or reveal the permanently masked text data."

As we've understood it from before, the permissions were:

  1. Unmask/mask
  2. Permanently redact (delete) masked information Where a recipient could then be permitted 1, 2 or 1&2, according to the admin's preference.

Please correct me if I've misunderstood

evilaliv3 commented 1 year ago

Thank you @msmannan00 for sharing this notes.

I think like @danielvaknine that:

1) after temporary masking any recipient with the privilege to temporary mask information should be able to unmask; this is necessary also to correct mistakes;

2) after after permanent masking (that i suggest to call redaction) no one should be able to recover the data; this operation should permanently delete the redacted information.

Regarding the mask_date i think it should be updated containing the last date of the editing of the mask. Then in the audit log we should probably keep track of every mask applied to know when mask where changed and which was their value. The log will need to track the user that modified the mask as well as user_id.

Regarding the Foreign Key on the object_id, unfortunately i think that could not be applied. The same attribute in fact could not be a foreign key with different tables. I suggest to add internaltip_id as foreign (so that the masks is deleted if the report is deleted) and then leave the object_id without any foreign key

gianlucagilardi commented 1 year ago

Thank you @msmannan00 for sharing this notes.

[...]

  1. after after permanent masking (that i suggest to call redaction) no one should be able to recover the data; this operation should permanently delete the redacted information.

I stress the point that permanent masking should be obtained by removing the (previously temporarily) masked data from the DB as such and not simply making invisible that information to anyone while keeping it in the DB, which if I understand correctly might be the explanation of having both a temporary_ranges and a permanent_ranges field in the new table. Since we are required to stop processing data, we cannot keep a copy of them in the DB and simply making them "programmatically invisible" unfortunately would not work.

Of course I might have interpreted incorrectly what @msmannan00 explained and we are actually deleting data, so we would be good :)

msmannan00 commented 1 year ago

Thank you @msmannan00 for sharing this notes.

[...]

  1. after after permanent masking (that i suggest to call redaction) no one should be able to recover the data; this operation should permanently delete the redacted information.

I stress the point that permanent masking should be obtained by removing the (previously temporarily) masked data from the DB as such and not simply making invisible that information to anyone while keeping it in the DB, which if I understand correctly might be the explanation of having both a temporary_ranges and a permanent_ranges field in the new table. Since we are required to stop processing data, we cannot keep a copy of them in the DB and simply making them "programmatically invisible" unfortunately would not work.

Of course I might have interpreted incorrectly what @msmannan00 explained and we are actually deleting data, so we would be good :)

Thanks, actually permanent keys range are added for different reason, When permanent masking is done it removes the content itself from database. permanent ranges are saved to display stars the make deleted report more intractable. temporary masking is separated because user can mask report multiple times before permanently masking it. both of the masking dates are deleted at the point when report is deleted itself

msmannan00 commented 1 year ago

Thank you @msmannan00 for sharing this notes.

I think like @danielvaknine that:

1. after temporary masking any recipient with the privilege to temporary mask information should be able to unmask; this is necessary also to correct mistakes;

2. after after permanent masking (that i suggest to call redaction) no one should be able to recover the data; this operation should permanently delete the redacted information.

Regarding the mask_date i think it should be updated containing the last date of the editing of the mask. Then in the audit log we should probably keep track of every mask applied to know when mask where changed and which was their value. The log will need to track the user that modified the mask as well as user_id.

Regarding the Foreign Key on the object_id, unfortunately i think that could not be applied. The same attribute in fact could not be a foreign key with different tables. I suggest to add internaltip_id as foreign (so that the masks is deleted if the report is deleted) and then leave the object_id without any foreign key

Thanks that was something in mind actually, the content id was meant to be the placeholder to decide what would be the foreign key. if rest is fine do tell so that I can start making the suggested changes

msmannan00 commented 1 year ago

Current update, User option to grant right to temporary and permanent mask the report elements

Screenshot from 2023-06-18 15-59-26

on tip page the header file option to edit report hides and shows accordingly

Screenshot from 2023-06-18 15-51-52 Screenshot from 2023-06-18 15-51-44

after toggle of masking button, edit button would appear on each field that can be masked Screenshot from 2023-06-18 15-52-09

after clicking on any button text for that report would show in popup with three buttons

  1. mask (to mask text)
  2. unmask (to unmask text)
  3. reduct (will add temporary mask entry for user with temporary rights) (would completely remove it permanetly for user with related rights)

Screenshot from 2023-06-18 15-52-35 Screenshot from 2023-06-18 15-52-43

for files user can only mark it to mask the file and other user with permanent right can only delete the file. since the current user is having both rights you would be able to see both options

Screenshot from 2023-06-18 15-54-30 Screenshot from 2023-06-18 15-54-41

after permanent masking you would be able to see the report being masked completely even from backend

Screenshot from 2023-06-18 15-57-30

msmannan00 commented 1 year ago

we will start testing of the entire feature from tomorrow and would share you the live demo before making pull request to globaleaks repository

gianlucagilardi commented 1 year ago

Brilliant @msmannan00!

Just a tip: it is "redact", not "reduct" ;)

danielvaknine commented 1 year ago

Awesome @msmannan00 ! Please let us know when the demo is live!

msmannan00 commented 1 year ago

Awesome @msmannan00 ! Please let us know when the demo is live!

Sure

JaviAlama commented 1 year ago

Great work and very wise to incorporate screenshots and instructions on how this feature will work.

That's what I was referring to in the last meeting, all new features should be properly explained and with examples.

Congratulations again.