search

topcoder-platform / forums-plugins

0 stars 0 forks source link

[$250] Admin plugin? #14

Closed jmgasper closed 4 years ago

jmgasper commented 4 years ago

@atelomycterus - Topcoder needs to comply with some GDPR and security requirements. To make that easy, we've been asked to investigate if we can create an admin plugin that allows an admin to:

Idea

What I'm thinking (and I'm not sure if this is possible) is that we create a new plugin (or extend the existing one) to add a new tab and / or new options to an existing tab, on the dashboard site with the required functionality:

Screen Shot 2020-05-06 at 09 41 26

jmgasper commented 4 years ago

Contest https://www.topcoder.com/challenges/30124447 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

jmgasper commented 4 years ago

Contest https://www.topcoder.com/challenges/30124447 has been updated - it has been assigned to obog.

This is an automated message for ghostar via Topcoder X

atelomycterus commented 4 years ago

@jmgasper It is necessary to clarify some details in order to understand how much work is required, whether the required data is saved in the forum, or it is required to implement new functionality.

  1. Vanilla provides default logging for some functionality. I am worried that, for example, if we need to save data for 3 months or more, then maybe with large amounts of data this can affect system performance. Perhaps we need to make your own history table, it depends on requirements. Vanilla uses caching in some cases, but actively uses tables that store messages and notifications. The default settings are to delete data and truncate tables after a few weeks. What period should we store data?

  2. Admin have the ability to identify all messages from a user

What did you mean by all messages? Did you mean any user generated conent? if yes, then user generated content in Vanilla:

I think that there are no problems and the data can be easily identified by user.

  1. Platform must support the ability to export data on a per-user basis

I haven't seen any export data functionality in Vanilla. I'll check it.

I think the following data should be exported on a per-user basis:

Please correct me if I am wrong or some data missing.

I will provide information in which tables and what format the data is stored. Some data is stored as templates (e.g. '{userId} wrote {message}') and template parsing is performed during page loading/rendering. Perhaps here some difficulties will arise, because Vanilla provides the ability to modify content and modify data in plugins using hooks.

  1. Must log who posted a message, date, time, and contents if text (seems like this is already done by default - I'm getting clarification)

Vanilla logs some data. As far as I remember, there is no data logging in some cases, for example, the author has changed the message. The message table contains the creation date, update date, author, message body but the previous message body is not logged. The previous body is logged if the message is edited by another user who has the role of a moderator.

  1. What about the "Right to be forgotten" GDPR rule? It means to give users the ability to get their account data removed on request Do you plan to remove data manually? This task can be automated:

    • an user data (email and so on) should be replaced with a new specific one because there might be discussions/posts/comments/messages and so on
    • replace a profile image with a default Vanilla icon
    • username/password need to be changed because user could try to log in
    • all notifications should be set to be off
    • maybe something else.

  2. Implementation I'll look through the source code to understand how best to implement it as a plugin or as an application. Keep you posted.

jmgasper commented 4 years ago

  1. The default is to remove public messages from the forum? We'll have to disable that completely, if so.

  2. This would be chats and forum content - basically anything generated by a given user.

  3. That should work for now. I don't think we need IP / login date information.

  4. We can just export raw data, as long as it captures what the user has input. I don't really think it matters if it's very "user friendly", as long as the data is there.

  5. I'll ask about "forgotten" rights and what that entails.

atelomycterus commented 4 years ago

@jmgasper

  1. The default is to remove public messages from the forum? We'll have to disable that completely, if so.

It became clear how it works. Vanilla saves discussions into the 'discussion' table, comments into the 'comment' table and so on. Vanilla uses an auxiliary table called 'Activity' that stores activity data. This table is pruned periodically. It might be configured with 'Garden.PruneActivityAfter' (2 monthes by default). Actual data is not deleted from tables (discussion, comments and so on) when the 'Activity' table is pruned.

For example, User creates a comment. On the backend side:

  1. Most of the required data can be exported using the REST API. We can call the appropriate REST methods, convert the data to a suitable format and generate a report.

    Discussions Get all discussions created by User: http://127.0.0.1/settings/swagger#/Discussions/get_discussions http://127.0.0.1/api/v2/discussions Query parameters insertUserID - Filter by author. dateInserted - Filter by insert date dateUpdated - Filter by update date

We can filter based on a range of date values. Read about date filters - https://success.vanillaforums.com/kb/articles/45-date-filters.

Output of the REST API method:

[
  {
    "discussionID": 6,
    "type": null,
    "name": "Tech requirements",
    "body": "<p>The list of all tech requirements:</p><ul><li>Requirement 1</li><li>Requirement 2</li><li>Requirement 3</li><li>Requirement 4</li></ul>",
    "categoryID": 17,
    "dateInserted": "2020-05-06T13:43:05+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "pinned": false,
    "pinLocation": null,
    "closed": false,
    "sink": false,
    "countComments": 0,
    "countViews": 1,
    "score": null,
    "url": "http://127.0.0.1/discussion/6/tech-requirements",
    "canonicalUrl": "http://127.0.0.1/discussion/6/tech-requirements",
    "bookmarked": false,
    "unread": false
  }
]

Drafts (not published discussion) Rest API /drafts - get drafts only for CURRENT user. Need to expand REST API to filter by userID, dates.

[
  {
    "draftID": 2,
    "recordType": "discussion",
    "parentRecordID": 17,
    "attributes": {
      "name": "Tech requirements",
      "tags": null,
      "closed": 0,
      "announce": 0,
      "sink": 0,
      "body": "[{\"insert\":\"The list of all tech requirements:\\nRequirement 1\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 2\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 3\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 4\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"}]",
      "format": "Rich"
    },
    "insertUserID": 2,
    "dateInserted": "2020-05-06T13:43:17+00:00",
    "updateUserID": 2,
    "dateUpdated": "2020-05-06T13:43:17+00:00"
  }
]

Comments
http://127.0.0.1/settings/swagger#/Comments/get_comments http://127.0.0.1/api/v2/comments Query parameters insertUserID - Filter by author. dateInserted Filter by insert date dateUpdated Filter by update date

Output:

[
  {
    "commentID": 1,
    "discussionID": 1,
    "body": "<div class=\"js-embed embedResponsive\" data-embedJson=\"{&quot;recordID&quot;:1,&quot;recordType&quot;:&quot;discussion&quot;,&quot;body&quot;:&quot;&lt;p&gt;public body&lt;\\/p&gt;&quot;,&quot;bodyRaw&quot;:[{&quot;insert&quot;:&quot;public body\\n&quot;}],&quot;format&quot;:&quot;Rich&quot;,&quot;dateInserted&quot;:&quot;2020-05-06T10:23:03+00:00&quot;,&quot;insertUser&quot;:{&quot;userID&quot;:2,&quot;name&quot;:&quot;obog&quot;,&quot;photoUrl&quot;:&quot;http:\\/\\/127.0.0.1\\/applications\\/dashboard\\/design\\/images\\/defaulticon.png&quot;,&quot;dateLastActive&quot;:&quot;2020-05-01T16:41:23+00:00&quot;},&quot;displayOptions&quot;:{&quot;showUserLabel&quot;:false,&quot;showCompactUserInfo&quot;:true,&quot;showDiscussionLink&quot;:true,&quot;showPostLink&quot;:true,&quot;showCategoryLink&quot;:false,&quot;renderFullContent&quot;:false,&quot;expandByDefault&quot;:false},&quot;url&quot;:&quot;http:\\/\\/127.0.0.1\\/discussion\\/1\\/public-thread&quot;,&quot;embedType&quot;:&quot;quote&quot;,&quot;name&quot;:&quot;Public  thread&quot;}\">\n    <a href=\"http://127.0.0.1/discussion/1/public-thread\">\n        http://127.0.0.1/discussion/1/public-thread\n    </a>\n</div><p>Quoted</p>",
    "dateInserted": "2020-05-06T10:23:41+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "score": null,
    "url": "http://127.0.0.1/discussion/comment/1#Comment_1"
  },
  {
    "commentID": 2,
    "discussionID": 3,
    "body": "<p>I have some questions.</p>",
    "dateInserted": "2020-05-06T13:37:08+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "score": null,
    "url": "http://127.0.0.1/discussion/comment/2#Comment_2"
  }
]

Conversations and Messages Get all conversations with messages: http://127.0.0.1/settings/swagger#/Conversations/get_conversations http://127.0.0.1/api/v2/conversations Query parameters insertUserID - Filter by author. participantUserID Filter by participating user. dateInserted Filter by insert date dateUpdated Filter by update date

Example of conversations: image

[
  {
    "conversationID": 1,
    "name": "tonyj",
    "body": "<p>Hello again</p>",
    "url": "http://127.0.0.1/messages/1",
    "dateInserted": "2020-05-06T14:27:53+00:00",
    "insertUserID": 3,
    "countParticipants": 2,
    "participants": [
      {
        "userID": 3,
        "user": {
          "userID": 3,
          "name": "tonyj",
          "photoUrl": "https://topcoder-dev-media.s3.amazonaws.com/member/profile/TonyJ-1558928928786.png",
          "dateLastActive": "2020-05-06T13:44:56+00:00"
        },
        "status": "participating"
      },
      {
        "userID": 2,
        "user": {
          "userID": 2,
          "name": "obog",
          "photoUrl": "http://127.0.0.1/applications/dashboard/design/images/defaulticon.png",
          "dateLastActive": "2020-05-06T14:42:09+00:00"
        },
        "status": "participating"
      }
    ],
    "countMessages": 3,
    "unread": false,
    "countUnread": 0
  }
]

Get messages by Author only http://127.0.0.1/settings/swagger#/Messages/get_messages http://127.0.0.1/api/v2/messages Query parameters insertUserID - Filter by author

Output:

[
  {
    "messageID": 3,
    "conversationID": 1,
    "body": "<p>Hello all</p>",
    "insertUserID": 2,
    "dateInserted": "2020-05-06T14:44:25+00:00"
  }
]

Users http://127.0.0.1/settings/swagger#/Users/get_users__id_ http://127.0.0.1/api/v2/users/{userid}

{
  "userID": 2,
  "name": "obog",
  "email": "test@test.com",
  "photoUrl": "http://127.0.0.1/applications/dashboard/design/images/defaulticon.png",
  "points": 0,
  "emailConfirmed": true,
  "showEmail": false,
  "bypassSpam": false,
  "banned": 0,
  "dateInserted": "2020-05-01T16:41:23+00:00",
  "dateLastActive": "2020-05-06T14:42:09+00:00",
  "dateUpdated": "2020-05-06T14:17:34+00:00",
  "roles": [
    {
      "roleID": 16,
      "name": "Administrator"
    },
    {
      "roleID": 32,
      "name": "Moderator"
    }
  ]
}

No REST API for entities

a. Tagging allows users to add a tag to discussions they start in order to make them more discoverable. Admin settings has the 'Enable Tagging' option. It is disabled by default.

No REST API to get Tags by User. The tags is saved into the 'Tag' table.

b. System messages. The 'Message' table doesn't have any user-related columns image

It is displayed: image

c. Attachments Attachments can be made to the following content: -- discussions -- comments

the 'Attachment' has the InsertUserID/UpdateUserID column.

d. the 'UserIP' table contains UserID, IPAddress, DateInserted, DateUpdated. The GDPR states that IP addresses should be considered personal data as it enters the scope of ‘online identifiers’ ( See https://eugdprcompliant.com/personal-data/)

e. Media (images) No REST API to search by media items. The 'Media' table has the 'insertUserID' column.

  1. Implementation It can be implemented as a plugin. Admin Dashboard fires navigation events ('init' and 'GetAppSettingsMenuItems'). So we can add a new tab with with permissions.

The tab configurations looks like 

    "TabName" => [
            "permission" => [
              "permission name"
            ],
            "section" => "TabName",
            "title" => "Tab name",
            "description" => "Tab descripton",
            "url" => "url to handle it",
            "empty" => true // no side menu
        ],
jmgasper commented 4 years ago
  1. Ok, cool. We're more interested in the discussion content itself and not so much the activity

It sounds like we're in a good spot to get this implemented into the plugin. Do we need to clarify anything else?

jmgasper commented 4 years ago

Payment task has been updated: https://software.topcoder.com/review/actions/ViewProjectDetails?pid=30124447

This is an automated message for ghostar via Topcoder X

atelomycterus commented 4 years ago

@jmgasper We need to clarify only the 'forgotten' rule

I'll ask about "forgotten" rights and what that entails.