Enh: Download User-data

[ArchBlood]

resolves #20

[ArchBlood]

I'm still new to using the Rest API, so here's a suggestion if we want to rely on the Rest API module for ExportController, I've tested this method which works the same as the original P/R but it requires that the Rest module actually be enabled before use, so I'd also have to make this configurable from the admin UI and maybe add a warning message stating that the Rest module is required to use this function;

<?php

namespace humhub\modules\legal\controllers;

use Yii;
use humhub\modules\user\components\BaseAccountController;
use humhub\modules\rest\definitions\UserDefinitions;
use humhub\modules\rest\definitions\PostDefinitions;
use humhub\modules\post\models\Post;
use yii\web\Response;

/**
 * ExportController handles exporting user data as JSON.
 */
class ExportController extends BaseAccountController
{
    /**
     * Renders the view displaying the exported user data as JSON.
     *
     * @return string The rendered view.
     */
    public function actionIndex()
    {
        return $this->renderAjax('index');
    }

    /**
     * Downloads the exported user data as a JSON file.
     *
     * @return Response The file response.
     */
    public function actionDownload()
    {
        // Check if the REST module is enabled
        if (!Yii::$app->hasModule('rest')) {
            throw new \yii\base\Exception('REST module is not enabled.');
        }

        $userData = $this->getUserData();
        $postData = $this->getPostData();

        // Combine user and post data
        $data = [
            'user' => $userData,
            'post' => $postData,
        ];

        // Convert data to JSON
        $jsonData = json_encode($data, JSON_PRETTY_PRINT);

        // Set headers for JSON file download
        Yii::$app->response->format = Response::FORMAT_RAW;
        Yii::$app->response->headers->add('Content-Type', 'application/json');
        Yii::$app->response->headers->add('Content-Disposition', 'attachment; filename="userdata.json"');

        // Output JSON data to response body
        Yii::$app->response->content = $jsonData;

        // Send the response
        return Yii::$app->response;
    }

    /**
     * Retrieves user data.
     *
     * @return array The user data.
     */
    private function getUserData()
    {
        $currentUser = Yii::$app->user->getIdentity();
        if (Yii::$app->hasModule('rest')) {
            return UserDefinitions::getUser($currentUser);
        } else {
            return $currentUser->attributes;
        }
    }

    /**
     * Retrieves current user's posts and returns as JSON.
     *
     * @return array The JSON data.
     */
    private function getPostData()
    {
        $currentUser = Yii::$app->user->getIdentity();
        if (Yii::$app->hasModule('rest')) {
            $userPosts = Post::find()
                ->where(['created_by' => $currentUser->id])
                ->all();
            return array_map(function($post) {
                return PostDefinitions::getPost($post);
            }, $userPosts);
        } else {
            return [];
        }
    }
}

userdata.json

{
    "user": {
        "id": 1,
        "guid": "redacted",
        "display_name": "Tony GM",
        "url": "https:\/\/domain.com\/u\/archblood\/",
        "account": {
            "id": 1,
            "guid": "redacted",
            "username": "ArchBlood",
            "email": "hello@greenmeteor.net",
            "visibility": 2,
            "status": 1,
            "tags": [],
            "language": "en-US",
            "time_zone": "America\/New_York",
            "contentcontainer_id": 1,
            "authclient": "local",
            "authclient_id": null,
            "last_login": "2024-03-01 23:50:50"
        },
        "profile": {
            "firstname": "Tony",
            "lastname": "GM",
            "title": null,
            "gender": "male",
            "street": null,
            "zip": null,
            "city": "New York",
            "country": "United States",
            "state": null,
            "birthday_hide_year": 0,
            "birthday": "1992-11-07",
            "about": "This is about me?",
            "phone_private": null,
            "phone_work": null,
            "mobile": null,
            "fax": null,
            "im_skype": null,
            "im_xmpp": null,
            "url": null,
            "url_facebook": null,
            "url_linkedin": null,
            "url_xing": null,
            "url_youtube": null,
            "url_vimeo": null,
            "url_flickr": null,
            "url_myspace": null,
            "url_twitter": null,
            "certified": null,
            "certified_by": null
        }
    },
    "post": [
        {
            "id": redacted,
            "message": "Test",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-26 08:56:11",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 10:51:22",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Testing",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-27 15:23:54",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 10:51:24",
                    "scheduled_at": null,
                    "url": "\/s\/support\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 11,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Test",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 22:44:38",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Test",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 22:50:41",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Testing",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 23:11:29",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Testing",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-31 13:55:11",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-31 13:55:11",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 12,
                        "name": "Test"
                    }
                ],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Testing",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-01-31 16:39:42",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-31 16:39:42",
                    "scheduled_at": null,
                    "url": "\/s\/general-discussions\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 10,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 13,
                        "name": "Demo"
                    }
                ],
                "files": []
            }
        },
        {
            "id": redacted,
            "message": "Demo test",
            "content": {
                "id": redacted,
                "metadata": {
                    "id": redacted,
                    "guid": "redacted",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": redacted,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "created_at": "2024-02-01 07:37:32",
                    "updated_by": {
                        "id": 1,
                        "guid": "redacted",
                        "display_name": "Tony GM",
                        "url": "https:\/\/domain.com\/u\/archblood\/"
                    },
                    "updated_at": "2024-02-01 07:37:32",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=redacted",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 12,
                        "name": "Test"
                    },
                    {
                        "id": 14,
                        "name": "Demo"
                    }
                ],
                "files": []
            }
        }
    ]
}

[ArchBlood]

Should be good for a review, I've made it so that it uses the Rest API module and it is configurable with a checkbox from the admin settings.

[ArchBlood]

Now the checkbox can only be clicked if the REST API module is installed and enabled else the field is disabled.

[luke-]

Thank you, I like the PR apporach.

I would like to clarify/implement the following points, before adding this feature

• All core comments (like, profiles, comments) should be included.
• Also attached files
• There must be a possibility that modules can also inject their content
• We need to check whether activities/notifications must also be included according to GDPR.
• When we include more content types & files, the more we might have to create a file on the server via an ActiveJob and send the user a download link by e-mail. (Could be implemented in a later stage).

[ArchBlood]

Thank you, I like the PR apporach.

I would like to clarify/implement the following points, before adding this feature

• All core comments (like, profiles, comments) should be included.
• Also attached files
• There must be a possibility that modules can also inject their content
• We need to check whether activities/notifications must also be included according to GDPR.
• When we include more content types & files, the more we might have to create a file on the server via an ActiveJob and send the user a download link by e-mail. (Could be implemented in a later stage).

I completely agree here, and from looking at https://github.com/humhub-contrib/legal/pull/68#issuecomment-1974308215 it does include comments, files, topics and like entries if they are included in post data, but I'm not sure if we should make files downloadable through this method as it would more than likely cause server load on large communities and require refactoring to retrieve the correct files;

Data from modules also should be possible as well as long as the data can be implemented from the REST API module definitions, this would require more functions or a single getModuleData() function, but is easily implementable by following the provided code within the ExportController;

I would think activities would probably fall under this, but that's only depending on how activities are used, not sure about notifications themselves;

I had thought of this but didn't think it would be a good idea unless the download link and the data generated for the download link came with an expirable date attached. 🤔

[luke-]

So it is important that we only include comments to download where the user is also the author. The current solution adds all latest comments to a posts. https://marketplace.humhub.com/module/rest/docs/html/post.html#/paths/~1post~1%7Bid%7D/get

[ArchBlood]

So it is important that we only include comments to download where the user is also the author. The current solution adds all latest comments to a posts. https://marketplace.humhub.com/module/rest/docs/html/post.html#/paths/~1post~1%7Bid%7D/get

In the current implementation it should already be possible to include the comments to the download where the user is the author, the issue that I'm seeing is that it does not allow for getting comments on other people's posts, this is more than likely because of using Yii::$app->user->getIdentity() as current user;

Test Payload

{
    "user": {
        "id": 1,
        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
        "display_name": "Tony GM",
        "url": "https:\/\/greenmeteor.net\/u\/archblood\/",
        "account": {
            "id": 1,
            "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
            "username": "ArchBlood",
            "email": "hello@greenmeteor.net",
            "visibility": 2,
            "status": 1,
            "tags": [],
            "language": "en-US",
            "time_zone": "America\/New_York",
            "contentcontainer_id": 1,
            "authclient": "local",
            "authclient_id": null,
            "last_login": "2024-03-04 10:10:38"
        },
        "profile": {
            "firstname": "Tony",
            "lastname": "GM",
            "title": null,
            "gender": "male",
            "street": null,
            "zip": null,
            "city": "New York",
            "country": "United States",
            "state": null,
            "birthday_hide_year": 0,
            "birthday": "1992-11-07",
            "about": "This is about me?",
            "phone_private": null,
            "phone_work": null,
            "mobile": null,
            "fax": null,
            "im_skype": null,
            "im_xmpp": null,
            "url": null,
            "url_facebook": null,
            "url_linkedin": null,
            "url_xing": null,
            "url_youtube": null,
            "url_vimeo": null,
            "url_flickr": null,
            "url_myspace": null,
            "url_twitter": null,
            "certified": null,
            "certified_by": null
        }
    },
    "post": [
        {
            "id": 3763,
            "message": "Test",
            "content": {
                "id": 8026,
                "metadata": {
                    "id": 8026,
                    "guid": "5a94471c-7be6-4354-a173-99017d2039ad",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3763,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-26 08:56:11",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 10:51:22",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=3763",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": 3764,
            "message": "Testing",
            "content": {
                "id": 8029,
                "metadata": {
                    "id": 8029,
                    "guid": "9e392bd4-cf58-48e5-a96f-b2729345a301",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3764,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-27 15:23:54",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 10:51:24",
                    "scheduled_at": null,
                    "url": "\/s\/support\/post\/post\/view?id=3764",
                    "contentcontainer_id": 11,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": 3765,
            "message": "Test",
            "content": {
                "id": 8043,
                "metadata": {
                    "id": 8043,
                    "guid": "df3bf333-a830-4880-b0f3-ea7c7d68406c",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3765,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 22:44:38",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=3765",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": 3766,
            "message": "Test",
            "content": {
                "id": 8046,
                "metadata": {
                    "id": 8046,
                    "guid": "cc390449-c714-4ed3-a37c-6b73d8f13201",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3766,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 22:50:41",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=3766",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": 3767,
            "message": "Testing",
            "content": {
                "id": 8048,
                "metadata": {
                    "id": 8048,
                    "guid": "cdc9ef05-28e2-48b1-9a34-b6320545e9f1",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3767,
                    "visibility": 0,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-28 23:11:29",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-28 23:13:02",
                    "scheduled_at": null,
                    "url": "\/s\/testing\/post\/post\/view?id=3767",
                    "contentcontainer_id": 13,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        },
        {
            "id": 3774,
            "message": "Testing",
            "content": {
                "id": 8073,
                "metadata": {
                    "id": 8073,
                    "guid": "fb913a75-0661-4b1b-98e9-109dc88b8c7c",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3774,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-31 13:55:11",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-31 13:55:11",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=3774",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 12,
                        "name": "Test"
                    }
                ],
                "files": []
            }
        },
        {
            "id": 3775,
            "message": "Testing",
            "content": {
                "id": 8075,
                "metadata": {
                    "id": 8075,
                    "guid": "d9995604-9772-4498-907f-62aae7e93f5f",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3775,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-01-31 16:39:42",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-01-31 16:39:42",
                    "scheduled_at": null,
                    "url": "\/s\/general-discussions\/post\/post\/view?id=3775",
                    "contentcontainer_id": 10,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 13,
                        "name": "Demo"
                    }
                ],
                "files": []
            }
        },
        {
            "id": 3776,
            "message": "Demo test",
            "content": {
                "id": 8077,
                "metadata": {
                    "id": 8077,
                    "guid": "2bd5cd8c-f15f-4cff-8813-f19a8552fd50",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3776,
                    "visibility": 1,
                    "state": 1,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-02-01 07:37:32",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-02-01 07:37:32",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=3776",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "1",
                    "latest": [
                        {
                            "id": 45,
                            "message": "Test",
                            "objectModel": "humhub\\modules\\post\\models\\Post",
                            "objectId": 3776,
                            "createdBy": {
                                "id": 1,
                                "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                                "display_name": "Tony GM",
                                "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                            },
                            "createdAt": "2024-03-04 11:31:05",
                            "likes": {
                                "total": 0
                            },
                            "files": []
                        }
                    ]
                },
                "likes": {
                    "total": 0
                },
                "topics": [
                    {
                        "id": 12,
                        "name": "Test"
                    },
                    {
                        "id": 14,
                        "name": "Demo"
                    }
                ],
                "files": []
            }
        },
        {
            "id": 3778,
            "message": ":undefined:",
            "content": {
                "id": 8104,
                "metadata": {
                    "id": 8104,
                    "guid": "955b3cb6-fbd2-4aa2-9183-6b2912953b59",
                    "object_model": "humhub\\modules\\post\\models\\Post",
                    "object_id": 3778,
                    "visibility": 1,
                    "state": 100,
                    "archived": false,
                    "hidden": false,
                    "pinned": false,
                    "locked_comments": false,
                    "created_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "created_at": "2024-03-03 18:12:57",
                    "updated_by": {
                        "id": 1,
                        "guid": "a85620fa-5368-46ed-bea8-1f06bf5f31ad",
                        "display_name": "Tony GM",
                        "url": "https:\/\/greenmeteor.net\/u\/archblood\/"
                    },
                    "updated_at": "2024-03-03 20:26:10",
                    "scheduled_at": null,
                    "url": "\/u\/archblood\/post\/post\/view?id=3778",
                    "contentcontainer_id": 1,
                    "stream_channel": "default"
                },
                "comments": {
                    "total": "0",
                    "latest": []
                },
                "likes": {
                    "total": 0
                },
                "topics": [],
                "files": []
            }
        }
    ]
}

[ArchBlood]

But again, this should be an easy fix with the REST API module definitions.

[ArchBlood]

@luke- so when trying to implement ContentDefinitions::getContent I get the following error;

Error: Call to a member function getContentUrl() on null in /home/LOCAL/protected/modules/custom_pages/models/CustomContentContainer.php:409
Stack trace:
#0 /home/LOCAL/protected/humhub/modules/content/models/Content.php(746): humhub\modules\custom_pages\models\CustomContentContainer->getUrl()
#1 /home/LOCAL/protected/modules/rest/definitions/ContentDefinitions.php(55): humhub\modules\content\models\Content->getUrl()
#2 /home/LOCAL/protected/modules/rest/definitions/ContentDefinitions.php(28): humhub\modules\rest\definitions\ContentDefinitions::getContentMetadata()
#3 /home/LOCAL/protected/modules/legal/controllers/ExportController.php(119): humhub\modules\rest\definitions\ContentDefinitions::getContent()
#4 [internal function]: humhub\modules\legal\controllers\ExportController->humhub\modules\legal\controllers\{closure}()
#5 /home/LOCAL/protected/modules/legal/controllers/ExportController.php(118): array_map()
#6 /home/LOCAL/protected/modules/legal/controllers/ExportController.php(45): humhub\modules\legal\controllers\ExportController->getContentData()
#7 [internal function]: humhub\modules\legal\controllers\ExportController->actionDownload()
#8 /home/LOCAL/protected/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array()
#9 /home/LOCAL/protected/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams()
#10 /home/LOCAL/protected/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction()
#11 /home/LOCAL/protected/vendor/yiisoft/yii2/web/Application.php(103): yii\base\Module->runAction()
#12 /home/LOCAL/protected/vendor/yiisoft/yii2/base/Application.php(384): yii\web\Application->handleRequest()
#13 /home/LOCAL/index.php(25): yii\base\Application->run()
#14 {main}

https://github.com/humhub/custom-pages/blob/master/models/CustomContentContainer.php#L404-L410

[ArchBlood]

Please review https://github.com/humhub/rest/pull/157 as this seems to fix the issue when it comes to the ContentDefinitions class.

[ArchBlood]

Also, looking into both activities & notifications, they are not explicitly required by GDPR but are optional for organizations that wish to also provide such data as a courtesy for user-friendliness. So I don't think we really need this unless requested in a later update? 🤔

Overall, I'd need more feedback from the community on whether it is needed or not, but as it stands I don't see any need in adding either.

[ArchBlood]

On closer look at the exported userdata.json some activities are fetched regardless of implementing ActivityDefinitions so that would make our jobs easier. But I'd like to note that the more functions for data collected to export the longer the load time can be, the larger the community the bigger the file will be and the load time will reflect this, if there are any documentation or examples that can be given for ActiveJobs I'd gladly work on implementing this by late Friday.

[ArchBlood]

@luke- the following APIs have been implemented with each providing the information that the APIs are able to provide;

• User API
• Post API
• Like API
• File API
• Comment API

For the File API this does not provide a direct download possibility but will provide the link to the file based on the platform, other modules are also taken into account when it comes to these APIs as well as long as the user has data generated for those modules, but for a more precise data table an API definition must be created for those modules, in that case a base definition should be created for the Rest API module for other modules to extend.

[ArchBlood]

Idea, maybe when it comes to downloading their files we make it so that this is an optional second download button and we compress the files into a ZIP/RAR file after unhashing them?

[luke-]

Thanks again for this PR, it's really good. I would like to implement the following points before release. Yura may also be able to contribute here. Can you please create this PR against an own branch e.g. enh/export-data, than I can merge it and we can work further on it.

• [ ] Create Download as Background Job

  • [ ] Create ZIP file as ActiveJob and keep for X days
  • [ ] If exist, show Download on "Export Data" Page
  • [ ] Send Notification once the Export Package is Ready

• [ ] Create an Event allow other modules to Inject own Data. E.g. Polls

  • [ ] Create PRs for official modules
  • [ ] Inform "Marketplace Publisher" Space of new Event

• [ ] Create Tests

• [ ] We should remove post|polls|etc.content.comments because it contains other users data.

[ArchBlood]

Thanks again for this PR, it's really good. I would like to implement the following points before release. Yura may also be able to contribute here. Can you please create this PR against an own branch e.g. enh/export-data, than I can merge it and we can work further on it.

• [ ] Create Download as Background Job

  • [ ] Create ZIP file as ActiveJob and keep for X days
  • [ ] If exist, show Download on "Export Data" Page
  • [ ] Send Notification once the Export Package is Ready

• [ ] Create an Event allow other modules to Inject own Data. E.g. Polls

  • [ ] Create PRs for official modules
  • [ ] Inform "Marketplace Publisher" Space of new Event

• [ ] Create Tests

• [ ] We should remove post|polls|etc.content.comments because it contains other users data.

Once I'm back at the office I'll get on it as soon as possible!

[ArchBlood]

Would it be alright to close this P/R then continue with the other?

[luke-]

@ArchBlood I've added a Branch her: https://github.com/humhub-contrib/legal/tree/enh/data-export

[ArchBlood]

@ArchBlood I've added a Branch her: https://github.com/humhub-contrib/legal/tree/enh/data-export

Would it be possible for a new issue opened for discussions on enchantments to be made for the data-export branch or update #20? I'd also recommend the following points be added as well;

• [ ] Possibility of removing parts of the Post API (i.e. latest comment is not useful in this case)
• [ ] Possibility of extracting files through the File API while unhashing before creating the ZIP achieve
• [ ] Create function createAchieve() which handles the ZIP headers and achieve process
• [ ] Create separate buttons for JSON & ZIP
• [ ] Optional, add userdata.json in ZIP as well?