Notification webhook framework

DeftNerd commented 8 years ago

It would be nice if there was a global settings page and the framework to enable activity log notifications.

Notification when a page is created/modified/deleted Notification when a user is created/modified/deleted Notification when a file is created/modified/deleted

Notifications could be sent to several possible endpoints, but Slack and Rocket.Chat would be likely good first choices. Eventual additions could be email or messenger platform notifications to the admins

ghost commented 7 years ago

I support this suggestion. The ability to add a webhook for slack would remove quite a few other "nice to have" features from our wish list.

dennisoderwald commented 7 years ago

+1

Shackelford-Arden commented 7 years ago

Always a fan of webhooks! +1

therealscottcarlow commented 7 years ago

Definite +1

jacksonp2008 commented 7 years ago

+1, anything in the works for this? IF not, can you point me to where I might make the changes myself?

ssddanbrown commented 7 years ago

Nothing currently in the works for this at the moment. Notifications is a wide topic so I'd say as a stage 1 to complete this issue the focus is put on the basics of the original issue description:

Implementation of a global setting of slack-compatible web-hooks that are called upon page/chapter/book/user creation/modification/deletion.

The Web-hook URL and the subscribed events should be configurable and multiple web hooks should be able to be configured. For now it may be best to keep the contents of these hooks simple, With just the common text property within JSON as supported by slack, matter-most & teams.

There's currently an activity service which all activity creation runs through so that may be the best place to manage these events.

After the implementation of this we can then move focus to user-level notifications and expanding admin level notifications.

I am a bit worried about the performance cost of expanding out a notification system like this on a PHP based application. Once it gets to the point that saving a page fires of a couple of web-hooks and a bunch of emails things will start to get slow. I know a queue could be implemented but that will complicate application requirements, installation and maintenance.

PriyaPallavi commented 7 years ago

Hi @ssddanbrown , I am really interested in the idea of having notification webhook framework. I have divided it into three steps. They are:

Bookmarks/Subscription
Email Notification
Webhook Notification

Currently I am focusing on the first two modules.

Initial implementation can have the following:

At first we need to have a subscription/bookmark button/icon for all the books, chapters and pages.
If the user wants to subscribe to a particular book/chapter/page, he has to click on the bookmark icon available there.
Once the user clicks on the bookmark icon, the details of the user along with the details of the bookmarked book/chapter/page will be stored in the database and a subscription message will be sent to all the subscribers of that page along with the new user using message queue.
We will be providing this option of entering the credentials of message queue tool in the .env file while installation of the application only.
If the credentials for the message queue tool installed in the system is not provided, the user can only see the notifications.
In the global settings page we’ll have a message queue flag which will be turned on when you will provide the credentials of the installed message queue tool in the .env file.
If there is no message queue tool (for example- RabbitMQ, ActiveMQ) installed on the user’s system, then there is no point of subscription.
We can even build a notification system like in Facebook which will continuously update the user about his/her subscribed pages. Examples of such notifications are:
- User subscribed to Book/Chapter/Page Name.
- User created Chapter/Page Name.
- User Modified Book/Chapter/Page Name.

Please do give your feedback.

s0n- commented 7 years ago

+1 for this as well - Slack is probably a great use case for this

galaxyfeeder commented 6 years ago

+1 very interesting for slack

jacksonp2008 commented 6 years ago

Still interested in some way to get notifications from bookstack when someone creates a page or comments. I see that slack has an RSS plugin, so if you output an RSS feed that might be a quick win and I could integrate into slack.

"The RSS integration allows you to subscribe to an RSS or Atom feed URL and receive updates in Slack. Feeds will be fetched periodically, and new items will be posted to the specified channel."

thanks all

tdmalone commented 6 years ago

A temporary (far from ideal, but possible) way of doing this could be setting up a little service that connects directly to the DB, stores the ID of the last entry in the relevant table, and then checks periodically for new entries. It could then send notification of the new entries somewhere.

If getting notifications into Bookstack is difficult, would this be worth writing? I might be able to do it if it would help people.

jonakoudijs commented 6 years ago

A small service that can run periodically sounds like a nice idea. This does not complicate the setup and configuring a cronjob is a default feature on almost every shared hosting company so you will retain compatibility.

Let me know if you are going down that road and if I can help out.

Liandriz commented 6 years ago

+1 Rss feed.

tommyz4 commented 5 years ago

Has there been any progress on email notifications? Is this in the RoadMap?

ssddanbrown commented 5 years ago

@tommyz4 Kind of on the road map. Email notifications appears to be open under #935.

There's a little bit of conversation about this issue. Is one of the highest rated issues so won't go ignored. Email notifications would be an extension/continuation of this issue once in-place as per my comment here: https://github.com/BookStackApp/BookStack/issues/147#issuecomment-291268041

mark-james commented 5 years ago

This feature plus the api feature #823 together would allow for custom workflows and automation. This would mean bookstack could stay simple yet developers could develop very specific use cases outside of the codebase. I think this could be very powerful.

nekromoff commented 5 years ago

Basic RSS (with secret token for private access) would work well for any read-only "notifications" such as displaying latest edited items somewhere else etc.

ezzra commented 5 years ago

Because we have a need for RSS in our team, I hacked a dirty feed solution in https://github.com/ezzra/BookStack/tree/dirty_feed or you can comment here in this fake pull request: https://github.com/BookStackApp/BookStack/pull/1165

It is actually working, if you want to reuse it (@nekromoff and others), commit it to your code. Then run composer install and add APP_FEED_TOKEN=xxx to your .env. Of course, xxx should be any random string, that makes the feed accessible without logging in!

Be aware! The code is really dirty and its just working without beeing pretty, so please keep in mind:

there are separated feeds for a book (all sortings and page changes within a book), a chapter (all page changes within a chapter), a page (all changes to this page)
shelves are not included yet because we dont use them (harhar), but you might be able to adapt it to the code if you like
you can just use your browsers feed button, the feed button is everywhere, but it only works on a book, chapter or page site
you must set the feed token, otherwise the feed will be accessible from not logged in users easily.
any user who has a login and gets knowledge about the feed token has access to the whole feed! So even when she has no access to a specific book or page, she can access the feed because it is only guarded by the system wide feed token!
this is also the fact if you delete an account, the user will still have access to the feed (unless you change the token)!
the feed does not tell anything about the content of a page, just "metadata"

so you can see, its a fast dirty hack, but it works. It could be made better of course. But its a wrong way, because there much more needs to be a system internal solution for subscriptions where notifications can be created in different ways (email, OS notification, RSS, ...) only for specific user accounts. That is why I wont put more work into that dirty way here.

nekromoff commented 5 years ago

Great, much appreciated. I am going to look into it and report back.

On Sat, Dec 8, 2018, 13:51 ezzra <notifications@github.com wrote:

Because we have a need for RSS in our team, I hacked a dirty feed solution in https://github.com/ezzra/BookStack/tree/dirty_feed

It is actually working, if you want to reuse it (@nekromoff https://github.com/nekromoff and others), commit it to your code. Then run composer install and add APP_FEED_TOKEN=xxx to your .env. Of course, xxx should be any random string, that makes the feed accessible without logging in!

Be aware! The code is really dirty and its just working without beeing pretty, so please keep in mind:

there are separated feeds for a book (all sortings and page changes within a book), a chapter (all page changes within a chapter), a page (all changes to this page)

shelves are not included yet because we dont use them (harhar), but you might be able to adapt it to the code if you like

you can just use your browsers feed button, the feed button is everywhere, but it only works on a book, chapter or page site

you must set the feed token, otherwise the feed will be accessible from not logged in users easily.

any user who has a login and gets knowledge about the feed token has access to the whole feed! So even when she has no access to a specific book or page, she can access the feed because it is only guarded by the system wide feed token!

this is also the fact if you delete an account, the user will still have access to the feed (unless you change the token)!

so you can see, its a fast dirty hack, but it works. It could be made better of course. But its a wrong way, because there much more needs to be a system internal solution for subscriptions where notifications can be created in different ways (email, OS notification, RSS, ...) only for specific user accounts. That is why I wont put more work into that dirty way here.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BookStackApp/BookStack/issues/147#issuecomment-445456955, or mute the thread https://github.com/notifications/unsubscribe-auth/AIJ3zSRgLbHwTdPOPrQMfhMq6ANM-pUoks5u27XigaJpZM4JK9si .

irman commented 5 years ago

Any update on this? Would love to contribute and help out, or even kickstart it.

ssddanbrown commented 5 years ago

@irman Upcoming on the roadmap is an API for BookStack. That will likely provide a lot of the groundwork/cleanup for a feature like this. This feature may be part of that API process, Depending on how things go.

ezzra commented 5 years ago

Upcoming on the roadmap

@ssddanbrown looks like this roadmap link is somehow broken :)

ssddanbrown commented 5 years ago

@ezzra Whoops, thanks for letting me know, now updated.

akurzawa commented 5 years ago

+1 :) When I create page I would like to inform somehow the peoples that new page was created for them.

BoxedBrain commented 5 years ago

+1

Laravel supports queues which we could run with "supervisor"! https://laravel.com/docs/5.8/queues

deividas-balysevas commented 4 years ago

+1 Email notifications, please!

GameBurrow commented 4 years ago

Yes please. Need webhook or at least RSS feed so we can make our own.

Albirew commented 4 years ago

There already are columns called "recently updated pages" and "recent activity". Isn't it possible to use same source to make an RSS feed for now?

tomtrm commented 4 years ago

+1

nnsense commented 4 years ago

I've just spent an hour configuring bookstack email backend, just to find out that this feature, which I've taken for granted, isn't available. So the whole point of configuring the mail backed is to send the registration email? I thought it was quite common for a team working on the same documentation to be notified on new/updated pages. I'm not speaking about something complex, such as a third party integration with slack for example, but the very basic "email update" should definitely be there.

ssddanbrown commented 4 years ago

So the whole point of configuring the mail backed is to send the registration email?

No, It can also be used for email confirmation & password resets :love_letter:

romainguerrero commented 3 years ago

+1 for Slack notifications !

thimisc commented 3 years ago

+1 Slack notifications would be a great feature

nothavelizard commented 3 years ago

+1 for email notifications

dekardkrr commented 3 years ago

+1

hwcltjn commented 3 years ago

+1

GD1m commented 3 years ago

+1 for email notifications or any, please

rubn-g commented 3 years ago

+1 for updates notifications, this issue is open since 5 years ago, any news about it?

Redsandro commented 3 years ago

I've started a Bookstack for my old school club of 20 years ago. :heart: And a decent lot of them ask me to "enable email notifications." I guess they just assume I can do that because they also received the registration email. :monocle_face:

They want to know if anyone contributed to their memory/story, without having to come back and check the stories they wrote periodically.

@ssddanbrown could you reconsider the priority for this? It really helps motivate the club if people receive an email and want to check and amend the edits. :muscle: A simplified minimalist implementation - even a beta one without subscription management that can only be enabled using an environment variable and simply emails all authors and/or all commenters, whichever is possible - would be beneficial and greatly appreciated.

Username edited [link to chapter]

nekromoff commented 3 years ago

Just to set it straight. You now have access to Bookstack notifications via a notification URL, such as: Example implementation for Laravel: config.php

'url'          => '..../public/api/books?sort=-created_at',
    'token_id'     => 'XXX',
    'token_secret' => 'XXX',

code to import notifications:

    public function importWiki(Request $request)
    {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_HTTPHEADER, array('Authorization: Token ' . config('wiki.token_id') . ':' . config('wiki.token_secret')));
        curl_setopt($ch, CURLOPT_URL, config('wiki.url'));
        $content = curl_exec($ch);
        $content = json_decode($content);
        if (isset($content->data[0])) {
            $item = $content->data[0];
            DashboardWiki::truncate();
            $wiki = new DashboardWiki;
            $wiki->name = $item->name;
            $wiki->description = $item->description;
            $wiki->save();
        }
    }

You can easily plug your email system into this and send it out. However, right now there is no API endpoint for users. So you would have to either send everything to everyone or have a separate config for matching IDs to users (as created_by, updated_by user IDs are provided via API).

ssddanbrown commented 3 years ago

As an alternative solution, If anyone is handy with Laravel, you could attempt to use our backend-theme-system to listen to the APP_BOOT event we provide to then set event listeners on certain models.

Redsandro commented 3 years ago

Just to set it straight. You now have access to Bookstack notifications via a notification URL

@nekromoff Is this documented somewhere so I can better understand how to use this with a docker image? I did search and found a puzzling lack of those keywords in the code; I suspect the actual function is provided through a dependency.

nekromoff commented 3 years ago

The public documentation is unavailable. Only "hacking the Bookstack" mentions it.

https://www.bookstackapp.com/docs/admin/hacking-bookstack/

However, once you have Bookstack installed, you can access it by using /api/docs URL, where all the details are provided.

copy&paste of the first few paragraphs:

Getting Started Authentication

To access the API a user has to have the "Access System API" permission enabled on one of their assigned roles. Permissions to content accessed via the API is limited by the roles & permissions assigned to the user that's used to access the API.

Authentication to use the API is primarily done using API Tokens. Once the "Access System API" permission has been assigned to a user, a "API Tokens" section should be visible when editing their user profile. Choose "Create Token" and enter an appropriate name and expiry date, relevant for your API usage then press "Save". A "Token ID" and "Token Secret" will be immediately displayed. These values should be used as a header in API HTTP requests in the following format:

1

Authorization: Token :

Here's an example of an authorized cURL request to list books in the system:

1

curl --request GET \

2

--url https://example.com/api/books \

3

--header 'Authorization: Token C6mdvEQTGnebsmVn3sFNeeuelGEBjyQp:NOvD3VlzuSVuBPNaf1xWHmy7nIRlaj22'

If already logged into the system within the browser, via a user account with permission to access the API, the system will also accept an existing session meaning you can browse API endpoints directly in the browser or use the browser devtools to play with the API.

Request Format

The API is primarily design to be interfaced using JSON so the majority of API endpoints, that accept data, will read JSON request data although application/x-www-form-urlencoded request data is also accepted. Endpoints that receive file data will need data sent in a multipart/form-data format although this will be highlighted in the documentation for such endpoints.

For endpoints in this documentation that accept data, a "Body Parameters" table will be available showing the parameters that will accepted in the request. Any rules for the values of such parameters, such as the data-type or if they're required, will be shown alongside the parameter name.

On Sat, May 22, 2021 at 4:17 PM Sander Steenhuis @.***> wrote:

Just to set it straight. You now have access to Bookstack notifications via a notification URL

@nekromoff https://github.com/nekromoff Is this documented somewhere so I can better understand how to use this with a docker image? I did search https://github.com/search?q=org%3ABookStackApp+token_secret&type=code and found a puzzling lack of those keywords in the code; I suspect the actual function is provided through a dependency.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BookStackApp/BookStack/issues/147#issuecomment-846414665, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBHPTKY3YEDMG376FIJCZDTO64JBANCNFSM4CJL3MRA .

Redsandro commented 3 years ago

This is helpful. I hadn't found out about this yet. Thank you. :+1:

Redsandro commented 3 years ago

Just so I'm clear: The API allows you to access book/chapter/page metadata such as last update timestamp and user_id, but you cannot get the "recent activity" log (with both edits and comments) through API, correct? There is no way to fetch comment activity, and an edit notification script would need to poll all pages, correct?

Just to double check before I do something the hard way.

ssddanbrown commented 3 years ago

@Redsandro That's correct, although you should be able to sort the results by the updated column and check those updated since last poll.

As an alternative, using the logical theme system posted above, you could quickly get something running like so (functions.php example):

<?php

use BookStack\Entities\Models\Page;
use BookStack\Theming\ThemeEvents;
use BookStack\Facades\Theme;
use Illuminate\Mail\Message;
use Illuminate\Support\Facades\Mail;

Theme::listen(ThemeEvents::APP_BOOT, function () {

    // Listen to page saving events
    Page::saving(function (Page $page) {
        // Ensure the page is being altered by someone else
        $changedByNonCreator = ($page->createdBy && $page->updatedBy && $page->createdBy->id !== $page->updatedBy->id);

        // If the content has been changed by the non-creator user, send a message.
        if ($changedByNonCreator && $page->isDirty('html')) {
            $messageText = "Your page \"{$page->name}\" has been updated by {$page->updatedBy->name}.\n";
            $messageText .= "View the page here: {$page->getUrl()}";

            Mail::raw($messageText, function (Message $message) use ($page) {
                $creatorEmail = $page->createdBy->email;
                $subject = setting('app-name') . " - Your page \"{$page->name}\" has been updated";

                $message->to($creatorEmail)
                    ->subject($subject);
            });
        }
    });

});

Note: Only the theme functions themselves are considered somewhat stable, Most of the code there is not considered stable and subject to change upon any update.

Redsandro commented 3 years ago

Thank you for the help. I don't feel at ease with PHP, otherwise I'd probably try to implement a proper notification system and do a PR. I will however share my node.js script in case it may be useful.

You can take multiple approaches. My approach stores the last edit dates in a sqlite3 file so that the script can be executed through cron.

Ideally you should touch a dedicated log file and chown it to the same user that runs cron:

touch /var/log/bookstack-notify.log && chmod `whoami`: $_

I run this once per day to make sort of an "daily digest", but you can run it every minute if you want everyone to receive notifications near realtime.

0 0 * * * node /var/www/bookstack-notify/index.js >> /var/log/bookstack-notify.log

It's not smart. You'll need to hardcode the notification recipients. Everyone receives the updates, even if they trigger them themselves.

const sqlite3   = require('sqlite3').verbose()
const { open }  = require('sqlite');
const axios     = require('axios')
const nodemailer= require('nodemailer')

// Change these
const url       = 'https://bookstack.example.com'
const token     = 'xxxxxx:yyyyyy'
const user      = 'EXAMPLE_GMAIL_USER@gmail.com'
const pass      = 'EXAMPLE_APP_PASSWORD'
const from      = 'Bookstack Bot <EXAMPLE_GMAIL_USER@gmail.com>'
const to        = 'EXAMPLE_EMAIL@1, EXAMPLE_EMAIL@2, EXAMPLE_EMAIL@3'
const subject   = 'Bookstack Updates'

// Globals
const headers   = {Authorization: `Token ${token}`}
const params    = {sort: '-updated_at'}
const rows      = []
let links       = []
let html        = '<p>Updates to your Bookstack</p>'

async function main() {
    try {
        const db = await open({filename: './bookstack.db', driver: sqlite3.Database})

        await db.run(`CREATE TABLE IF NOT EXISTS bookstack_pages(
            id INTEGER PRIMARY KEY,
            name TEXT,
            slug TEXT,
            updated_at TEXT
        )`)

        await db.each('SELECT * FROM bookstack_pages', (err, row) => {
            if (err) throw err

            rows.push(row)
        })

        const res = await axios.get(`${url}/api/pages`, {headers, params})
        const { data } = res.data

        for (let idx = 0, len = data.length; idx < len; idx++) {
            const {id, name, slug, updated_at} = data[idx]
            const row = rows.find(row => row.id == id)

            if (!row || new Date(updated_at).getTime() > new Date(row.updated_at).getTime()) {
                console.log(`${new Date().toISOString()} - Updating page ${id}`)
                await db.run('INSERT OR REPLACE INTO bookstack_pages(id,name,slug,updated_at) VALUES(?, ?, ?, ?)', [id, name, slug, updated_at])
                links.push(data[idx])
            }
        }

        await db.close()

        if (links.length) {
            links = links.map(link => `<li><a href="${url}/link/${link.id}">${link.name}</a> (${link.updated_at})</li>`).join('')
            html += `<ul>${links}</ul>`
            html += '<p>This bot is not intelligent. The update could be triggered by anyone including you.</p>'

            let info = await nodemailer.createTransport({
                host: 'smtp.gmail.com',
                secureConnection: false,
                port: 587,
                auth: { user, pass }
            }).sendMail({ from, to, subject, html })

            console.log(`${new Date().toISOString()} - Message sent: ${info.messageId}`)
        }
    }
    catch (err) {
        console.log(`${new Date().toISOString()} - Error: ${err.message}`)
    }
}

main()

ssddanbrown commented 2 years ago

I've started building a webhook implementation within PR #3099. To confirm, this PR (And this issue) are specific to the implementation of outbound HTTP webhooks. Other forms of notification & integration are outside the scope of the PR and this issue.

jacksonp2008 commented 2 years ago

Excellent news!!

On Wed, Dec 8, 2021 at 06:44 Dan Brown @.***> wrote:

I've started building a webhook implementation within PR #3099 https://github.com/BookStackApp/BookStack/pull/3099. To confirm, this PR (And this issue) are specific to the implementation of outbound HTTP webhooks. Other forms of notification & integration are outside the scope of the PR and this issue.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BookStackApp/BookStack/issues/147#issuecomment-988876109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN2WT4T2MZ4EV7W2YNVQ2DUP5VN7ANCNFSM4CJL3MRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Regards,

-Steve

(415) 320-1102 https://www.google.com/voice/#phones

ssddanbrown commented 2 years ago

Initial webhook implementation, as per my previous comment, now merged in within #3099 to be part of the next feature release.

Merry Christmas / happy holidays!

BookStackApp / BookStack

Notification webhook framework #147

If already logged into the system within the browser, via a user account with permission to access the API, the system will also accept an existing session meaning you can browse API endpoints directly in the browser or use the browser devtools to play with the API.