search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.79k stars 2.13k forks source link

TemplateNotFound: notif_mail.html in Matrix Synapse homeserver on Ubuntu 16.04 #2331

Closed MurzNN closed 5 years ago

MurzNN commented 7 years ago

I try to configure email notifications in Matrix Synapse homeserver on Ubuntu 16.04. Matrix server is on version 0.21.1-1. Here is my config part:

public_baseurl: https://chat.mydomain.com/

# Enable sending emails for notification events
email:
   enable_notifs: true
   smtp_host: "localhost"
   smtp_port: 25
        #   smtp_user: "exampleusername"
        #   smtp_pass: "examplepassword"
   notif_from: "Matrix notify <admin+chat@mydomain.com>"
   app_name: Matrix
   template_dir: res/templates
#   notif_template_html: notif_mail.html
   notif_template_html: notif_mail2.html
#   notif_template_html: /etc/matrix-synapse/notif_mail2.html
#   notif_template_text: notif_mail.txt
   notif_template_text: notif_mail2.txt
#   notif_template_text: /etc/matrix-synapse/notif_mail2.txt
   notif_for_new_users: True
   riot_base_url: "https://chat.mydomain.com/"

And after trying enable notifications in some user, I see in logs:

июл 04 17:33:10 tad python[7902]: 2017-07-04 17:33:10,958 - synapse.http.server - 139 - ERROR - POST-34- Failed handle request synapse.http.server._async_render on <synapse.rest.ClientRestResource object at 0x7f46a2210b50>: <SynapseRequest at 0x7f46a149c1b8 method=POST uri=/_matrix/client/r0/pushers/set?access_token=<redacted> clientproto=HTTP/1.0 site=8008>
июл 04 17:33:10 tad python[7902]: Traceback (most recent call last):
июл 04 17:33:10 tad python[7902]:   File "/usr/lib/python2.7/dist-packages/synapse/http/server.py", line 116, in wrapped_request_handler
июл 04 17:33:10 tad python[7902]:     yield request_handler(self, request, request_metrics)
июл 04 17:33:10 tad python[7902]: TemplateNotFound: notif_mail2.html

How I can debug in which path Synapse really try to find this file?

MurzNN commented 7 years ago

I also try to fill full path from root to file: /etc/matrix-synapse/notif_mail2.html and give permissions:

# ls -lah /etc/matrix-synapse/notif_mail2.html
-rw-r--r-- 1 root root 2,8K июл  4 17:04 /etc/matrix-synapse/notif_mail2.html

But the error is same:

TemplateNotFound: /etc/matrix-synapse/notif_mail2.html

Moving files to /etc/matrix-synapse/res/templates/notif_mail2.html - not solve problem.

MurzNN commented 7 years ago

If I try to fill full path into template_dir: 

template_dir: /etc/matrix-synapse/res/templates

I got another error:

июл 04 18:23:48 tad python[3833]: 2017-07-04 18:23:48,514 - synapse.http.server - 139 - ERROR - POST-35- Failed handle request synapse.http.server._async_render on <synapse.rest.ClientRestResource object at 0x7fe42ded2890>: <SynapseRequest at 0x7fe42d0d3248 method=POST uri=/_matrix/client/r0/pushers/set?access_token=<redacted> clientproto=HTTP/1.0 site=8008>
июл 04 18:23:48 tad python[3833]: Traceback (most recent call last):
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/http/server.py", line 116, in wrapped_request_handler
июл 04 18:23:48 tad python[3833]:     yield request_handler(self, request, request_metrics)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
июл 04 18:23:48 tad python[3833]:     result = result.throwExceptionIntoGenerator(g)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
июл 04 18:23:48 tad python[3833]:     return g.throw(self.type, self.value, self.tb)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/http/server.py", line 255, in _async_render
июл 04 18:23:48 tad python[3833]:     callback_return = yield callback(request, **kwargs)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
июл 04 18:23:48 tad python[3833]:     result = result.throwExceptionIntoGenerator(g)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
июл 04 18:23:48 tad python[3833]:     return g.throw(self.type, self.value, self.tb)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/rest/client/v1/pusher.py", line 129, in on_POST
июл 04 18:23:48 tad python[3833]:     profile_tag=content.get('profile_tag', ""),
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
июл 04 18:23:48 tad python[3833]:     result = result.throwExceptionIntoGenerator(g)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
июл 04 18:23:48 tad python[3833]:     return g.throw(self.type, self.value, self.tb)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/push/pusherpool.py", line 86, in add_pusher
июл 04 18:23:48 tad python[3833]:     profile_tag=profile_tag,
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
июл 04 18:23:48 tad python[3833]:     result = result.throwExceptionIntoGenerator(g)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
июл 04 18:23:48 tad python[3833]:     return g.throw(self.type, self.value, self.tb)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/pusher.py", line 234, in add_pusher
июл 04 18:23:48 tad python[3833]:     yield self.runInteraction("add_pusher", f)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
июл 04 18:23:48 tad python[3833]:     result = result.throwExceptionIntoGenerator(g)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
июл 04 18:23:48 tad python[3833]:     return g.throw(self.type, self.value, self.tb)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 319, in runInteraction
июл 04 18:23:48 tad python[3833]:     inner_func, *args, **kwargs
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 246, in inContext
июл 04 18:23:48 tad python[3833]:     result = inContext.theWork()
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
июл 04 18:23:48 tad python[3833]:     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
июл 04 18:23:48 tad python[3833]:     return self.currentContext().callWithContext(ctx, func, *args, **kw)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
июл 04 18:23:48 tad python[3833]:     return func(*args,**kw)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/twisted/enterprise/adbapi.py", line 285, in _runWithConnection
июл 04 18:23:48 tad python[3833]:     result = func(conn, *args, **kw)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 313, in inner_func
июл 04 18:23:48 tad python[3833]:     func, *args, **kwargs
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 242, in _new_transaction
июл 04 18:23:48 tad python[3833]:     r = func(txn, *args, **kwargs)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/pusher.py", line 226, in f
июл 04 18:23:48 tad python[3833]:     "id": stream_id,
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 495, in _simple_upsert_txn
июл 04 18:23:48 tad python[3833]:     txn.execute(sql, sqlargs)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 80, in execute
июл 04 18:23:48 tad python[3833]:     self._do_execute(self.txn.execute, sql, *args)
июл 04 18:23:48 tad python[3833]:   File "/usr/lib/python2.7/dist-packages/synapse/storage/_base.py", line 110, in _do_execute
июл 04 18:23:48 tad python[3833]:     sql, *args
июл 04 18:23:48 tad python[3833]: ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

Template files now contain only one string "test" with latin symbols, no other symbols present.

verymilan commented 7 years ago

ran into this as well after upgrading synapse to 0.22.1.

ghost commented 7 years ago

Same here, used to work fine with 0.21.1.

Here's the log :

systemd[1]: Started Synapse Matrix homeserver.
synapse[24194]: twisted: [] Traceback (most recent call last):
synapse[24194]: twisted: []   File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
synapse[24194]: twisted: []     "__main__", fname, loader, pkg_name)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
synapse[24194]: twisted: []     exec code in run_globals
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/app/homeserver.py", line 491, in <module>
synapse[24194]: twisted: []     main()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/app/homeserver.py", line 486, in main
synapse[24194]: twisted: []     hs = setup(sys.argv[1:])
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/app/homeserver.py", line 345, in setup
synapse[24194]: twisted: []     hs.start_listening()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/app/homeserver.py", line 208, in start_listening
synapse[24194]: twisted: []     self._listener_http(config, listener)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/app/homeserver.py", line 123, in _listener_http
synapse[24194]: twisted: []     client_resource = ClientRestResource(self)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/rest/__init__.py", line 65, in __init__
synapse[24194]: twisted: []     self.register_servlets(self, hs)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/rest/__init__.py", line 72, in register_servlets
synapse[24194]: twisted: []     room.register_servlets(hs, client_resource)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/rest/client/v1/room.py", line 794, in register_servlets
synapse[24194]: twisted: []     RoomStateEventRestServlet(hs).register(http_server)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/rest/client/v1/room.py", line 84, in __init__
synapse[24194]: twisted: []     self.handlers = hs.get_handlers()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/server.py", line 343, in _get
synapse[24194]: twisted: []     dep = builder()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/server.py", line 179, in build_handlers
synapse[24194]: twisted: []     return Handlers(self)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/handlers/__init__.py", line 51, in __init__
synapse[24194]: twisted: []     self.message_handler = MessageHandler(hs)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/handlers/message.py", line 53, in __init__
synapse[24194]: twisted: []     self.pusher_pool = hs.get_pusherpool()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/server.py", line 343, in _get
synapse[24194]: twisted: []     dep = builder()
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/server.py", line 270, in build_pusherpool
synapse[24194]: twisted: []     return PusherPool(self)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/push/pusherpool.py", line 31, in __init__
synapse[24194]: twisted: []     self.pusher_factory = PusherFactory(_hs)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/push/pusher.py", line 46, in __init__
synapse[24194]: twisted: []     templates = load_jinja2_templates(hs.config)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/synapse/push/mailer.py", line 534, in load_jinja2_templates
synapse[24194]: twisted: []     config.email_notif_template_html
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
synapse[24194]: twisted: []     return self._load_template(name, self.make_globals(globals))
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
synapse[24194]: twisted: []     template = self.loader.load(self, name, globals)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
synapse[24194]: twisted: []     source, filename, uptodate = self.get_source(environment, name)
synapse[24194]: twisted: []   File "/usr/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source
synapse[24194]: twisted: []     raise TemplateNotFound(template)
synapse[24194]: twisted: [] jinja2.exceptions.TemplateNotFound: notif_mail.html
systemd[1]: synapse.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: synapse.service: Unit entered failed state.
systemd[1]: synapse.service: Failed with result 'exit-code'.
esackbauer commented 7 years ago

Same here after upgrading Ubuntu 14.04 LTS and rebooting today

Homeserver.yaml says: template_dir: res/templates notif_template_html: notif_mail.html notif_template_text: notif_mail.txt

However there is no such directory (anymore?) and these files to not exist on the whole filesystem.

Darlelet commented 7 years ago

Same issue here, v0.22.1

Darlelet commented 7 years ago

Actually you just need to manually specify the path to res/templates from the config file and you should be good to go.

If you're using the unofficial docker image then clone the project's repository and specify its path (email.template_dir).

Quite an easy fix.

Though I don't know if this could be related to this issue (from synapse's log): 2017-08-01 23:37:00,171 - synapse.util.versionstring - 82 - INFO - - Failed to check for git repository: [Errno 2] No such file or directory

MurzNN commented 7 years ago

@Darlelet which system did you use and how you specify path to res/templates files - full or relative path? In 2 comment I already try to specify full path, but got TemplateNotFound error. Can you provide your example of configuration and file location?

Darlelet commented 7 years ago

@MurzNN Docker image using latest Debian version (see docker-matrix), and I did specify the full path to it.

izzy commented 6 years ago

Can confirm this, the res/templates folder simply doesn't exist after I upgraded via pip, so it's probably missing in the pip release tarball as well?

svenakela commented 6 years ago

I ran into this as well as late as today (see date of comment). Downloaded the res/template dir to the server but there were also complaints about missing Py packages. The output was better when I started the server manually /usr/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/

and then I found that jinja2 and bleach had to be installed. Also, the email:public_baseurl is not enough, the root version public_baseurl must be set as well.

mt7479 commented 6 years ago

The template folder is also missing from the the debian package. Regarding the dependencies, my system was missing python-bleach, python-jinja2 was installed though.

matrix-synapse:
Installed: 0.26.0-1
Candidate: 0.26.0-1
Version table:
*** 0.26.0-1 500
500 https://matrix.org/packages/debian stretch/main amd64 Packages
100 /var/lib/dpkg/status

@svenakela

Could you please share your obfuscated config ? Never read about the option email:public_baseurl only riot_public_baseurl. Unfortunately the debian package ships an outdated version that has none of the options above. See #1433

leprasmurf commented 6 years ago

Just upgraded to synapse version 0.31.1-1 via apt and decided to try the email integration again. Here's what I had to do to get the server back up and running:

1) Install a few more deps

$ apt install python-jinja2 python-bleach

2) Clone the templates from the repo

$ mkdir /etc/matrix-synapse/templates
$ cd /etc/matrix-synapse/templates
$ for template in "mail-Vector.css" "mail.css" "notif.txt" "notif_mail.html" "notif_mail.txt" "room.html" "room.txt";
do
  wget https://raw.githubusercontent.com/matrix-org/synapse/master/res/templates/${template}
done

3) Set the appropriate values for public_baseurl and template_dir in homeserver.yaml

It got the server back up and running, but I have not yet received any email (not really sure what the trigger is).

ssbarnea commented 6 years ago

I tried to use the official container and once I tried to configure email notification I was hit by this bug, mainly enabling them required me to specify a template dir, which didn't exist. So I was forced to clone 6 files and to put them inside docker data directory.

mail.css
mail-Vector.css
notif_mail.html
notif_mail.txt
notif.txt
room.html
room.txt

This is far from being a good user experience. I would expect to to be forced to clone files from source in order to enable email notifications. The default path used in the example of "res/templates" does mot work with the container.

@leprasmurf Thanks for posting the workaround. Not sure if is enough but it sorted at least the visible errors.

Now I do hope that the pip requirements are already installed in the container as I do want to avoid having to build my own container on top in order to make it work.

jkuester commented 5 years ago

@ssbarnea (or anyone else for that matter) did you ever get email notifications to send? I have followed basically the same steps for a server deployed via Docker (https://github.com/AVENTER-UG/docker-matrix).

Now the server will start and run fine but I am not seeing any emails and I am not seeing any errors in the logs. But, I do not get any email notifications. (Tried sending messages to a user with email notifications enabled that had been offline for > 2m and then waited >2m to see if the emails were sent and nothing came...)

r4dh4l commented 5 years ago

I have a problem as well but at first I would like to know: Is it mandatory to set an Identity server to provide a mail address? Somehow I can not set a mail address in settings when I've removed the preset identity server logging in.

Edit: According to https://matrix.to/#/!HsxjoYRFsDtWBgDQPh:matrix.org/$15423781942979WfmVZ:kamax.io it is not possible to set mail addresses without providing an identity server:

So if I don't want to use Identity servers (deleting the address logging in to Riot desktop/mobile client) I can not set a mail address?

Your users cannot set one, no. Only the server admin could and only by editing the DB directly. If you don't want to use a public server, you can always self-host with https://github.com/kamax-matrix/mxisd (official room is #mxisd:kamax.io )

richvdh commented 5 years ago

The original cause of this bug seems to have been that the reporter didn't point the configuration to the right directory. If you have another bug, please open a new issue. If you'd like support setting up your homeserver, please ask in #synapse:matrix.org, not in some (unrelated) github issue.

richvdh commented 5 years ago

oh, and if your problem is that synapse can't find the email templates, please do let us know how you installed synapse. We can't guess!

r4dh4l commented 5 years ago

Sorry @richvdh that I misused this thread (actually I hate idling around in chats waiting for help because of known reasons).

But I will give it a try again. Let me leave a copy of my request here because I found this issue looking for a solution (so maybe others will have the same problem as well):

Edit: Opened a new issue in https://github.com/matrix-org/synapse/issues/4195 because this one is closed.