Open tresni opened 4 years ago
@throttlemeister How else should all the "Remember this browser as secure" functions for websites with 2FA should work without storing some information on the clients computer for a period of time?
@lippertmarkus they shouldn't. I understand why it's there (convenience, people would stop using MFA if they had to log in each time) but leaving a session authorized is a security risk.
Allowing the session token to be transported from a browser to another computer and used as an authenticated user, that's just a plain and simple security bug that needs fixing. If we can use it for a deploy-hook, a hacker could use it to bypass 2FA.
@throttlemeister / @lippertmarkus / @BartSiwek - Synology is using simple TOTP, then it should be possible to use oathtool
similar to dns_inwx or dns_cyon. Now if you are running on your Synology device, getting oathtool is an exercise left to the reader as it's not pre-installed. You would also have to be comfortable storing the TOTP secret in the configuration file for the domain. This would be the "correct" way to support accounts with 2FA enabled. I'll try to look into this, but first, I'm going to try to get DSM 7 working.
I'm running the latest public DSM 7 beta, and fixed the deploy hook for me. Can people still on DSM 6 test if my changes still work for them as well?
Thijn/acme.sh@cff4956
@Thijn nice work! Also good find on the Auth method for WebAPI, I'd been stuck trying to figure out the new encryption scheme they are using (started in DSM 6 but looks required on DSM 7 for WebUI.)
Checking it out on my DSM 6 now.
Small changes for DSM 6:
enable_syno_token
as explained below) must either be in the GET params or the POST params, you can't mix GET and POST paramsenable_syno_token=yes
must be in both the GET and POST params.
If enable_syno_token=yes
is only in the POST fields, then DSM6 returns a synotoken of --------
. If enable_syno_token=yes
is only in the GET params, then it returns no synotoken at all. It must be in both to work.Verified with DSM 6.2.3-25426 Update 2 and DSM 7.0-40850 . I'll get a PR open, great work @Thijn, the path I was going down was not awesome...
diff --git a/deploy/synology_dsm.sh b/deploy/synology_dsm.sh
index bca12d8..edbc1e3 100644
--- a/deploy/synology_dsm.sh
+++ b/deploy/synology_dsm.sh
@@ -90,7 +90,7 @@ synology_dsm_deploy() {
_debug3 H1 "${_H1}"
fi
- response=$(_post "method=login&account=$encoded_username&passwd=$encoded_password&enable_device_token=yes&enable_syno_token=yes" "$_base_url/webapi/entry.cgi?api=SYNO.API.Auth&version=$api_version")
+ response=$(_post "method=login&account=$encoded_username&passwd=$encoded_password&api=SYNO.API.Auth&version=$api_version&enable_syno_token=yes" "$_base_url/webapi/auth.cgi?enable_syno_token=yes")
token=$(echo "$response" | grep "synotoken" | sed -n 's/.*"synotoken" *: *"\([^"]*\).*/\1/p')
_debug3 response "$response"
_debug token "$token"
@Thijn Great work indeed, I can also confirm that this does work for DSM 7 Beta
Opened #3299 a couple weeks back to incorporate @Thijn ’s change and DSM6 support.
Sorry for interrupting this issue, however I am trying to create a similar deploy script for synology srm. https://github.com/acmesh-official/acme.sh/issues/3332 Hoped someone described to this thread could give me a hint what goes wrong. Thanks
I actually recently got a RT2600AC for my parents and was looking at it as well. The API endpoints are not vastly different (though the paths are.) Do you have a fork you can share with your work so far? I haven’t really done more than a cursory glance so I’d need to see where you are and try it against mine locally. I remember when doing some packet capturing that the cert management was really different. While it looks like a single certificate I believe there were multiple represented under the hood. Chrome + Dev Tools is definitely your friend and how I got a lot of this started.
@tresni Thanks for the quick response. https://github.com/divx118/acme.sh/blob/synology_srm/deploy/synology_srm.sh It is more or less basically the same as the synology DSM deploy hook. I for now created a different deploy file for srm, but using the same env variables as you do in dsm deploy hook. BTW I noticed the recent changes for dsm deploy hook to logging in didn't work for the srm. I didn't look into it yet, just for now keep using the old code.
@tresni It works, but there is something with the intermediate certificate, it wants the domain.csr and not the ca.cer at least that is the case when importing manually. I now commented out the intermediate and importing works fine. It doesn´t seem to have an intermediate certificate
SynologyRouter> ls -al /usr/syno/etc/ssl
drwx------ 8 root root 4096 Jan 2 11:42 .
drwxr-xr-x 46 root root 4096 Jan 2 11:40 ..
drwxrwxrwx 2 admin users 4096 Jan 2 11:42 .gen
-rw-r--r-- 1 root root 15 Jan 2 11:42 .mkcert.serial
drwxr-xr-x 2 root system 4096 Jan 2 11:42 cfg.d
drwx------ 2 root root 4096 Jan 1 2020 cgi.key
-rwxr-xr-x 1 root root 8268 Jun 15 2020 mkcert.sh
-rwxr-xr-x 1 root root 626 Jun 15 2020 mkcgikey.sh
drwxr-xr-x 2 root root 4096 Jan 2 11:42 ssl.crt
drwxr-xr-x 2 root root 4096 Jan 1 2020 ssl.csr
drwx------ 2 root root 4096 Jan 2 11:42 ssl.key
Edit: I submitted a support ticket with synology, seems to me it is a bug.
It was the first blank line of the ca.cer that caused the import problem of intermediate certificate on the synology routers SRM. This will be resolved in the future https://github.com/acmesh-official/acme.sh/pull/3211 For now I remove the blank line in the deploy-script for SRM
Hi! Thanks a lot for the easy deploy script! I just wanted to add that while I was setting it up on my Synology NAS I noticed that the script is not able to find the default certificate if it has a description This seems to be the reason why, as the wiki guide tells users to keep $SYNO_Certificate empty to create/replace the default cert. There should be a few ways to fix this (like checking for "is_default":true
instead of an empty description in the response, for example). It would be great in any case if you could add a short note to the guide in the wiki for users to make sure the description text of the default certificate is empty in DSM in case another person has the same problem.
Hey guys,
I'm trying to set up the (auto-) deployment, but struggle heavily with DSM 6.2.3-25426 Update 3 rejecting ECC certs while 6.2.2 seemed to accept them. Since I don't want to downgrad DSM just for the sake of getting the ECC imported & than update again, can somebody tell me how to kinda "brute force" DSM into using the ECC cert anyway?
Using the shell, as I did earlier (prior to discovering ACME.sh's deploy-hook for DSM 😅), always left me with the MailPlus web-UI using the wrong, i. e. Synology's self-signed, cert until I stopped & restarted the application via DSM's control panel.
Any help would be greatly appreciated!
Hi there! I'm using the latest (c311ce146890
) docker acme.sh image to run acme on my synology, and then deploy the certificate to my nas.
I'm using aws_dns
; issuing works, and deploying works, however, renew --force
also works but for some reason the certificate gets removed from acme.sh
itself (i.e. acme.sh --list shows nothing). Before digging more I wanted to know if this happened to anybody else.
This happens both invoking docker --exec
as well as from a sh
shell inside the container itself.
Any hint about what can be happening? It's really weird since the command exists without any error and everything works.
My first insight that something was happening was that retrying the renew
command was saying that I needed to supply standalone, dns ... etc
parameter. (I am using dns_aws
)
Hi! Thanks a lot for the easy deploy script! I just wanted to add that while I was setting it up on my Synology NAS I noticed that the script is not able to find the default certificate if it has a description This seems to be the reason why, as the wiki guide tells users to keep $SYNO_Certificate empty to create/replace the default cert. There should be a few ways to fix this (like checking for
"is_default":true
instead of an empty description in the response, for example). It would be great in any case if you could add a short note to the guide in the wiki for users to make sure the description text of the default certificate is empty in DSM in case another person has the same problem.
@dibas - Updated the Wiki to indicate that the instructions only work if the description hasn't been added/modified. The is_default
could be interesting, but since default certificates can be changed, I don't want to introduce that as a direct replacement. Potentially as a switch (you must set either SYNO_DefaultCertificate or SYNO_Certificate), still mulling this over.
Since I don't want to downgrad DSM just for the sake of getting the ECC imported & than update again, can somebody tell me how to kinda "brute force" DSM into using the ECC cert anyway?
@Eagle3386 - At this point, there is no way I have found to do this via the GUI or API that we are currently using. My suggestion is that people report this as a bug/regression to Synology and we hope for a future update to fix the issue.
@Eagle3386 - At this point, there is no way I have found to do this via the GUI or API that we are currently using. My suggestion is that people report this as a bug/regression to Synology and we hope for a future update to fix the issue.
@tresni while I happily forward that issue to Synology, by writing "'brute force' DSM", I meant that literally - up to the point where I renamed the mkcert
tool to mkcert_RenamedToPreventFurtherSynologyCertBullshit
so that it definitely stops messing around with valid certificates by replacing them with its self-signed 💩.
After that, I replaced all 4 .pem
files in all occasions & even merged cert.pem
, privkey.pem
, chain.pem
& fullchain
into privkey_fullchain.pem
for MailPlus-Server
- just because Synology, nginx, both or my pure lack of further Linux knowledge weren't able to get nginx up & running with symbolic links in each required location, pointing to the freshly acquired LE certificates.
[Edit] I've used @dvandonkelaar's SynologyLetsEncrypt.sh as a starting point - just in case somebody want's to give a damn about what Synology likes to prevent them from having.
Next, I'm going to modify nginx so that OCSP Stapling works. Because Synology, in their infinite stupidity, even manages to maintain an nginx that is both, basically outdated regarding newer (security) features and lacking to provide such features via DSM web-UI.
When I deploy a certificate issued with acme.sh to my DSM, the certificate is always set as the default certificate, even if I do export SYNO_Certificate="My certificate description"
and export SYNO_Create=1
.
What if I want to keep a certificate already issued using the DSM web interface as the default certificate and then add to it the certificate issued by acme.sh as an additional one, but not as the default one? Is this possible?
Many thanks
I'm attaching the (edited) debug output of the deploy command. deploytoDSMdebug.1.txt
acme.sh v2.8.9
Something seems off here. We specifically check to see if the certificate was default before we update and default to false. And your log shows it should NOT be the default:
[Sat Mar 27 14:47:28 CET 2021] default='false'
@buxm I suggest changing your password as it does appear you missed it in one place in the log. Without seeing the full debug 3 output which includes the list of certificates, I'm not sure I can help further. My non-default certificates do not get made default on updates. Does this only happen when the certificate is first created? (SYNO_Create=1
only matters if the certificate does not already exist.)
@buxm Without seeing the full debug 3 output which includes the list of certificates, I'm not sure I can help further
Please see below the debug 3 output deploytoDSMdebug.txt
Does this only happen when the certificate is first created? (
SYNO_Create=1
only matters if the certificate does not already exist.)
No, it happens every time the deploy hook to DSM is run. I.e. when I manually run it and every time I renew the certificate.
Many thanks for looking into this.
What version of DSM are you on? All the output looks as expected. It is correctly getting the id for the "Test" certificate and then pushes it to synology with the "is_default" flag set to false. The only thing that seems odd to me (compared to my setup) is that you have 2 certificates with blank descriptions. The default one and one from Synology. If it's causing issues when rotating that LE certificate, that could be why. Try giving it a description and see if that helps.
Random background information: as far as I can find the certificate ids are not exposed in the WebUI in any easy way. So I choose to use the Certificate Description as the identifier. The code is not setup to handle a situation where multiple certificates have the same description and results would be undefined. Really it should use the certificate id, but then it would require people either use the Developer Console in their browser or do some API calls on the command line to get the ids.
What version of DSM are you on?
latest: 6.2.4-25556
Try giving it a description and see if that helps.
I have tried now. Unfortunately it didn't help. Here's the new log file deploytoDSMdebug.txt
Many thanks for looking into this.
What version of DSM are you on?
latest: 6.2.4-25556
Try giving it a description and see if that helps.
I have tried now. Unfortunately it didn't help. Here's the new log file deploytoDSMdebug.txt
Many thanks for looking into this.
Hello, I just started using acme.sh and the Synology DSM deploy hook and run into the same problem. The certificate is set as default even though it wasn't the default. The log shows default='false' and the HTTP request POST body contains the following :
Content-Disposition: form-data; name="as_default"
false
Still, the certificate is set as default. It occured to me that the body content differs from the one that is used when importing from the DSM UI, the text "false" is not included :
Content-Disposition: form-data; name="as_default"
A quick and dirty fix at line 124 changing default=false to default="" seems to result in the expected behaviour. https://github.com/acmesh-official/acme.sh/blob/master/deploy/synology_dsm.sh#L124
I'm currently at DSM 6.2.3-25426 Update 3.
Hope this is of any help.
Thank you so much for tracking this down. I am not seeing that behavior on my synology, but I'll see if this change works and get it integrated.
Hi @Run-King & @tresni, many thanks for identifying the potential issue.
I tried again with acme.sh v2.8.9, but unfortunately this didn't resolve the issue on my DSM v. 6.2.4-25556. Updated log file: deploytoDSMdebug.txt
I have tried to change https://github.com/acmesh-official/acme.sh/blob/d0a16b0ec06dd00337b7e2043ab1e3eb35bd6c3c/deploy/synology_dsm.sh#L138
to
if [ ${default} = true ]; then
content="$content${nl}--$delim${nl}Content-Disposition: form-data; name=\"as_default\"${nl}${nl}${default}"
fi
And it has worked :-) New log file with my fix: deploytoDSMdebug_withfix.txt
I have also tested that if I set the same certificate as default on my DSM and then re-deploy it to DSM using acme.sh, that certificate remains the default one. But unfortunately I have not managed to test the fix in all possible scenarios. So that may well break functionality in other cases I have not though/I am not aware of...
@buxm Good to hear you got it running. Apparently the behaviour changed in DSM v. 6.2.4-25556. I tested your change and on my side with DSM 6.2.3-25426 Update 3 and it works fine as well. I actually should have upgraded to the current DSM version so i could have provided an up-to-date solution. Some local issue is withholding me from upgrading though.
Sorry for the delay, but to confirm, we have verified that simply not including the parameter works on 6.2.4 and 6.2.3? I'll see if I can test on 7 shortly.
@tresni Thank you for confirming and verifying the solution for those DSM 6.2.x versions. Hopefully it will work on version 7 as well.
It doesn't seem to work on DSM7, throws an error "synology_dsm.sh: line 139: [: =: unary operator expected" and certificate is added without replacing the default. @Run-King @tresni
What I am seeing on DSM7 is that the only way you can deploy a cert is with SYNO_Create=1. If not, it will fail. Having it set though will not replace the cert, it will add a new one. And next time another. And another. Meanwhile, the original cert expires and everything utilizing it will complain. Very strange @tresni
@throttlemeister what commit hash or version are you using? The current line 139 is as follows and does not contain any [
(test) comparisons.
@throttlemeister what commit hash or version are you using? The current line 139 is as follows and does not contain any
[
(test) comparisons.
@tresni I did a ./acme.sh --upgrade just before trying. The line 139 comes from the if - fi statements around for the fix from @buxm as it shifts the original 138 down one line. I shall try the new file committed.
The problem is that default is likely empty due to a change in the synology_dsm.sh code (changing the default value from false
to ""
.) That would cause the if statement to evaluate to if [ = true ];
which would lead to the error you are seeing. Either look at my change that I PR'd in #3520 or modify the script to use false
as the default value again.
I must be having a different issue, as using the file you committed still gives me the same output (though no longer script error): [Wed May 19 22:29:26 CEST 2021] Unable to find certificate: acme.sh for .sport-touring.eu and $SYNO_Create is not set [Wed May 19 22:29:26 CEST 2021] Error deploy for domain:.sport-touring.eu [Wed May 19 22:29:26 CEST 2021] Deploy error.
certificate is present, it is default, but it doesn't see it nor replace it. Adding new one will work with SYNO_Create=1 but subsequent tries, even with the same cert will fail if it is set to 0. Set it to 1 again, it will work but it will just add a new one and you will end up with a long list of identical certs.
Please attach a --debug 3
log. Make sure to sanitize your password(s) out of the logs as it tends to be printed as part of form data.
debug_log.txt
@tresni attached debug log, and screenshot from inside synology certificates
If I set $SYNO_Create=1, a identical cert is created in addition to the one already present for the same name, same description, same everything except a newer date obviously. Running it again, will create a third, etc.
If $SYNO_Create is not set, always this error.
To illustrate the above:
Ran "./acme.sh --insecure --deploy -d *.sport-touring.eu --deploy-hook synology_dsm" 4 times in succession on the certificates it had created already, 3 times with SYNO_Create set and the last time with it unset.
This was the result:
┌──(throttlemeister웃yggdrasil.sport-touring.eu)-[~/.acme.sh]
└─$ ./deploy.sh
[Sat May 22 19:11:54 CEST 2021] Logging into yggdrasil.sport-touring.eu:5001
[Sat May 22 19:11:54 CEST 2021] Getting certificates in Synology DSM
[Sat May 22 19:11:55 CEST 2021] Generate form POST request
[Sat May 22 19:11:55 CEST 2021] Upload certificate to the Synology DSM
[Sat May 22 19:11:55 CEST 2021] http services were NOT restarted
[Sat May 22 19:11:55 CEST 2021] Success
┌──(throttlemeister웃yggdrasil.sport-touring.eu)-[~/.acme.sh]
└─$ ./deploy.sh
[Sat May 22 19:11:57 CEST 2021] Logging into yggdrasil.sport-touring.eu:5001
[Sat May 22 19:11:57 CEST 2021] Getting certificates in Synology DSM
[Sat May 22 19:11:58 CEST 2021] Generate form POST request
[Sat May 22 19:11:58 CEST 2021] Upload certificate to the Synology DSM
[Sat May 22 19:11:58 CEST 2021] http services were NOT restarted
[Sat May 22 19:11:58 CEST 2021] Success
┌──(throttlemeister웃yggdrasil.sport-touring.eu)-[~/.acme.sh]
└─$ ./deploy.sh
[Sat May 22 19:12:00 CEST 2021] Logging into yggdrasil.sport-touring.eu:5001
[Sat May 22 19:12:00 CEST 2021] Getting certificates in Synology DSM
[Sat May 22 19:12:01 CEST 2021] Generate form POST request
[Sat May 22 19:12:01 CEST 2021] Upload certificate to the Synology DSM
[Sat May 22 19:12:01 CEST 2021] http services were NOT restarted
[Sat May 22 19:12:01 CEST 2021] Success
┌──(throttlemeister웃yggdrasil.sport-touring.eu)-[~/.acme.sh]
└─$ ./deploy.sh
[Sat May 22 19:13:23 CEST 2021] Logging into yggdrasil.sport-touring.eu:5001
[Sat May 22 19:13:24 CEST 2021] Getting certificates in Synology DSM
[Sat May 22 19:13:24 CEST 2021] Unable to find certificate: acme.sh for *.sport-touring.eu and $SYNO_Create is not set
[Sat May 22 19:13:24 CEST 2021] Error deploy for domain:*.sport-touring.eu
[Sat May 22 19:13:24 CEST 2021] Deploy error.
┌──(throttlemeister웃yggdrasil.sport-touring.eu)-[~/.acme.sh]
└─$
and in the Synology:
Weird uh? :)
From my limited understanding on how it should work, it appears the check to see if the certificate exists does not work properly? In theory, in this case, it should have replaced the default certificate 4 times, regardless of $SYNO_Create, no?
Could you add the possibility to mention more than 1 DSM for deployment?
@throttlemeister The problem is the asterisk () in your certificate description. That gets dropped into a sed
and it looks for a certificate with "any number of spaces, followed by any character, followed by 'sport-touring'". That won't match `.sport-touring.eu`
The easy fix is to remove the asterisk from your description and it should start working.
I'll take a look and see if there is a "sed escape" function in acme.sh we can use, but I haven't seen it previously.
@tresni i'm not setting a description, the script is. If setting one avoids a bug, I can work with that. :)
@throttlemeister You had to supply the description in DSM and to acme.sh at some point. Update the description in the DSM for the certificate you want to update (Control Panel -> Security -> Certificates, right click on the certificate -> Edit) and then do
$ SYNO_Certificate="New Certificate Description" ./acme.sh --insecure --deploy -d *.sport-touring.eu --deploy-hook synology_dsm
Or, you can check out tresni/acme.sh@c4bd876e265fca86f7192431d803b81a5d3e1538 where I have fixed the issue and your current descriptions should "just work." I'll be opening a PR for that shortly.
@tresni thanks Brian, that worked flawlessly.
Could you add the possibility to mention more than 1 DSM for deployment?
@CyrosX To make sure I understand correctly, you have multiple DSMs all serving the same certificate and you want to update each of them? Is the login information the same between them? There's no builtin support for calling the same deployhook multiple times. The closest I see is the Qiniu.com deployhook which iterates over some number of domains based on what is supplied. It'd be possible to do something similar with the upload of certificates, but only if login information is the same between all servers.
I'm thinking something like this:
SYNO_Username=admin
SYNO_Password=password
SYNO_Host="dsm1.example.com dsm2.example.com dsm3.example.com"
SYNO_Certificate="Shared Certificate"
If that setup would work for you, it's likely something that could be supported.
@tresni Yes, that sounds like a working and good idea in my setup :). I think acme.sh/deploy/ssh.sh did something similiar with: deploy_ssh_servers Maybe some parts of it, can also be useful?
@CyrosX I've got a first cut of that done. It is pretty straight forward. One decision I made though was to allow it to treat each host as its own failure case, that way if (e.g.) a host is offline, the rest of them should still be updated:
» SYNO_Hostname="192.168.1.1 192.168.1.9 192.168.1.127" ./acme.sh --deploy --deploy-hook synology_dsm -d xyz.example.com
[Wed May 26 17:29:59 MDT 2021] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 28
[Wed May 26 17:29:59 MDT 2021] Error connecting to http://192.168.1.1:5000, error code 28
[Wed May 26 17:29:59 MDT 2021] Logging into http://192.168.1.9:5000
[Wed May 26 17:30:00 MDT 2021] Getting certificates in Synology DSM
[Wed May 26 17:30:00 MDT 2021] Unable to find certificate: "💩" on 192.168.1.9 and $SYNO_Create is not set
[Wed May 26 17:30:00 MDT 2021] Logging into http://192.168.1.127:5000
[Wed May 26 17:30:06 MDT 2021] Getting certificates in Synology DSM
[Wed May 26 17:30:06 MDT 2021] Generate form POST request
[Wed May 26 17:30:06 MDT 2021] Upload certificate to the Synology DSM
[Wed May 26 17:30:07 MDT 2021] http services were NOT restarted
[Wed May 26 17:30:07 MDT 2021] Error deploy for domain:xyz.example.com
[Wed May 26 17:30:07 MDT 2021] Deploy error.
In this log, 192.168.1.1 (Timeout) & 192.168.1.9 (No Certificate) failed, but 192.168.1.127 succeeded. Overall it is still considered a failed deployment though. The major limit today is that everything except the hostname/IP must be the same: same schema, same port, same username/password. If you are using 2FA, this is unlikely to work for multiple hosts unless there is some setting to synchronize the 2FA secret between nodes. (2FA continues to work for 1 host.)
I'm thinking about dropping the SYNO_Host/SYNO_Port/SYNO_Scheme setup and moving to SYNO_URLs or something where you can list specifically SYNO_URLs="https://192.168.1.1:5001 http://192.168.1.9:5000 https://192.168.1.127"
or whatever.
Check it out at https://github.com/tresni/acme.sh/tree/multi_dsm . I'm going to sit on this one for a bit.
I can confirm that acme.sh v2.9.0 no longer sets the deployed certificate as default on my Synology DSM 6.2.4-25556 when the pre-existing version of the certificate was not already set as default. If the certificate was already the default one before deploying a new version, the deploy hook keeps it as the default. Many thanks @tresni for sorting this out (#3520) and @Run-King for helping identifying the issue.
Deployment suddenly started failing with the following in the HTTP POST trace: {"error":{"code":406},"success":false}
What does this error code mean?
The HTTP GET request for the API version succeeded: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":" 0044: auth.cgi"}},"success":true}
@sdebruyn What version of DSM? Where is that error happening? Can you post a debug log?
Did you enable 2 Factor Authentication for the user you are logging in as? See home-assistant/core#36715 which seems to indicate that is the issue. If you have, please review Synology NAS Guide for how to setup acme.sh to deal with 2FA.
Please report bugs for the Synology DSM hook here. Remember to include debug logs