search

usmannasir / cyberpanel

Cyber Panel - The hosting control panel for OpenLiteSpeed
GNU General Public License v3.0
1.54k stars 597 forks source link

[BUG] Self-Signed SSL Certs being Issued for Valid Domains due to Acme.sh Failure #1044

Closed packetdog closed 1 year ago

packetdog commented 1 year ago

Hello,

We're hosting 8 sites on CyberPanel 2.3.4-dev on Ubuntu 22.04 LTS. Everything is updated. We've been experiencing sites losing their SSL certificates as acme.sh fails, and CyberPanel issues a self-signed certificate. This has been documented on the forums (here, here, here, here, here, and this list goes on...), however it was not until today that I was able to pull a log file from acme.sh to see what was happening.

The attached log has been redacted, and all instances of MYDOMAIN are actually a valid and working .com domain name. All instances of IP.IP.IP.IP refer to our public IP address for this server. At issue, in the log file attached, you will find that acme.sh pulls in the website root of /usr/local/lsws/Example/html which is of course not valid, so the challenge can't be completed. The appropriate root for this site would be /home/MYDOMAIN.com/public_html/ which exitsts and has appropriate permissions.

Additionally, manually running the ISSUE SSL command from the CyberPanel web UI corrects this issue, for a few weeks. It seems that some 10-20 days after manually renewing the certificates, the failure reoccurs, and causes a self-signed certificate to be issued. So for whatever reason this seems to break during the automation, but not when running manually from the Web UI.

My only other request related to this is: Is there a way to force CyberPanel to issue a notification if a self-signed certificate is EVER issued? I'd much rather get an email for this issue before my client notices that their site is having an issue.

LOG FILE: acme.sh-MYDOMAIN.log

liufunyu commented 1 year ago

I have the same problem on Ubuntu 22.04 / Cyberpanel 2.3.3, it happens every Sunday !!!!

JosephChuks commented 1 year ago

I have this exact issue. Having to manually reissue SSL cert to over 150+ sites every 2 weeks (Sundays at 00:00 hours GMT) is annoying as clients wake you up with their sites showing security risk.

berksmbl commented 1 year ago

We are experiencing a similar issue where we already have an existing SSL certificate and we add the relevant SSL certificate to the website through CyberPanel. Everything works fine, but even though the certificate still has a long time to expire, CyberPanel replaces it with a self-signed certificate, which causes a security warning. When we manually add the same certificate again, the issue is resolved, but after a short while, the certificate is changed again automatically.

packetdog commented 1 year ago

FYI- The is some additional discussion about this in the CyberPanel Facebook Group at https://www.facebook.com/groups/cyberpanel/posts/3301268023518156/ and https://www.facebook.com/groups/cyberpanel/posts/3300444160267209/.

PorkStew commented 1 year ago

I've had the exact same issue, it seems that it's related to the update because i could issue SSL's without an issue before updates.

I've made a post on CyberPanel fourms explaining the issue.

https://community.cyberpanel.net/t/cant-issue-ssl-certs/41616

cymode commented 1 year ago

Same issue here since 2.3.3. re-creating SSL certs manually/via cron job via "cme.sh --renew -d domain.com --force" for dozens of domains every 2 weeks is not ideal. Cannot figure out why Cyberpanel can't issue it, not like it's a difficult command... 😅. Self-signed certificates should result in an immediate warning via e-mail in my opinion as that can be disastrous for sales and google ads budget if not caught quickly.

cymode commented 1 year ago

@usmannasir, I can provide access to server if it helps to troubleshoot. I've got a 5-6 unused domains that are currently still with self-signed certificate and have stayed that way for my own troubleshooting.

jintron commented 1 year ago

Wasted quite a bit of time trying to make sense of the SSL issue. The same issues come up, if you try to create multiple domains (incl SSL) via cyberpanel CLI on Rocky Linux 8.4. Sometimes changing the vhost conf to reference the right path to public_html works, sometimes it doesn´t. Sometimes fixing the folder permissions work, sometimes it doesn´t. Sometimes an Certificate for another website (which was created earlier) is assiged to a newly created website. Sometimes creating just the website via CLI without SSL and then issuing the Certificate via Certbot works, sometimes not. I really tried to wrap my head around the issue and isolate the problem, but I give up. Think the easiest fix for me (for now) is to skip this version.

usmannasir commented 1 year ago

@shbs9 where you able to reproduce issue on any of the mentioned users?

usmannasir commented 1 year ago

If any of you guys have opened ticket with server login details, feel free to mention ticket number.

usmannasir commented 1 year ago

PS: New webroot for latest version of CyberPanel is /usr/local/lsws/Example/html, can you guys show me your vhost conf?

cymode commented 1 year ago

@usmannasir what file do you want and I'll copy/paste the details, or did you just want screenshots from the OLS backend?

liufunyu commented 1 year ago

image

vhost.conf is in /usr/local/lsws/conf/vhosts/mydomain.com/vhost.conf

root@c2:/usr/local/lsws/conf/vhosts/c2.cocosuit.com# cat vhost.conf
docRoot                   $VH_ROOT/public_html
vhDomain                  $VH_NAME
vhAliases                 www.$VH_NAME
adminEmails               liufunyu000@gmail.com
enableGzip                1
enableIpGeo               1

index  {
  useServer               0
  indexFiles              index.php, index.html
}

errorlog $VH_ROOT/logs/$VH_NAME.error_log {
  useServer               0
  logLevel                WARN
  rollingSize             10M
}

accesslog $VH_ROOT/logs/$VH_NAME.access_log {
  useServer               0
  logFormat               "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""
  logHeaders              5
  rollingSize             10M
  keepDays                10
  compressArchive         1
}

scripthandler  {
  add                     lsapi:ccoco3242 php
}

extprocessor ccoco3242 {
  type                    lsapi
  address                 UDS://tmp/lshttpd/ccoco3242.sock
  maxConns                10
  env                     LSAPI_CHILDREN=10
  initTimeout             600
  retryTimeout            0
  persistConn             1
  pcKeepAliveTimeout      1
  respBuffer              0
  autoStart               1
  path                    /usr/local/lsws/lsphp80/bin/lsphp
  extUser                 ccoco3242
  extGroup                ccoco3242
  memSoftLimit            2047M
  memHardLimit            2047M
  procSoftLimit           400
  procHardLimit           500
}

phpIniOverride  {

}

module cache {
 storagePath /usr/local/lsws/cachedata/$VH_NAME
}

rewrite  {
 enable                  1
  autoLoadHtaccess        1
}

context /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

vhssl  {
  keyFile                 /etc/letsencrypt/live/c2.cocosuit.com/privkey.pem
  certFile                /etc/letsencrypt/live/c2.cocosuit.com/fullchain.pem
  certChain               1
  sslProtocol             24
  enableECDHE             1
  renegProtection         1
  sslSessionCache         1
  enableSpdy              15
  enableStapling           1
  ocspRespMaxAge           86400
}
Jomateix7 commented 1 year ago

I have the same issue on my servers. Will there be a fix soon?

VR51 commented 1 year ago

Seems like the SSL expires every 7 days.

The working certificate is listed as issued by Let's Encrypt The non working cert that becomes active every 7 days is listed as issued by Des.

Alma Linux CyberPanel 2.3 Build: 7e18b 8688c 61266 566d7d 401c 84570 1888b 08a32 All packages are up to date.

packetdog commented 1 year ago

FYI- Per @usmannasir comment in my feature request (here: https://github.com/usmannasir/cyberpanel/issues/1045#issuecomment-1499778846) we are able to turn on emailDebug to receive notifications about self-signed SSL certs. All you have to do is run the command: touch /usr/local/CyberCP/emailDebug which creates an empty file at that path, and tells CyberPanel to generate those logs via email.

irockbot commented 1 year ago

@usmannasir please verify the vhost posted by @liufunyu is correct. If any changes has to be made in this vhost config prior to ssl renew issue on Sundays, we have to do it now. We need right vhost template file.

usmannasir commented 1 year ago

image

vhost.conf is in /usr/local/lsws/conf/vhosts/ccoco.com/vhost.conf

`docRoot $VH_ROOT/public_html vhDomain $VH_NAME vhAliases www.$VH_NAME adminEmails liufunyu000@gmail.com enableGzip 1 enableIpGeo 1

index { useServer 0 indexFiles index.php, index.html }

errorlog $VH_ROOT/logs/$VH_NAME.error_log { useServer 0 logLevel WARN rollingSize 10M }

accesslog $VH_ROOT/logs/$VH_NAME.access_log { useServer 0 logFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" logHeaders 5 rollingSize 10M keepDays 10 compressArchive 1 }

scripthandler { add lsapi:ccoco3242 php }

extprocessor ccoco3242 { type lsapi address UDS://tmp/lshttpd/ccoco3242.sock maxConns 10 env LSAPI_CHILDREN=10 initTimeout 600 retryTimeout 0 persistConn 1 pcKeepAliveTimeout 1 respBuffer 0 autoStart 1 path /usr/local/lsws/lsphp80/bin/lsphp extUser ccoco3242 extGroup ccoco3242 memSoftLimit 2047M memHardLimit 2047M procSoftLimit 400 procHardLimit 500 }

phpIniOverride {

}

module cache { storagePath /usr/local/lsws/cachedata/$VH_NAME "vhost.conf" 91L, 2189B `

Hmm, this one is wrong, see the correct one with added context

docRoot                   $VH_ROOT/public_html
vhDomain                  $VH_NAME
vhAliases                 www.$VH_NAME
adminEmails               usman@cyberpersons.com
enableGzip                1
enableIpGeo               1

index  {
  useServer               0
  indexFiles              index.php, index.html
}

errorlog $VH_ROOT/logs/$VH_NAME.error_log {
  useServer               0
  logLevel                WARN
  rollingSize             10M
}

accesslog $VH_ROOT/logs/$VH_NAME.access_log {
  useServer               0
  logFormat               "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""
  logHeaders              5
  rollingSize             10M
  keepDays                10  
  compressArchive         1
}

scripthandler  {
  add                     lsapi:cyber2909 php
}

extprocessor cyber2909 {
  type                    lsapi
  address                 UDS://tmp/lshttpd/cyber2909.sock
  maxConns                10
  env                     LSAPI_CHILDREN=10
  initTimeout             600
  retryTimeout            0
  persistConn             1
  pcKeepAliveTimeout      1
  respBuffer              0
  autoStart               1
  path                    /usr/local/lsws/lsphp74/bin/lsphp
  extUser                 cyber2909
  extGroup                cyber2909
  memSoftLimit            2047M
  memHardLimit            2047M
  procSoftLimit           400
  procHardLimit           500
}

phpIniOverride  {

}

module cache {
 storagePath /usr/local/lsws/cachedata/$VH_NAME
}

rewrite  {
 enable                  1
  autoLoadHtaccess        1
}

context /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

vhssl  {
  keyFile                 /etc/letsencrypt/live/cyberpanel.net/privkey.pem
  certFile                /etc/letsencrypt/live/cyberpanel.net/fullchain.pem
  certChain               1
  sslProtocol             24
  enableECDHE             1
  renegProtection         1
  sslSessionCache         1
  enableSpdy              15
  enableStapling           1
  ocspRespMaxAge           86400
}

Take a special note of this part, which is actually used in generating SSL

context /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

You can just copy-paste it to your vhost conf.

cymode commented 1 year ago

I've copied the below into the vhost of one of the domains (that section was missing), but it made no difference.

 /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

Here is my full vhost for one of the domains that can't get a new SSL cert via Cyberpanel (only via command prompt it would work):

docRoot                   $VH_ROOT/public_html
vhDomain                  $VH_NAME
vhAliases                 www.$VH_NAME
adminEmails               removedforprivacy@mail.com
enableGzip                1
enableIpGeo               1

index  {
  useServer               0
  indexFiles              index.php, index.html
}

errorlog $VH_ROOT/logs/$VH_NAME.error_log {
  useServer               0
  logLevel                ERROR
  rollingSize             10M
}

accesslog $VH_ROOT/logs/$VH_NAME.access_log {
  useServer               0
  logFormat               "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""
  logHeaders              5
  rollingSize             10M
  keepDays                10  
  compressArchive         1
}

scripthandler  {
  add                     lsapi:group3111 php
}

extprocessor green3111 {
  type                    lsapi
  address                 UDS://tmp/lshttpd/green3111.sock
  maxConns                10
  env                     LSAPI_CHILDREN=10
  initTimeout             600
  retryTimeout            0
  persistConn             1
  pcKeepAliveTimeout      1
  respBuffer              0
  autoStart               1
  path                    /usr/local/lsws/lsphp80/bin/lsphp
  extUser                 group3111
  extGroup                group3111
  memSoftLimit            2047M
  memHardLimit            2047M
  procSoftLimit           400
  procHardLimit           500
}

phpIniOverride  {
php_admin_value open_basedir "/tmp:$VH_ROOT"
}

rewrite  {
  enable                  1
  autoLoadHtaccess        1
}

context /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

vhssl  {
  keyFile                 /etc/letsencrypt/live/removedforprivacy.de/privkey.pem
  certFile                /etc/letsencrypt/live/removedforprivacy.de/fullchain.pem
  certChain               1
  sslProtocol             24
  ciphers                 EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4
  enableECDHE             1
  renegProtection         1
  sslSessionCache         1
  enableSpdy              15
  enableStapling           1
  ocspRespMaxAge           86400
}
irockbot commented 1 year ago

@usmannasir Thank you for the correct template. I have correctly updated config as per this new template. Lets see whether the SSL issue produces tomorrow (Sunday). I have ensured all other parameters are correctly updated such as DNS records, updated acme, etc. If still it produces the issue, then it should be something else.

usmannasir commented 1 year ago

I've copied the below into the vhost of one of the domains (that section was missing), but it made no difference.

 /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

Here is my full vhost for one of the domains that can't get a new SSL cert via Cyberpanel (only via command prompt it would work):

docRoot                   $VH_ROOT/public_html
vhDomain                  $VH_NAME
vhAliases                 www.$VH_NAME
adminEmails               removedforprivacy@mail.com
enableGzip                1
enableIpGeo               1

index  {
  useServer               0
  indexFiles              index.php, index.html
}

errorlog $VH_ROOT/logs/$VH_NAME.error_log {
  useServer               0
  logLevel                ERROR
  rollingSize             10M
}

accesslog $VH_ROOT/logs/$VH_NAME.access_log {
  useServer               0
  logFormat               "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""
  logHeaders              5
  rollingSize             10M
  keepDays                10  
  compressArchive         1
}

scripthandler  {
  add                     lsapi:group3111 php
}

extprocessor green3111 {
  type                    lsapi
  address                 UDS://tmp/lshttpd/green3111.sock
  maxConns                10
  env                     LSAPI_CHILDREN=10
  initTimeout             600
  retryTimeout            0
  persistConn             1
  pcKeepAliveTimeout      1
  respBuffer              0
  autoStart               1
  path                    /usr/local/lsws/lsphp80/bin/lsphp
  extUser                 group3111
  extGroup                group3111
  memSoftLimit            2047M
  memHardLimit            2047M
  procSoftLimit           400
  procHardLimit           500
}

phpIniOverride  {
php_admin_value open_basedir "/tmp:$VH_ROOT"
}

rewrite  {
  enable                  1
  autoLoadHtaccess        1
}

context /.well-known/acme-challenge {
  location                /usr/local/lsws/Example/html/.well-known/acme-challenge
  allowBrowse             1

  rewrite  {

  }
  addDefaultCharset       off

  phpIniOverride  {

  }
}

vhssl  {
  keyFile                 /etc/letsencrypt/live/removedforprivacy.de/privkey.pem
  certFile                /etc/letsencrypt/live/removedforprivacy.de/fullchain.pem
  certChain               1
  sslProtocol             24
  ciphers                 EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4
  enableECDHE             1
  renegProtection         1
  sslSessionCache         1
  enableSpdy              15
  enableStapling           1
  ocspRespMaxAge           86400
}

Do you have latest commit of CyberPanel? Also try removing this line from your vhost conf.

php_admin_value open_basedir "/tmp:$VH_ROOT"
cymode commented 1 year ago

That worked!

removing the following from the vhost.conf worked:

php_admin_value open_basedir "/tmp:$VH_ROOT"

Yes, I'm on the latest commit: 7e18b8688c61266566d7d401c845701888b08a32 Do I now have to go into 100+ domains and delete this entry? Does this mean open_basedir protection is removed?

thanks

usmannasir commented 1 year ago

openbase_dir should be blocking php requests only, i will revert to you on this.

shbs9 commented 1 year ago

@usmannasir just reproduced this for one ssl and this part is effecting SSL issue. I think new update and check by acme

phpIniOverride { php_admin_value open_basedir "/tmp:$VH_ROOT" }

irockbot commented 1 year ago

@usmannasir Unfortunately issue still persists but this time no self signed. instead it generates letsencrypt certificate only for non www domain. Why this is trying to install certificate on every week Sunday?? that is our question. No answer for this question yet. Please check below error logs, i see modsec too blocking acme challenge. Isn't it?

[04.09.2023_00-00-10] Status Code: 403 for: http://www.mydomain.com/.well-known/acme-challenge/mydomain.com. Error: <!DOCTYPE html>

403 Forbidden

403

Forbidden

Access to this resource on the server is denied!


Proudly powered by LiteSpeed Web Server

Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site.

[04.09.2023_00-00-10] Status Code: 403 for: http://mydomain.com/.well-known/acme-challenge/mydomain.com. Error: <!DOCTYPE html>

403 Forbidden

403

Forbidden

Access to this resource on the server is denied!


Proudly powered by LiteSpeed Web Server

Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site.

[04.09.2023_00-00-11] /root/.acme.sh/acme.sh --issue -d mydomain.com -d www.mydomain.com --cert-file /etc/letsencrypt/live/mydomain.com/cert.pem --key-file /etc/letsencrypt/live/mydomain.com/privkey.pem --fullchain-file /etc/letsencrypt/live/mydomain.com/fullchain.pem -w /usr/local/lsws/Example/html -k ec-256 --force --server letsencrypt [04.09.2023_00-00-11] Failed to obtain SSL for: mydomain.com and: www.mydomain.com [04.09.2023_00-00-11] Trying to obtain SSL for: mydomain.com [04.09.2023_00-00-17] Successfully obtained SSL for: mydomain.com [04.09.2023_00-00-17] {'mydomain@gmail.com': (554, b'5.7.1 mydomain@gmail.com: Relay access denied')} [04.09.2023_00-00-19] [Errno 2] No such file or directory: 'postmap': 'postmap'. [ProcessUtilities.normalExecutioner.Base] [04.09.2023_00-00-19] [Errno 2] No such file or directory: '/home/cyberpanel/git'. [IncScheduler.git:90]

Now i have to go back to custom ssl and block crontab suggested here https://community.cyberpanel.net/t/ssl-auto-change-to-lets-encrypt-every-month/41586/7

I am done with this auto ssl issue. This is purely CP 2.3.3 BUG!!

liufunyu commented 1 year ago

I'm so sorry, previously pasted incomplete code, which I corrected just now. Then, I compared the vhost.conf files of CP 2.3.2 and CP 2.3.3 and found that they are exactly the same. So, it's not an issue with the vhost.conf file.

cymode commented 1 year ago

I don't know why removing php_admin_value open_basedir "/tmp:$VH_ROOT" worked, but it's not the solution. I've got a subdomin that just reset to self-signed and it does not have base_dir enabled, i.e, it's not in the config file either.

liufunyu commented 1 year ago

So the issue lies between the differences in SSL config between CP 2.3.3 and 2.3.2 versions. Is it possible to ask the development team what specific changes were made?

usmannasir commented 1 year ago

I don't know why removing php_admin_value open_basedir "/tmp:$VH_ROOT" worked, but it's not the solution. I've got a subdomin that just reset to self-signed and it does not have base_dir enabled, i.e, it's not in the config file either.

can you show vhost conf of subdomain?

usmannasir commented 1 year ago

@usmannasir Unfortunately issue still persists but this time no self signed. instead it generates letsencrypt certificate only for non www domain. Why this is trying to install certificate on every week Sunday?? that is our question. No answer for this question yet. Please check below error logs, i see modsec too blocking acme challenge. Isn't it?

[04.09.2023_00-00-10] Status Code: 403 for: http://www.mydomain.com/.well-known/acme-challenge/mydomain.com. Error:

403 Forbidden

403

Forbidden

Access to this resource on the server is denied!

Proudly powered by LiteSpeed Web Server

Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site.

[04.09.2023_00-00-10] Status Code: 403 for: http://mydomain.com/.well-known/acme-challenge/mydomain.com. Error:

403 Forbidden

403

Forbidden

Access to this resource on the server is denied!

Proudly powered by LiteSpeed Web Server

Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site.

[04.09.2023_00-00-11] /root/.acme.sh/acme.sh --issue -d mydomain.com -d www.mydomain.com --cert-file /etc/letsencrypt/live/mydomain.com/cert.pem --key-file /etc/letsencrypt/live/mydomain.com/privkey.pem --fullchain-file /etc/letsencrypt/live/mydomain.com/fullchain.pem -w /usr/local/lsws/Example/html -k ec-256 --force --server letsencrypt [04.09.2023_00-00-11] Failed to obtain SSL for: mydomain.com and: www.mydomain.com [04.09.2023_00-00-11] Trying to obtain SSL for: mydomain.com [04.09.2023_00-00-17] Successfully obtained SSL for: mydomain.com [04.09.2023_00-00-17] {'mydomain@gmail.com': (554, b'5.7.1 mydomain@gmail.com: Relay access denied')} [04.09.2023_00-00-19] [Errno 2] No such file or directory: 'postmap': 'postmap'. [ProcessUtilities.normalExecutioner.Base] [04.09.2023_00-00-19] [Errno 2] No such file or directory: '/home/cyberpanel/git'. [IncScheduler.git:90]

Now i have to go back to custom ssl and block crontab suggested here https://community.cyberpanel.net/t/ssl-auto-change-to-lets-encrypt-every-month/41586/7

I am done with this auto ssl issue. This is purely CP 2.3.3 BUG!!

your vhost conf?

usmannasir commented 1 year ago

c2.cocosuit.com

can you show your rewrite rules?

liufunyu commented 1 year ago

c2.cocosuit.com

can you show your rewrite rules?

image

# BEGIN LSCACHE
## LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block! ##
<IfModule LiteSpeed>
RewriteEngine on
CacheLookup on
RewriteRule .* - [E=Cache-Control:no-autoflush]
RewriteRule \.litespeed_conf\.dat - [F,L]

### marker NOCACHE COOKIES start ###
RewriteCond %{HTTP_COOKIE} /dashboard
RewriteRule .* - [E=Cache-Control:no-cache]
### marker NOCACHE COOKIES end ###

### marker CACHE RESOURCE start ###
RewriteRule wp-content/.*/[^/]*(responsive|css|js|dynamic|loader|fonts)\.php - [E=cache-control:max-age=3600]
### marker CACHE RESOURCE end ###

### marker LOGIN COOKIE start ###
RewriteRule .? - [E="Cache-Vary:,wp-postpass_cc1d30b2b219830f12ab6fcf27fdf7b2"]
### marker LOGIN COOKIE end ###

### marker FAVICON start ###
RewriteRule favicon\.ico$ - [E=cache-control:max-age=86400]
### marker FAVICON end ###

### marker DROPQS start ###
CacheKeyModify -qs:fbclid
CacheKeyModify -qs:gclid
CacheKeyModify -qs:utm*
CacheKeyModify -qs:_ga
### marker DROPQS end ###

</IfModule>
## LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block! ##
# END LSCACHE
# BEGIN NON_LSCACHE
## LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block! ##
## LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block! ##
# END NON_LSCACHE

# BEGIN WordPress
# The directives (lines) between "BEGIN WordPress" and "END WordPress" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress
# BEGIN WPvivid Rewrite Rule for LiteSpeed
# The directives (lines) between "BEGIN WPvivid Rewrite Rule for LiteSpeed" and "END WPvivid Rewrite Rule for LiteSpeed" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.

# END WPvivid Rewrite Rule for LiteSpeed
usmannasir commented 1 year ago

BEGIN LSCACHE

LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block!

RewriteEngine on CacheLookup on RewriteRule .* - [E=Cache-Control:no-autoflush] RewriteRule \.litespeed_conf\.dat - [F,L] ### marker NOCACHE COOKIES start ### RewriteCond %{HTTP_COOKIE} /dashboard RewriteRule .* - [E=Cache-Control:no-cache] ### marker NOCACHE COOKIES end ### ### marker CACHE RESOURCE start ### RewriteRule wp-content/.*/[^/]*(responsive|css|js|dynamic|loader|fonts)\.php - [E=cache-control:max-age=3600] ### marker CACHE RESOURCE end ### ### marker LOGIN COOKIE start ### RewriteRule .? - [E="Cache-Vary:,wp-postpass_cc1d30b2b219830f12ab6fcf27fdf7b2"] ### marker LOGIN COOKIE end ### ### marker FAVICON start ### RewriteRule favicon\.ico$ - [E=cache-control:max-age=86400] ### marker FAVICON end ### ### marker DROPQS start ### CacheKeyModify -qs:fbclid CacheKeyModify -qs:gclid CacheKeyModify -qs:utm* CacheKeyModify -qs:_ga ### marker DROPQS end ###

LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block!

END LSCACHE

BEGIN NON_LSCACHE

LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block!

LITESPEED WP CACHE PLUGIN - Do not edit the contents of this block!

END NON_LSCACHE

BEGIN WordPress

The directives (lines) between "BEGIN WordPress" and "END WordPress" are

dynamically generated, and should only be modified via WordPress filters.

Any changes to the directives between these markers will be overwritten.

RewriteEngine On RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}] RewriteBase / RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L]

END WordPress

BEGIN WPvivid Rewrite Rule for LiteSpeed

The directives (lines) between "BEGIN WPvivid Rewrite Rule for LiteSpeed" and "END WPvivid Rewrite Rule for LiteSpeed" are

dynamically generated, and should only be modified via WordPress filters.

Any changes to the directives between these markers will be overwritten.

END WPvivid Rewrite Rule for LiteSpeed

Can you create a static files in public_html and try to access it?

liufunyu commented 1 year ago

https://c2.cocosuit.com/test.txt 1681037590021

packetdog commented 1 year ago

So the issue lies between the differences in SSL config between CP 2.3.3 and 2.3.2 versions. Is it possible to ask the development team what specific changes were made?

@liufunyu Isn't the whole point of Github and platforms like this related to version control and being able to revert a change? I think there must be a way to get a side by side list of changes from the 2.3.2 and 2.3.3 branch. I'm not a developer or knowledgable enough to figure that out.

I'm so sorry, previously pasted incomplete code, which I corrected just now. Then, I compared the vhost.conf files of CP 2.3.2 and CP 2.3.3 and found that they are exactly the same. So, it's not an issue with the vhost.conf file.

Thank you for confirming this for us- I suspected this. I'm sure many people dig into the depths of OLS and their configurations and customize things. In my case, I installed CyberPanel and allowed everything to default. I'm not using the reselling features, or even any of the email/webmail features. I'm just hosting Wordpress sites on it, and all configured through the port 8090 web UI. The only changes I've ever made to the server are: apt-get update; apt-get upgrade, and the CyberPanel Update.sh script from Github following the instructions on how to update. So IF this issue was caused in a vhost.conf, or some other area of OLS, then it's something that came in through the package manager, or something that the update script changed. This issue started happening for me after the update to 2.3.3 for sure. Hopefully the development team can figure out what change was pushed and tell us how to revert it, or revert it in 2.3.4-dev.

Jomateix7 commented 1 year ago

My use case is almost the same as @packetdog here.

packetdog commented 1 year ago

FYI, apparently this is how you can see all of the changes between branches on Github, so someone that is smarter than I- can you figure out where SSL broke?

https://github.com/usmannasir/cyberpanel/compare/v2.3.2...v2.3.3

usmannasir commented 1 year ago

I was talking to George (LiteSpeed developer). He confirmed that openbase_dir protection does not affect static files, unless there is a rewrite rules that process all files through php.

Apart from that following things can be considered

  1. Cloudflare full strick mode
  2. Missing context in vhost conf
  3. Rewrite rules preventing from accessing the verification files

These are all the cases beyond what I can do, I've also explained that here -> https://community.cyberpanel.net/t/how-to-fix-ssl-issues-in-cyberpanel/90

We are also brining SSL via dns verification method. (Cloudflare and CyberPanel DNS supported), that will be much more easy for anything using Cloudflare cf.

berksmbl commented 1 year ago

@usmannasir

My problem is that despite having a valid certificate for 15 years that I manually entered, it constantly keeps changing the valid certificate automatically.

Jomateix7 commented 1 year ago

Only one of my domain is connected to CloudFlare, but others which have the same SSL issue are not connected neither CloudFlare nor Quic. All problematic instances are on the latest version of CP and trying to use Let's Encrypt.

packetdog commented 1 year ago

These are all the cases beyond what I can do, I've also explained that here

@usmannasir It sounds like you're giving up on this issue? Please listen to us, several users here have confirmed that something changed from 2.3.2 to 2.3.3 that's causing acme.sh to look in /usr/local/lsws/Example/html for the acme-challenge file, and it's not there. This only happens with automated renewals, on a weekly or bi-weekly basis. You can point to issues with a vhost.conf, but that is beside the point- many of us are using the default configuration that CyberPanel installs! I understand there are probably many users that make changes to the default files, I can assure you that I have not.

So when acme.sh can't find the acme-challenge file, CyberPanel issues a self-signed certificate for the domain. When we then use IssueSSL from inside the web UI this temporarily fixes the issue. So that means that the IssueSSL process is working correctly, but some part of the cron automation of acme.sh was broken between changes in 2.3.2 and 2.3.3.

We need to figure out why acme.sh is getting this incorrect path passed to it when doing automated checks/reissues. As I've said before, I'm not a developer, but something changed. You can add new features or ways of working with Let's Encrypt, but if you don't fix this bug, people will stop using CyberPanel. I can't rely on a solution that loses the SSL certificate every few weeks.

Only one of my domain is connected to CloudFlare, but others which have the same SSL issue are not connected neither CloudFlare nor Quic.

@Jomateix7 This is the same for me, only 1 site on cloudflare but all sites not working.

liufunyu commented 1 year ago

Yes! This is the key point "We need to figure out why acme.sh is getting this incorrect path passed to it when doing automated checks/reissues." @usmannasir Could you please report to George (LiteSpeed developer) this point!! I really enjoy using Cyberpanel with OpenLiteSpeed ... please fix it~ I don't want to use Plesk Nginx >___<

irockbot commented 1 year ago

@usmannasir Unfortunately issue still persists but this time no self signed. instead it generates letsencrypt certificate only for non www domain. Why this is trying to install certificate on every week Sunday?? that is our question. No answer for this question yet. Please check below error logs, i see modsec too blocking acme challenge. Isn't it? [04.09.2023_00-00-10] Status Code: 403 for: http://www.mydomain.com/.well-known/acme-challenge/mydomain.com. Error:

403 Forbidden

403

Forbidden

Access to this resource on the server is denied! Proudly powered by LiteSpeed Web Server Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site. [04.09.2023_00-00-10] Status Code: 403 for: http://mydomain.com/.well-known/acme-challenge/mydomain.com. Error:

403 Forbidden

403

Forbidden

Access to this resource on the server is denied! Proudly powered by LiteSpeed Web Server Please be advised that LiteSpeed Technologies Inc. is not a web hosting company and, as such, has no control over content found on this site. [04.09.2023_00-00-11] /root/.acme.sh/acme.sh --issue -d mydomain.com -d www.mydomain.com --cert-file /etc/letsencrypt/live/mydomain.com/cert.pem --key-file /etc/letsencrypt/live/mydomain.com/privkey.pem --fullchain-file /etc/letsencrypt/live/mydomain.com/fullchain.pem -w /usr/local/lsws/Example/html -k ec-256 --force --server letsencrypt [04.09.2023_00-00-11] Failed to obtain SSL for: mydomain.com and: www.mydomain.com [04.09.2023_00-00-11] Trying to obtain SSL for: mydomain.com [04.09.2023_00-00-17] Successfully obtained SSL for: mydomain.com [04.09.2023_00-00-17] {'mydomain@gmail.com': (554, b'5.7.1 mydomain@gmail.com: Relay access denied')} [04.09.2023_00-00-19] [Errno 2] No such file or directory: 'postmap': 'postmap'. [ProcessUtilities.normalExecutioner.Base] [04.09.2023_00-00-19] [Errno 2] No such file or directory: '/home/cyberpanel/git'. [IncScheduler.git:90] Now i have to go back to custom ssl and block crontab suggested here https://community.cyberpanel.net/t/ssl-auto-change-to-lets-encrypt-every-month/41586/7 I am done with this auto ssl issue. This is purely CP 2.3.3 BUG!!

your vhost conf?

docRoot $VH_ROOT/public_html vhDomain $VH_NAME vhAliases www.$VH_NAME adminEmails mydomain@gmail.com enableGzip 1 enableIpGeo 1

errorlog $VH_ROOT/logs/$VH_NAME.error_log { useServer 0 logLevel WARN rollingSize 10M }

accesslog $VH_ROOT/logs/$VH_NAME.access_log { useServer 0 logFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" logHeaders 5 rollingSize 10M keepDays 10 compressArchive 1 }

index { useServer 0 indexFiles index.php, index.html }

scripthandler { add lsapi:user99 php }

extprocessor user99 { type lsapi address UDS://tmp/lshttpd/user99.sock maxConns 10 env LSAPI_CHILDREN=10 initTimeout 600 retryTimeout 0 persistConn 1 pcKeepAliveTimeout 1 respBuffer 0 autoStart 1 path /usr/local/lsws/lsphp70/bin/lsphp extUser user99 extGroup user99 memSoftLimit 2047M memHardLimit 2047M procSoftLimit 400 procHardLimit 500 }

context /.well-known/acme-challenge { location /usr/local/lsws/Example/html/.well-known/acme-challenge allowBrowse 1 addDefaultCharset off }

rewrite { enable 1 autoLoadHtaccess 1 }

vhssl { keyFile /etc/letsencrypt/live/mydomain.com/privkey.pem certFile /etc/letsencrypt/live/mydomain.com/fullchain.pem certChain 1 enableECDHE 1 renegProtection 1 sslSessionCache 1 enableSpdy 15 enableStapling 1 ocspRespMaxAge 86400 }

module cache { storagePath /usr/local/lsws/cachedata/$VH_NAME }

@packetdog spot on! this issue is nothing to do with vhost, rewrite rules, etc. This is purely 2.3.3 cp bug since 2.3.2 works earlier. Still our question is not answered by @usmannasir why cyberpanel acme.h tries to install ssl every week instead of 15 days before ssl expiry?

shbs9 commented 1 year ago

@irockbot let me explain that cyberpanel does not run the SSL issue command every 7 days. what cyberpanel does cyberpanel check for SSL every day and when cyberpanel gets that SSL is expired it runs the SSL issue command? So basically cyberoanel do not run SSL issue command every 7 days it only runs before expiry. Also maybe the issue the rewrite rules and vhost effect the acme SSL because you know acme have to verify domain before issuing SSL so rewrite rule and vhost config some time create issue for this

irockbot commented 1 year ago

@shbs9 Thank you for your explanation. I have already pasted my vhost config above and here is the rewrite rules, `

1.To use URL Alias you need to be running apache with mod_rewrite enabled.

2. In your opencart directory rename htaccess.txt to .htaccess.

For any support issues please visit: http://www.opencart.com

Options +FollowSymlinks

Prevent Directoy listing

Options -Indexes

SEO URL Settings

RewriteEngine On RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} !^www. RewriteRule ^(.*)$ https://www.%{SERVER_NAME}/$1 [L,R=301]

If your opencart installation does not run on the main web folder make sure you folder it does run in ie. / becomes /shop/

RewriteBase / RewriteRule ^..(log|ini|tpl)$ - [F,L,NC] RewriteRule ^sitemap.xml$ index.php?route=extension/feed/google_sitemap [L] RewriteRule ^googlebase.xml$ index.php?route=extension/feed/google_base [L] RewriteRule ^system/download/(.) index.php?route=error/not_found [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_URI} !..(ico|gif|jpg|jpeg|png|webp|js|css) RewriteRule ^([^?]) index.php?route=$1 [L,QSA]

Additional Settings that may need to be enabled for some servers

Uncomment the commands by removing the # sign in front of it.

If you get an "Internal Server Error 500" after enabling any of the following settings, restore the # as this means your host doesn't allow that.

1. If your cart only allows you to add one item at a time, it is possible register_globals is on. This may work to disable it:

php_flag register_globals off

2. If your cart has magic quotes enabled, This may work to disable it:

php_flag magic_quotes_gpc Off

3. Set max upload file size. Most hosts will limit this and not allow it to be overridden but you can try

php_value upload_max_filesize 999M

4. set max post size. uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields

php_value post_max_size 999M

5. set max time script can take. uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields

php_value max_execution_time 200

6. set max time for input to be recieved. Uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields

php_value max_input_time 200

7. disable open_basedir limitations

php_admin_value open_basedir none

` Please let me know what type of issue you have found with my vhost and rewrite rules which was obviously worked well on cp 2.3.2 and earlier.

Jomateix7 commented 1 year ago

Sorry if I step on any toes here, but then I would like to ask, what changed on version 2.3.3? Because we did not change any configuration on our side. There are CP instances I run, which I have not upgraded to 2.3.3 and they have no issue.

VR51 commented 1 year ago

As others have said, this issue began with the upgrade from 2.3.2 to 2.3.3

Hope that info helps you resolve this issue.

liufunyu commented 1 year ago

Could I rollback to CP 2.3.2 and how to?

I use this : sh <(curl https://raw.githubusercontent.com/usmannasir/cyberpanel/2.3.2/cyberpanel_upgrade.sh || wget -O - https://raw.githubusercontent.com/usmannasir/cyberpanel/2.3.2/cyberpanel_upgrade.sh)

but it did not work.

kydtf commented 1 year ago

I have encountered the same problem and hope to find a solution here.

liufunyu commented 1 year ago

I have reinstalled a new server with Ubuntu 20.04 and CyberPanel 2.3.2, and then transferred all the sites to the new server. I will upgrade to version 2.3.3 only after it is running SSL smoothly in the future. >____<