adamcooke / staytus

💡 An open source solution for publishing the status of your services
http://staytus.co
MIT License
2.17k stars 284 forks source link

Error trying to access subscribers page #313

Closed cprofitt closed 4 years ago

cprofitt commented 4 years ago

When trying to access the subscribers page as admin I get the following error:


We're sorry, but something went wrong.

If you are the application owner check the logs for more information.

I am not sure where the find the logs the application is asking me to review.

misterjaytee commented 4 years ago

Depends on how you installed the application, but if you followed the guide at https://atech.blog/atech/install-staytus-tutorial then the log should be here:

/opt/staytus/staytus/log/production.log

cprofitt commented 4 years ago

Thank you for helping me find the logs. I did not see anything useful in the log. At this point the entire application is now reporting the same error.

Here is the last 30 lines. tail -30 production.log [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/activesupport-5.1.6.2/lib/active_support/callbacks.rb:97:in run_callbacks' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/callbacks.rb:24:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/debug_exceptions.rb:59:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/show_exceptions.rb:31:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/railties-5.1.6.2/lib/rails/rack/logger.rb:36:in call_app' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/railties-5.1.6.2/lib/rails/rack/logger.rb:24:inblock in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/activesupport-5.1.6.2/lib/active_support/tagged_logging.rb:69:in block in tagged' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/activesupport-5.1.6.2/lib/active_support/tagged_logging.rb:26:intagged' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/activesupport-5.1.6.2/lib/active_support/tagged_logging.rb:69:in tagged' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/railties-5.1.6.2/lib/rails/rack/logger.rb:24:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/remote_ip.rb:79:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/request_id.rb:25:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/rack-2.0.6/lib/rack/method_override.rb:22:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/rack-2.0.6/lib/rack/runtime.rb:22:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/activesupport-5.1.6.2/lib/active_support/cache/strategy/local_cache_middleware.rb:27:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/executor.rb:12:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/actionpack-5.1.6.2/lib/action_dispatch/middleware/static.rb:125:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/rack-2.0.6/lib/rack/sendfile.rb:111:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/railties-5.1.6.2/lib/rails/engine.rb:522:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/commonlogger.rb:38:incall' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/configuration.rb:225:in call' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/server.rb:624:inhandle_request' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/server.rb:438:in process_client' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/server.rb:302:inblock in run' [3bbf45ea-fb86-4269-b092-8d10bc2217f6] vendor/bundle/ruby/2.3.0/gems/puma-3.11.3/lib/puma/thread_pool.rb:120:in block in spawn_thread' D, [2020-02-28T08:40:14.342167 #1230] DEBUG -- : SQL (1.0ms) UPDATEdelayed_jobsSETdelayed_jobs.locked_at= '2020-02-28 13:40:14',delayed_jobs.locked_by= 'host:status pid:1230' WHERE ((run_at <= '2020-02-28 13:40:14.338941' AND (locked_at IS NULL OR locked_at < '2020-02-28 09:40:14.339052') OR locked_by = 'host:status pid:1230') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T08:40:19.346542 #1230] DEBUG -- : SQL (1.0ms) UPDATEdelayed_jobsSETdelayed_jobs.locked_at= '2020-02-28 13:40:19',delayed_jobs.locked_by= 'host:status pid:1230' WHERE ((run_at <= '2020-02-28 13:40:19.343305' AND (locked_at IS NULL OR locked_at < '2020-02-28 09:40:19.343399') OR locked_by = 'host:status pid:1230') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T08:40:24.350715 #1230] DEBUG -- : SQL (0.9ms) UPDATEdelayed_jobsSETdelayed_jobs.locked_at= '2020-02-28 13:40:24',delayed_jobs.locked_by= 'host:status pid:1230' WHERE ((run_at <= '2020-02-28 13:40:24.347560' AND (locked_at IS NULL OR locked_at < '2020-02-28 09:40:24.347633') OR locked_by = 'host:status pid:1230') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T08:40:29.355221 #1230] DEBUG -- : SQL (1.0ms) UPDATEdelayed_jobsSETdelayed_jobs.locked_at= '2020-02-28 13:40:29',delayed_jobs.locked_by= 'host:status pid:1230' WHERE ((run_at <= '2020-02-28 13:40:29.352021' AND (locked_at IS NULL OR locked_at < '2020-02-28 09:40:29.352102') OR locked_by = 'host:status pid:1230') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T08:40:34.359655 #1230] DEBUG -- : SQL (1.0ms) UPDATEdelayed_jobsSETdelayed_jobs.locked_at= '2020-02-28 13:40:34',delayed_jobs.locked_by` = 'host:status pid:1230' WHERE ((run_at <= '2020-02-28 13:40:34.356313' AND (locked_at IS NULL OR locked_at < '2020-02-28 09:40:34.356399') OR locked_by = 'host:status pid:1230') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1

Any ideas?

misterjaytee commented 4 years ago

Can't really tell from the log, but I'd probably restart the app first if you haven't done so already:

sudo -u staytus procodile stop --root /opt/staytus/staytus
sudo -u staytus procodile start --root /opt/staytus/staytus
cprofitt commented 4 years ago

That did not help... is there any chance this is an issue with the mysql database?

cprofitt commented 4 years ago

After entering the commands you suggested the tail -30 of the logs is as follows:

tail -30 production.log D, [2020-02-28T23:45:56.887050 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Site Load (0.8ms) SELECT sites. FROM sites ORDER BY sites.id ASC LIMIT 1 D, [2020-02-28T23:45:56.889772 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Service Load (0.8ms) SELECT services. FROM services ORDER BY services.position ASC D, [2020-02-28T23:45:56.894081 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] ServiceGroup Load (0.7ms) SELECT service_groups. FROM service_groups WHERE service_groups.id IN (1, 2, 3) D, [2020-02-28T23:45:56.897441 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] ServiceStatus Load (0.6ms) SELECT service_statuses. FROM service_statuses WHERE service_statuses.id = 1 D, [2020-02-28T23:45:56.905112 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] SQL (0.9ms) SELECT maintenance_service_joins.id AS t0_r0, maintenance_service_joins.maintenance_id AS t0_r1, maintenance_service_joins.service_id AS t0_r2, maintenance_service_joins.created_at AS t0_r3, maintenance_service_joins.updated_at AS t0_r4, maintenances.id AS t1_r0, maintenances.title AS t1_r1, maintenances.description AS t1_r2, maintenances.start_at AS t1_r3, maintenances.finish_at AS t1_r4, maintenances.length_in_minutes AS t1_r5, maintenances.user_id AS t1_r6, maintenances.service_status_id AS t1_r7, maintenances.created_at AS t1_r8, maintenances.updated_at AS t1_r9, maintenances.closed_at AS t1_r10, maintenances.identifier AS t1_r11, maintenances.notify AS t1_r12 FROM maintenance_service_joins LEFT OUTER JOIN maintenances ON maintenances.id = maintenance_service_joins.maintenance_id WHERE maintenances.closed_at IS NULL AND (start_at <= '2020-02-29 04:45:56.899558') AND maintenance_service_joins.service_id IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15) D, [2020-02-28T23:45:56.909928 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Issue Load (0.6ms) SELECT issues. FROM issues WHERE (issues.state != 'resolved') ORDER BY issues.id DESC D, [2020-02-28T23:45:56.911704 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Maintenance Load (0.6ms) SELECT maintenances. FROM maintenances WHERE maintenances.closed_at IS NULL ORDER BY maintenances.start_at ASC I, [2020-02-28T23:45:56.914507 #4728] INFO -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Rendering content/themes/default/views/pages/index.html.erb within layouts/default D, [2020-02-28T23:45:56.921169 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Nifty::Attachments::Attachment Load (0.7ms) SELECT nifty_attachments.id, nifty_attachments.token, nifty_attachments.digest, nifty_attachments.parent_id, nifty_attachments.parent_type, nifty_attachments.file_name, nifty_attachments.file_type FROM nifty_attachments WHERE nifty_attachments.parent_id = 1 AND nifty_attachments.parent_type = 'Site' AND nifty_attachments.role = 'cover_image' LIMIT 1 D, [2020-02-28T23:45:56.924354 #4728] DEBUG -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Nifty::Attachments::Attachment Load (0.7ms) SELECT nifty_attachments.id, nifty_attachments.token, nifty_attachments.digest, nifty_attachments.parent_id, nifty_attachments.parent_type, nifty_attachments.file_name, nifty_attachments.file_type FROM nifty_attachments WHERE nifty_attachments.parent_id = 1 AND nifty_attachments.parent_type = 'Site' AND nifty_attachments.role = 'logo' LIMIT 1 I, [2020-02-28T23:45:56.963332 #4728] INFO -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Rendered content/themes/default/views/pages/index.html.erb within layouts/default (48.6ms) I, [2020-02-28T23:45:57.011591 #4728] INFO -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] Completed 500 Internal Server Error in 127ms (ActiveRecord: 6.4ms) F, [2020-02-28T23:45:57.012989 #4728] FATAL -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6]
F, [2020-02-28T23:45:57.013105 #4728] FATAL -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] ActionView::Template::Error (undefined method privacy_policy_url' for #<Site:0x000000068e5cc8>): F, [2020-02-28T23:45:57.013377 #4728] FATAL -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 22: <ul class='siteFooter__nav'> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 23: <li><a href='/history'><%= t("themes.default.footer.history", :default => 'History') %></a></li> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 24: <li><a href='<%= site.website_url %>'><%= t("themes.default.footer.homepage", :default => 'Our Homepage') %></a></li> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 25: <% if site.privacy_policy_url.present? %> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 26: <li><a href='<%= site.privacy_policy_url %>'><%= t("themes.default.footer.privacy_policy", :default => 'Privacy Policy') %></a></li> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 27: <% end %> [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] 28: <li><a href='mailto:<%= site.support_email %>'><%= t("themes.default.footer.contact_us", :default => 'Contact Us') %></a></li> F, [2020-02-28T23:45:57.013465 #4728] FATAL -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] F, [2020-02-28T23:45:57.013552 #4728] FATAL -- : [5d428a60-e1d6-4ee0-8c48-f7af24b81fb6] app/controllers/application_controller.rb:20:inset_time_zone' D, [2020-02-28T23:45:59.092583 #4730] DEBUG -- : SQL (1.0ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:45:59', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:45:59.089464' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:45:59.089534') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:04.096270 #4730] DEBUG -- : SQL (1.0ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:04', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:04.093060' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:04.093135') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:09.100338 #4730] DEBUG -- : SQL (1.1ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:09', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:09.097270' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:09.097346') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:14.104624 #4730] DEBUG -- : SQL (1.0ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:14', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:14.101487' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:14.101563') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:19.108632 #4730] DEBUG -- : SQL (1.0ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:19', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:19.105594' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:19.105668') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:24.114362 #4730] DEBUG -- : SQL (1.4ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:24', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:24.109673' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:24.109750') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1 D, [2020-02-28T23:46:29.118499 #4730] DEBUG -- : SQL (0.9ms) UPDATE delayed_jobs SET delayed_jobs.locked_at = '2020-02-29 04:46:29', delayed_jobs.locked_by = 'host:status pid:4730' WHERE ((run_at <= '2020-02-29 04:46:29.115519' AND (locked_at IS NULL OR locked_at < '2020-02-29 00:46:29.115594') OR locked_by = 'host:status pid:4730') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1

cprofitt commented 4 years ago

Also, is there a way to backup the DB and reinstall then attach the DB?

misterjaytee commented 4 years ago

The key part for me (which wasn't in the previous log) is:

(undefined method privacy_policy_url' for #)

I haven't been using staytus long myself, but I haven't seen that part of the code blow-up except for when I've modified the default view (/opt/staytus/staytus/content/themes/default/views/layouts/default.html.erb).

I guess the question now is, if this was a working site then what changed? Did you update any Linux components, or did you try modifying one of the files for staytus? Maybe look at your bash history for clues: history | less

If you have edited any files, go to your backup and restore them (you did make one before making changes didn't you? ;) ). If you changed something and don't have a backup, then get the file from the github repo.

Personally, from the limited info we have above, I don't think it is a database issue (unless you modified the database). You should really have a backup of the database already, but a simple way is this:

mkdir -p /backups/database

mysqldump --user=root --password --host=localhost --databases staytus > /backups/database/$HOSTNAME-staytus-$(date +"%Y-%m-%d-%H-%M").sql

To restore after a fresh install (make sure it all works first and maybe do a backup of the new database just in case!):

mysql -u root -p
drop database staytus;
create database staytus;
quit;

mysql -u root -p staytus < /backups/database/name-of-your-backup-file.sql

Good luck!

cprofitt commented 4 years ago

I backed up the DB, did a fresh install -- entire server -- and tested. The application worked. Restored the DB and then got the error. While I was able to read my users, incidents, etc from the file something else must have been an issue. I am going to rebuild the information and continue... but it does appear the DB got corrupted.

misterjaytee commented 4 years ago

Ouch! That's unlucky. Once you're back up and running, I'd suggest you write a script to incorporate the mysqldump command and then run it as a cron job as regularly as fits your use case.

cprofitt commented 4 years ago

I looked in the DB again and I wonder if this is an issue with my background image. All the other tables made sense... and the background image is the one file that would have affected all the pages.

cprofitt commented 4 years ago

It was not the background image. I am not sure what in the DB was the issue. Site is back up... reconstructed from the old one.