Open 8cjx7 opened 4 years ago
Regarding scheduled backups, these are handled through cronjobs, which should automatically be set up if you deployed via capistrano. This is done transparently through the whenever
gem in tandem with its capistrano plugin: https://github.com/standardnotes/syncing-server/blob/7de32981dba1b2791d31cf3c5e16ed7b531d96e2/Capfile#L12
If you're not using cap production deploy
, then you'd have to manually set up the cronjobs. You can see what cronjobs you have on your system with crontab -e
(and try with sudo as well, may give different results).
This is what you should see:
# Begin Whenever generated tasks for: standard-notes-server_production at: 2020-02-27 16:35:40 +0000
0 5 * * * /bin/bash -l -c 'cd /syncing-server/releases/_release_timestamp_ && bundle exec bin/rails runner -e production '\''Tasks.perform_daily_backup_jobs'\'' >> log/whenever.log 2>&1'
Also you are correct that each client-side save made to the server will trigger a job, as you are using the revision history extension that keeps track of all saves (with internal pruning to minimize the overall number of revisions).
@mobitar thanks for you reply.
If I use cap production deploy
, do I have to have a cap.yml
file? If yes, could you please give me a sample file for syncing-server? I found a cap.example.yml
but I can't make it work.
If I use the crontab -e
command and I got log like this:
Please specify a valid ruby command or the path of a script to run.
Run 'bin/rails runner -h' for help.
uninitialized constant Rails::Command::RunnerCommand::Tasks
Did I miss some steps?
Thanks.
Yes the cap.example.yml should do it. What doesn't work in particular? Also crontab -e
is a Linux command and not a Ruby command, so you would enter it in the main console and not in the the Ruby interpreter (I'm not sure if that's your issue, but based on the output that's my guess).
Thanks.
I don't know about the cap. So I just copy cap.example.yml
to cap.yml
and I'd like to deploy on the server locally. If I run cap production deploy
, I got error:
(Backtrace restricted to imported tasks)
cap aborted!
LoadError: cannot load such file -- capistrano/shoryuken
/home/ubuntu/syncing-server/Capfile:11:in `<top (required)>'
(See full trace by running task with --trace)
Then I run bundle exec cap production deploy
, I got error:
rvm 1.29.10 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
system
#<Thread:0x00005579d653bd30 /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
9: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
8: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
7: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
6: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
5: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
4: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
3: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
2: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status=': ruby exit status: 2 (SSHKit::Command::Failed)
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as ubuntu@127.0.0.1: ruby exit status: 2 (SSHKit::Runner::ExecuteError)
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@127.0.0.1: ruby exit status: 2
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
Caused by:
SSHKit::Command::Failed: ruby exit status: 2
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
Tasks: TOP => rvm:check
(See full trace by running task with --trace)
Here is my cap.yml
:
default: &default
key_path: /home/ubuntu/key.pem
repo_url: https://github.com/standardnotes/syncing-server.git
user: ubuntu
staging:
<<: *default
server: staging.yourdomain.com
branch: staging
deploy_to: ~/standard-notes-staging-server
production:
<<: *default
server: 127.0.0.1
deploy_to: ~/standard-notes-prod-server
For the manually method (crontab), before I create a crontab command, I run the command in the terminal:
bundle exec bin/rails runner -e production 'Tasks.perform_daily_backup_jobs' >> log/whenever.log 2>&1
I got error:
Please specify a valid ruby command or the path of a script to run.
Run 'bin/rails runner -h' for help.
uninitialized constant Rails::Command::RunnerCommand::Tasks
You'll need Ruby 2.6.5 on your server. Or you can comment out this line and try again: https://github.com/standardnotes/syncing-server/blob/master/config/deploy.rb#L38
Thanks.
After I comment the line, I run bundle exec cap production deploy --trace
, I got this error:
** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke rvm:hook (first_time)
** Invoke passenger:rvm:hook (first_time)
** Invoke passenger:test_which_passenger (first_time)
** Execute passenger:test_which_passenger
** Execute passenger:rvm:hook
** Execute rvm:hook
** Invoke rvm:check (first_time)
** Execute rvm:check
rvm 1.29.10 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
system
#<Thread:0x000055b2aeabf4a8 /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
9: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
8: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
7: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
6: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
5: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
4: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
3: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
2: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status=': ruby exit status: 127 (SSHKit::Command::Failed)
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as ubuntu@127.0.0.1: ruby exit status: 127 (SSHKit::Runner::ExecuteError)
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@127.0.0.1: ruby exit status: 127
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
Caused by:
SSHKit::Command::Failed: ruby exit status: 127
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status='
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
Tasks: TOP => rvm:check
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
Hi, thanks. It turned out I didn't have rvm. After installing rvm, I run the deploy command and I got an error:
00:06 deploy:check:linked_files
ERROR linked file ~/standard-notes-prod-server/shared/.env does not exist on 127.0.0.1
Isn't the .env file created or push automatically? I already have .env file in current path.
By the way, before I run bundle exec cap production deploy
, what dependencies do I need to install on the server? Let's say I run a new AWS instance, what do I need to install and config before I can run bundle exec cap production deploy
successfully?
You'll need dependencies close to what we have here: https://docs.standardnotes.org/self-hosting/ec2-nginx
And right, on your remote server, you'll need to create a .env file in ~/standard-notes-prod-server/shared/.env where you put your production values. These values should not be the same as what you have in your local file.
Thanks @mobitar I've read the link you wrote but it doesn't cover the CloudLink Extension setup.
Back to this issue, after create .env
file on the server side, then I run the deploy command bundle exec cap production deploy
, I got this error:
00:38 passenger:restart
~/.rvm/bin/rvm default do passenger-config restart-app ~/standard-notes-prod-server --ignore-app-not-running
*** ERROR: Phusion Passenger doesn't seem to be running. If you are sure that it
is running, then the causes of this problem could be one of:
1. You customized the instance registry directory using Apache's
PassengerInstanceRegistryDir option, Nginx's
passenger_instance_registry_dir option, or Phusion Passenger Standalone's
--instance-registry-dir command line argument. If so, please set the
environment variable PASSENGER_INSTANCE_REGISTRY_DIR to that directory
and run this command again.
2. The instance directory has been removed by an operating system background
service. Please set a different instance registry directory using Apache's
PassengerInstanceRegistryDir option, Nginx's passenger_instance_registry_dir
option, or Phusion Passenger Standalone's --instance-registry-dir command
line argument.
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@52.12.161.15: passenger-config exit status: 1
I think I must miss some setups on the server side before I run the deploy command. Do you have any idea please?
This issue is a tricky one. It happens sometimes and I haven't found a good solution. The only thing that has worked for me before is restarting the instance. You can also ssh into the server and see the difference between running rvmsudo passenger-status
and just bare passenger-status
.
Hi,
I got problems with CloudLink extension and after several emails with your support team, they suggest opening an issue here. Since this is the first issue I submit in GitHub, please feel free to let me know if I miss anything.
Here is the whole thing:
After running these commands (I only changed the queue name), I got it work but with some problems.
2.1 Backup can't run automatically. Only the first backup after installing it and backups which I click manually can work well. It can't backup automatically everyday. None of Dropbox, Google Drive, OneDrive can work automatically. With the syncing-server running, it's also running command
bundle exec shoryuken -q hello -R
in the background.2.2 Too many queue messages. From March 26 to March 29, the queue created by
shoryuken
had more than 4 million messages to SQS. Please see the image, which was got on March 29. The final messages amount I checked yesterday is more than 4 million 700 thousand.At the same time, I found it in the production.log that the server will trigger more queue messages as long as I type in the macOS app. Please see the image. Almost every letter I type will trigger this log. I think it means too many queue messages were created.
2.3 Email backup failed. Email backup doesn't work. I got error like this image. Before I submit this issue, I found from other issues that it seems like I miss SMTP settings configured but which I can't find in the document on StandardNotes website.
I was also asked by your support team how I setup ActiveJob and if I switch to the built-in adapters: async or inline. I have no idea about these setup and I think I just keep the default setup. I changed nothing other than running the commands I got from your support team email.
The syncing-server code I'm running.
That's all I got.