standardnotes / syncing-server

[Deprecated: Use our new Node server: https://github.com/standardnotes/syncing-server-js]
https://standardnotes.org
GNU Affero General Public License v3.0
209 stars 48 forks source link

CloudLink extension doesn't work well #46

Open 8cjx7 opened 4 years ago

8cjx7 commented 4 years ago

Hi,

I got problems with CloudLink extension and after several emails with your support team, they suggest opening an issue here. Since this is the first issue I submit in GitHub, please feel free to let me know if I miss anything.

Here is the whole thing:

  1. I run a syncing-server on AWS LightSail and install CloudLink extension on macOs app according to https://docs.standardnotes.org/self-hosting/ec2-nginx . The backup feature doesn't work. After I contact the support team, I was told that I need to setup AWS SQS on my syncing-server. And they gave me some guides:
1. Set up your AWS credentials (see here: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html#aws-ruby-sdk-setting-credentials)
2. Run bundle exec shoryuken sqs create dev_queue
3. Update SQS_QUEUE and AWS_REGION on your .env file
4. Start shoryuken with: bundle exec shoryuken -q hello -R
  1. After running these commands (I only changed the queue name), I got it work but with some problems.

    2.1 Backup can't run automatically. Only the first backup after installing it and backups which I click manually can work well. It can't backup automatically everyday. None of Dropbox, Google Drive, OneDrive can work automatically. With the syncing-server running, it's also running command bundle exec shoryuken -q hello -R in the background.

    2.2 Too many queue messages. From March 26 to March 29, the queue created by shoryuken had more than 4 million messages to SQS. Please see the image, which was got on March 29. The final messages amount I checked yesterday is more than 4 million 700 thousand.

    image

    At the same time, I found it in the production.log that the server will trigger more queue messages as long as I type in the macOS app. Please see the image. Almost every letter I type will trigger this log. I think it means too many queue messages were created.

78072949-cce18400-7354-11ea-8aab-c17816d3e5ce

2.3 Email backup failed. Email backup doesn't work. I got error like this image. Before I submit this issue, I found from other issues that it seems like I miss SMTP settings configured but which I can't find in the document on StandardNotes website.

[ERROR] Error performing ActionMailer::DeliveryJob (Job ID: 7a6239eb-89a8-4835-9d20-5d9c8bd78cb9) from Shoryuken(cloud_backup_queue) in 79.13ms: Errno::ECONNREFUSED (Connection refused - connect(2) for nil port 25):
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `initialize'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `open'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `tcp_socket'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:549:in `block in do_start'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/timeout.rb:95:in `block in timeout'
  1. I was also asked by your support team how I setup ActiveJob and if I switch to the built-in adapters: async or inline. I have no idea about these setup and I think I just keep the default setup. I changed nothing other than running the commands I got from your support team email.

  2. The syncing-server code I'm running.

    image

That's all I got.

moughxyz commented 4 years ago

Regarding scheduled backups, these are handled through cronjobs, which should automatically be set up if you deployed via capistrano. This is done transparently through the whenever gem in tandem with its capistrano plugin: https://github.com/standardnotes/syncing-server/blob/7de32981dba1b2791d31cf3c5e16ed7b531d96e2/Capfile#L12

If you're not using cap production deploy, then you'd have to manually set up the cronjobs. You can see what cronjobs you have on your system with crontab -e (and try with sudo as well, may give different results).

This is what you should see:

# Begin Whenever generated tasks for: standard-notes-server_production at: 2020-02-27 16:35:40 +0000
0 5 * * * /bin/bash -l -c 'cd /syncing-server/releases/_release_timestamp_ && bundle exec bin/rails runner -e production '\''Tasks.perform_daily_backup_jobs'\'' >> log/whenever.log 2>&1'

Also you are correct that each client-side save made to the server will trigger a job, as you are using the revision history extension that keeps track of all saves (with internal pruning to minimize the overall number of revisions).

8cjx7 commented 4 years ago

@mobitar thanks for you reply.

If I use cap production deploy, do I have to have a cap.yml file? If yes, could you please give me a sample file for syncing-server? I found a cap.example.yml but I can't make it work.

If I use the crontab -e command and I got log like this:

Please specify a valid ruby command or the path of a script to run.
Run 'bin/rails runner -h' for help.

uninitialized constant Rails::Command::RunnerCommand::Tasks

Did I miss some steps?

Thanks.

moughxyz commented 4 years ago

Yes the cap.example.yml should do it. What doesn't work in particular? Also crontab -e is a Linux command and not a Ruby command, so you would enter it in the main console and not in the the Ruby interpreter (I'm not sure if that's your issue, but based on the output that's my guess).

8cjx7 commented 4 years ago

Thanks. I don't know about the cap. So I just copy cap.example.yml to cap.yml and I'd like to deploy on the server locally. If I run cap production deploy, I got error:

(Backtrace restricted to imported tasks)
cap aborted!
LoadError: cannot load such file -- capistrano/shoryuken
/home/ubuntu/syncing-server/Capfile:11:in `<top (required)>'
(See full trace by running task with --trace)

Then I run bundle exec cap production deploy, I got error:

rvm 1.29.10 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
system
#<Thread:0x00005579d653bd30 /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
    9: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
    8: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
    7: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
    6: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
    5: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
    4: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
    3: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
    2: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
    1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status=': ruby exit status: 2 (SSHKit::Command::Failed)
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
    1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as ubuntu@127.0.0.1: ruby exit status: 2 (SSHKit::Runner::ExecuteError)
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@127.0.0.1: ruby exit status: 2
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.

Caused by:
SSHKit::Command::Failed: ruby exit status: 2
ruby stdout: Nothing written
ruby stderr: Ruby ruby-2.6.5 is not installed.

Tasks: TOP => rvm:check
(See full trace by running task with --trace)

Here is my cap.yml:

default: &default
  key_path: /home/ubuntu/key.pem
  repo_url: https://github.com/standardnotes/syncing-server.git
  user: ubuntu

staging:
  <<: *default
  server: staging.yourdomain.com
  branch: staging
  deploy_to: ~/standard-notes-staging-server

production:
  <<: *default
  server: 127.0.0.1
  deploy_to: ~/standard-notes-prod-server

For the manually method (crontab), before I create a crontab command, I run the command in the terminal:

bundle exec bin/rails runner -e production 'Tasks.perform_daily_backup_jobs' >> log/whenever.log 2>&1

I got error:

Please specify a valid ruby command or the path of a script to run.
Run 'bin/rails runner -h' for help.

uninitialized constant Rails::Command::RunnerCommand::Tasks
moughxyz commented 4 years ago

You'll need Ruby 2.6.5 on your server. Or you can comment out this line and try again: https://github.com/standardnotes/syncing-server/blob/master/config/deploy.rb#L38

8cjx7 commented 4 years ago

Thanks. After I comment the line, I run bundle exec cap production deploy --trace, I got this error:

** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke rvm:hook (first_time)
** Invoke passenger:rvm:hook (first_time)
** Invoke passenger:test_which_passenger (first_time)
** Execute passenger:test_which_passenger
** Execute passenger:rvm:hook
** Execute rvm:hook
** Invoke rvm:check (first_time)
** Execute rvm:check
rvm 1.29.10 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
system
#<Thread:0x000055b2aeabf4a8 /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
    9: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
    8: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
    7: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
    6: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
    5: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
    4: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
    3: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
    2: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
    1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status=': ruby exit status: 127 (SSHKit::Command::Failed)
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
    1: from /home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as ubuntu@127.0.0.1: ruby exit status: 127 (SSHKit::Runner::ExecuteError)
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@127.0.0.1: ruby exit status: 127
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'

Caused by:
SSHKit::Command::Failed: ruby exit status: 127
ruby stdout: Nothing written
ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status='
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:66:in `capture'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capistrano-rvm-0.1.2/lib/capistrano/tasks/rvm.rake:11:in `block (3 levels) in <top (required)>'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
Tasks: TOP => rvm:check
moughxyz commented 4 years ago

ruby stderr: /home/ubuntu/.rvm/scripts/set: line 19: exec: ruby: not found

8cjx7 commented 4 years ago

Hi, thanks. It turned out I didn't have rvm. After installing rvm, I run the deploy command and I got an error:

00:06 deploy:check:linked_files
      ERROR linked file ~/standard-notes-prod-server/shared/.env does not exist on 127.0.0.1

Isn't the .env file created or push automatically? I already have .env file in current path.

By the way, before I run bundle exec cap production deploy, what dependencies do I need to install on the server? Let's say I run a new AWS instance, what do I need to install and config before I can run bundle exec cap production deploy successfully?

moughxyz commented 4 years ago

You'll need dependencies close to what we have here: https://docs.standardnotes.org/self-hosting/ec2-nginx

And right, on your remote server, you'll need to create a .env file in ~/standard-notes-prod-server/shared/.env where you put your production values. These values should not be the same as what you have in your local file.

8cjx7 commented 4 years ago

Thanks @mobitar I've read the link you wrote but it doesn't cover the CloudLink Extension setup.

Back to this issue, after create .env file on the server side, then I run the deploy command bundle exec cap production deploy, I got this error:

00:38 passenger:restart
      ~/.rvm/bin/rvm default do passenger-config restart-app ~/standard-notes-prod-server --ignore-app-not-running
      *** ERROR: Phusion Passenger doesn't seem to be running. If you are sure that it
      is running, then the causes of this problem could be one of:

       1. You customized the instance registry directory using Apache's
          PassengerInstanceRegistryDir option, Nginx's
          passenger_instance_registry_dir option, or Phusion Passenger Standalone's
          --instance-registry-dir command line argument. If so, please set the
          environment variable PASSENGER_INSTANCE_REGISTRY_DIR to that directory
          and run this command again.
       2. The instance directory has been removed by an operating system background
          service. Please set a different instance registry directory using Apache's
          PassengerInstanceRegistryDir option, Nginx's passenger_instance_registry_dir
          option, or Phusion Passenger Standalone's --instance-registry-dir command
          line argument.
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as ubuntu@52.12.161.15: passenger-config exit status: 1

I think I must miss some setups on the server side before I run the deploy command. Do you have any idea please?

moughxyz commented 4 years ago

This issue is a tricky one. It happens sometimes and I haven't found a good solution. The only thing that has worked for me before is restarting the instance. You can also ssh into the server and see the difference between running rvmsudo passenger-status and just bare passenger-status.