search

syncloud / platform

Run popular services on your device with one click
https://syncloud.org
GNU General Public License v3.0
400 stars 40 forks source link

storage - raid array not visible (helios) #433

Closed sative closed 4 years ago

sative commented 4 years ago

Hi synclouders :-)

I followed the instructions for setting up a new syncloud - https://wiki.kobol.io/syncloud/

Before installing any apps, i wanted to configure my storage, so i did this https://wiki.kobol.io/mdadm/#create-raid-10-array and https://wiki.kobol.io/mdadm/#create-and-mount-the-filesystem

The new RAID-array seems configured correct:

Bildschirmfoto_2019-12-08_21-24-01

But on the Website, i'm missing the mount-point for the array.

Screenshot_2019-12-08 Syncloud

g-provost commented 4 years ago

@sative Unfortunately it looks like a limitation in Syncloud implementation, where the web backend is designed to only add block device one by one, not already configured partition / mount point that could be exposing a raid array.

@cyberb Is there a work around for this user to add manually in one the syncloud config files the raid array he created as an External storage ?

cyberb commented 4 years ago

@sative Currentrly we are usong thia command to detect devices, could you send me the output:

lsblk -Pp -o NAME,SIZE,TYPE,MOUNTPOINT,PARTTYPE,FSTYPE,MODEL
g-provost commented 4 years ago

I have a similar setup running, here what this command would return 

NAME="/dev/sda" SIZE="1.8T" TYPE="disk" MOUNTPOINT="" PARTTYPE="" FSTYPE="linux_raid_member" MODEL="WDC WD20EFRX-68E"
NAME="/dev/sdb" SIZE="1.8T" TYPE="disk" MOUNTPOINT="" PARTTYPE="" FSTYPE="linux_raid_member" MODEL="WDC WD20EFRX-68E"
NAME="/dev/sdc" SIZE="1.8T" TYPE="disk" MOUNTPOINT="" PARTTYPE="" FSTYPE="linux_raid_member" MODEL="WDC WD20EFRX-68E"
NAME="/dev/sdd" SIZE="1.8T" TYPE="disk" MOUNTPOINT="" PARTTYPE="" FSTYPE="linux_raid_member" MODEL="WDC WD20EFRX-68E"
NAME="/dev/md0" SIZE="3.7T" TYPE="raid10" MOUNTPOINT="/mnt/md0" PARTTYPE="" FSTYPE="ext4" MODEL=""
NAME="/dev/md0" SIZE="3.7T" TYPE="raid10" MOUNTPOINT="/mnt/md0" PARTTYPE="" FSTYPE="ext4" MODEL=""
NAME="/dev/md0" SIZE="3.7T" TYPE="raid10" MOUNTPOINT="/mnt/md0" PARTTYPE="" FSTYPE="ext4" MODEL=""
NAME="/dev/md0" SIZE="3.7T" TYPE="raid10" MOUNTPOINT="/mnt/md0" PARTTYPE="" FSTYPE="ext4" MODEL=""
NAME="/dev/mmcblk0" SIZE="14.9G" TYPE="disk" MOUNTPOINT="" PARTTYPE="" FSTYPE="" MODEL=""
NAME="/dev/mmcblk0p1" SIZE="14.7G" TYPE="part" MOUNTPOINT="/" PARTTYPE="0x83" FSTYPE="ext4" MODEL=""
NAME="/dev/zram0" SIZE="50M" TYPE="disk" MOUNTPOINT="/var/log" PARTTYPE="" FSTYPE="" MODEL=""
NAME="/dev/zram1" SIZE="1008M" TYPE="disk" MOUNTPOINT="[SWAP]" PARTTYPE="" FSTYPE="" MODEL=""
sative commented 4 years ago

Hey you two! :-)

Here is the output:

Bildschirmfoto_2019-12-11_21-15-26

cyberb commented 4 years ago

Could you run two more commands as I need to understand what is the best format: With merge:

lsblk -JMp -o NAME,SIZE,TYPE,MOUNTPOINT,PARTTYPE,FSTYPE,MODEL

Without merge:

lsblk -Jp -o NAME,SIZE,TYPE,MOUNTPOINT,PARTTYPE,FSTYPE,MODEL
sative commented 4 years ago

Yes, sure!

But -J and -M are invalid options for lsblk. Did you mean -m?

Bildschirmfoto_2019-12-13_22-33-15

cyberb commented 4 years ago

I see, I actually J and M but they are not available in Syncloud yet. I have a test version for you to try, could you upgrade platform using this command and check what storage page shows?

snap refresh platform --channel=master
sative commented 4 years ago

root@syncloud:~# snap refresh platform --channel=master error: cannot perform the following tasks:

Wrong filename?

cyberb commented 4 years ago

Sorry new version was being uploaded, try again now.

sative commented 4 years ago

Bildschirmfoto_2019-12-15_10-56-40

:-))

Should I format it now?

cyberb commented 4 years ago

Sorry, did not see the reply. Try to unmount it first and activate here as it is already ext4 formatted.

cyberb commented 4 years ago

@sative did it work?

sative commented 4 years ago

I couldn't do either "Activate" or "Format", but I could install the Nextcloud app. The app also placed its data on the RAID array. Should I have unmounted and formatted before? (I couldn't wait... :-)

But now I have a new problem. The device was unreachable a few hours after the action and I had to pull the plug. Now after booting, I get the Syncloud.it activation website again and I can't log in anymore.

cyberb commented 4 years ago

Probably your public IP changed, are you using domain name to access device? Did you enable external access? Try devuce ip and check network page.

As to RAID, you should not try to format using Syncloud as it is not supported yet. You should prepare (ext4) raid disk as you did and NOT mount it. Then in Syncloud you can try to Activate it and send me logs from settings if it does not work.

Syncloud RAID disk creation and formatting will come later so you do not have to use terminal at all.

cyberb commented 4 years ago

@sative did you recover?

sative commented 4 years ago

@cyberb Sorry I answered so late. About your questions...

I didn't activate my device with a syncloud account, but chose the way of customizing the host file and initially worked without external access.

The state of the device is unchanged. As I said, I can only access the device on port 81 or the activation page. I think that something must have gone wrong here, because I had already activated the device.
Should I now perform the activation again?

Thanks for the information regarding RAID.

P.S. I have just switched on the device and I am curious if it runs stable or if it will leave after some hours :-/

cyberb commented 4 years ago

Strange that device became not activated. Did you change your boot disk? Can you post the output of:

lsblk

Also you can activate but I would try to understand what happened with existing activation.

sative commented 4 years ago

I wouldn't know I changed the boot drive. How could this have happened?

Here is the output: Bildschirmfoto_2020-01-26_12-02-19

When I try to activate it again, I now get this message: Bildschirmfoto_2020-01-26_12-06-37

cyberb commented 4 years ago

Can you push this send logs button so I can see the exact error message?

sative commented 4 years ago

When I press the button, I am asked to enter my account information for syncloud.it I then enter the data, press "Send Logs" again, but I get an error... Bildschirmfoto_2020-01-27_21-10-18

cyberb commented 4 years ago

Could try again and post the output of these commands:

tail -100 /var/snap/platform/common/log/uwsgi_internal.log

And this:

tail -100 /var/snap/platform/common/log/uwsgi_public.log
sative commented 4 years ago

root@syncloud:~# tail -100 /var/snap/platform/common/log/uwsgi_internal.log

Traceback (most recent call last):
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/syncloud_platform-191214121-py2.7.egg/syncloud_platform/rest/internal.py", line 64, in activate_custom_domain
    device_password,
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/syncloud_platform-191214121-py2.7.egg/syncloud_platform/rest/facade/internal.py", line 22, in activate_custom_domain
    full_domain, device_username, device_password)
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/syncloud_platform-191214121-py2.7.egg/syncloud_platform/device.py", line 67, in activate_custom_domain
    self._activate_common(name, device_username, device_password, email)
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/syncloud_platform-191214121-py2.7.egg/syncloud_platform/device.py", line 79, in _activate_common
    self.tls.generate_self_signed_certificate()
  File "/snap/platform/191214121/python/lib/python2.7/site-packages/syncloud_platform-191214121-py2.7.egg/syncloud_platform/certificate/certificate_generator.py", line 136, in generate_self_signed_certificate
    raise e
CalledProcessError: Command 'OPENSSL_CONF=/tmp/tmpAvg8UB /snap/platform/191214121/openssl/bin/openssl genrsa -out /var/snap/platform/common/syncloud.key 4096 2>&1' returned non-zero exit status 1
sative commented 4 years ago

Sorry for the formatting, but i don't get the problem here... :-)

cyberb commented 4 years ago

Have you already activated this device once? Looks like you have a real cert there.

ls -la /var/snap/platform/common/syncloud.key

Reflash or removal of this file should help.

cyberb commented 4 years ago

I have just pushed a fix for this situation. Could your run this command:

snap refresh platform

And try to activate again?

sative commented 4 years ago

More trouble, I'm afraid... Maybe I should just do a fresh install?

root@syncloud:~# snap refresh platform error: cannot perform the following tasks:

cyberb commented 4 years ago

Could you try again please, there was a broken version? It should download version 200130176

sative commented 4 years ago

Result:

root@syncloud:~# snap refresh platform error: cannot perform the following tasks:

cyberb commented 4 years ago

Ok, fresh install should help here.

sative commented 4 years ago

OK, I will. Thanks for your patience :-)

sative commented 4 years ago

So, now I did a fresh install and activation (with syncloud-account) via webinterface. Now I'm executing "snap refresh platform" again, right?

cyberb commented 4 years ago

Yes

rgururaj commented 4 years ago

I tried with the latest version of helios4 image (20.6) still not able to activate the raid array.

root@syncloud:~# tail -100 /var/snap/platform/common/log/uwsgi_public.log

Traceback (most recent call last):
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/syncloud_platform-200609406-py3.7.egg/syncloud_platform/rest/flask_decorators.py", line 20, in new_func
    return make_response(f(*args, **kwargs))
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/flask_login.py", line 758, in decorated_view
    return func(*args, **kwargs)
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/syncloud_platform-200609406-py3.7.egg/syncloud_platform/rest/public.py", line 283, in disk_activate
    return jsonify(success=True, disks=public.disk_activate(request.args['device'])), 200
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/syncloud_platform-200609406-py3.7.egg/syncloud_platform/rest/facade/public.py", line 85, in disk_activate
    return self.hardware.activate_disk(device)
  File "/snap/platform/200609406/python/lib/python3.7/site-packages/syncloud_platform-200609406-py3.7.egg/syncloud_platform/disks/hardware.py", line 73, in activate_disk
    raise ServiceException(error_message)
syncloud_platform.rest.service_exception.ServiceException
rgururaj commented 4 years ago

I have two disks on raid 1. image

rgururaj commented 4 years ago

with latest refresh: "snap refresh platform --channel=master" below are logs

root@syncloud:~# tail -100 /var/snap/platform/common/log/uwsgi_public.log

Traceback (most recent call last):
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/syncloud_platform-200615412-py3.7.egg/syncloud_platform/rest/flask_decorators.py", line 20, in new_func
    return make_response(f(*args, **kwargs))
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/flask_login.py", line 758, in decorated_view
    return func(*args, **kwargs)
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/syncloud_platform-200615412-py3.7.egg/syncloud_platform/rest/public.py", line 283, in disk_activate
    return jsonify(success=True, disks=public.disk_activate(request.args['device'])), 200
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/syncloud_platform-200615412-py3.7.egg/syncloud_platform/rest/facade/public.py", line 85, in disk_activate
    return self.hardware.activate_disk(device)
  File "/snap/platform/200615412/python/lib/python3.7/site-packages/syncloud_platform-200615412-py3.7.egg/syncloud_platform/disks/hardware.py", line 73, in activate_disk
    raise ServiceException(error_message)
syncloud_platform.rest.service_exception.ServiceException
cyberb commented 4 years ago

Did you mount this device manually from command line before pushing activate button? (you should not)

rgururaj commented 4 years ago

No I haven't tried mounting from command line. I only tried it from the GUI and always get "Server error".

cyberb commented 4 years ago

Could you send logs from settings - support and check include support?

rgururaj commented 4 years ago

Sent the logs

image
cyberb commented 4 years ago

Could you update and try again;

snap refresh platform --channel=master
rgururaj commented 4 years ago

Awesome, that worked like a charm image

image

cyberb commented 4 years ago

Great, let me close this one, feel free to open a new one for a raid creation (which is more complex).