search

apache / cloudstack

Apache CloudStack is an opensource Infrastructure as a Service (IaaS) cloud computing platform
https://cloudstack.apache.org/
Apache License 2.0
1.85k stars 1.07k forks source link

Allow multiple localstorage based on VolumeGroup #4391

Open hstan77 opened 3 years ago

hstan77 commented 3 years ago
ISSUE TYPE
COMPONENT NAME
CLOUDSTACK VERSION
Cloudstack 4.14
CONFIGURATION
OS / ENVIRONMENT
SUMMARY

Lack of preference storage type for Cloudstack, or less option for proper storage

Current SharedMountPoint idea can use for VolumGroup and ISCSI, but the system expecting to have Clustered File System for multiple host in the same cluster , in order to use the Same SharedMountPoint. So for those that wish to wish to use the shared mount point, it has to be create many cluster with single host.

Other like LocalStorage is only support File-based which a lot reduce the efficiency of VM.

It shall be able to make it a lot more efficient with ISCSI / VolumeGroup as primary storage. Administrator expecting know how to mount iscsi or create volumegroup before add Primary Storage.

With this way, it shall be make the Cloudstack more Efficient on Creating VM with same hardware budget .

STEPS TO REPRODUCE
EXPECTED RESULTS

Allow adding multiple local storage based on VolumeGroup.
ACTUAL RESULTS
DaanHoogland commented 3 years ago

@hstan77 sorry to ask but I don't think I'm clear on what you are asking here. Are you speaking of volumegroups or clustered deistributed filesystems?

hstan77 commented 3 years ago

Hi Dahn,

I means is the Local VolumeGroup , as alternative of LocalStorage that file based.

By the way, I can't find a good way of implementing clustered deistributed filesystems also . Now I am trying on the Ceph and NFS .

On Wed, Nov 25, 2020 at 4:05 AM dahn notifications@github.com wrote:

@hstan77 https://github.com/hstan77 sorry to ask but I don't think I'm clear on what you are asking here. Are you speaking of volumegroups or clustered deistributed filesystems?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/cloudstack/issues/4391#issuecomment-733204287, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARFNGASE2LAPZNZ43AGXETDSRQGYZANCNFSM4SJSK5RQ .

-- Regards, Hean Seng

andrijapanicsb commented 3 years ago

@hstan77 I'm not sure if I'm reading your explanation correctly or not. The existing ShareMountPoint means you would need to attach the same LUN (iSCSI or FC, etc) to all the hosts in the cluster and then format this LUN with a clustered filesystem so that multiple hosts can write to the same shared(mountPoint) storage without causing file system corruption.

How do you see (this is what I don't understand) your storage solution proposal being different - i.e. what is missing that you would like to see?

DaanHoogland commented 3 years ago

@hstan77 I'm afraid this issue is still not well defined. Do you have an update?

hstan77 commented 3 years ago

Hi

I understand Andrija said . Just I hope there is opton of LVM as localstorage insteaod of file based.

On Fri, Jan 29, 2021 at 9:01 PM dahn notifications@github.com wrote:

@hstan77 https://github.com/hstan77 I'm afraid this issue is still not well defined. Do you have an update?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/cloudstack/issues/4391#issuecomment-769791713, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARFNGARNWUMTYT3C7HO5K7DS4KWUDANCNFSM4SJSK5RQ .

-- Regards, Hean Seng

DaanHoogland commented 3 years ago

@hstan77 local is local filesystem based not file based any LGLVM that has a filesystem on it could be used. How filesystems are mounted locally on the hypervisor would not really matter to ACS would it? So i am still wondering what exactly you mean.

hstan77 commented 3 years ago

Currently for shared mount point , it need to be run on Clustered Filesystem, such as GFS2 or ocfs2 , OCFS2 is seems support till centos 6 only . GFS2 has not easy to maintain and seems many issue for implementing it .

Thats I am asking of possible support LVM base local storage .

On Fri, Jan 29, 2021 at 10:13 PM dahn notifications@github.com wrote:

@hstan77 https://github.com/hstan77 local is local filesystem based not file based any LGLVM that has a filesystem on it could be used. How filesystems are mounted locally on the hypervisor would not really matter to ACS would it? So i am still wondering what exactly you mean.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/cloudstack/issues/4391#issuecomment-769828792, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARFNGASKH3MORWSC4OJK3ETS4K7BLANCNFSM4SJSK5RQ .

-- Regards, Hean Seng

xuanyuanaosheng commented 2 years ago

@hstan77 Any update ? which storage solution do you use?

Related issue: https://github.com/apache/cloudstack/issues/6105

xuanyuanaosheng commented 2 years ago

I'm not sure if I'm reading your explanation correctly or not. The existing ShareMountPoint means you would need to attach the same LUN (iSCSI or FC, etc) to all the hosts in the cluster and then format this LUN with a clustered filesystem so that multiple hosts can write to the same shared(mountPoint) storage without causing file system corruption.

Which clustered filesystem do you recommend?

andrijapanicsb commented 2 years ago

Nothing is even close to "good enough" for production use.

xuanyuanaosheng commented 2 years ago

@andrijapanicsb How to add SAN storage like HP3PAR or HITACHI as cloudstack storage?

andrijapanicsb commented 2 years ago

Clustered File System is required on top of SAN/LUN (whatever the storage in the background). None of those clustered file systems is something I would have an easy time recommending.

xuanyuanaosheng commented 2 years ago

Clustered File System is required on top of SAN/LUN (whatever the storage in the background). None of those clustered file systems is something I would have an easy time recommending.

@andrijapanicsb @DaanHoogland I have searched all the doc and goole a lot.

  1. https://serverfault.com/questions/734364/add-shared-mountpoint-storage-in-cloudstack
  2. https://www.mail-archive.com/dev@cloudstack.apache.org/msg94600.html

But I still can not find a good way how to add a SAN/LUN storage like HP3PAR or HITACHI on the cloudstack, Could you please give some docs ?

As I know we can assigned the same 5T storage to these two computer node , our env:

The computer node1:

# multipath -ll
mpatham (360060e801262eb00504062eb000000ea) dm-26 HITACHI,OPEN-V
size=5.0T features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  |- 1:0:8:1  sdbd 67:112 active ready running
  `- 2:0:10:1 sdbe 67:128 active ready running

The computer node2:

# multipath -ll
mpatham (360060e801262eb00504062eb000000ea) dm-50 HITACHI,OPEN-V
size=5.0T features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  |- 1:0:8:1  sdbd 67:112 active ready running
  `- 2:0:10:1 sdbe 67:128 active ready running

I do not know how to add this to cloudstack.

hstan77 commented 2 years ago

Hi

You cannot attach it to Cloudstack directly.

But you can attach to a server that have NFS , and add NFS to Cloudstack.

Of if you use loca storage, then attach it directly to HyberVisor as storage .

On Wed, Apr 20, 2022 at 3:42 PM xuanyuanaosheng @.***> wrote:

Clustered File System is required on top of SAN/LUN (whatever the storage in the background). None of those clustered file systems is something I would have an easy time recommending.

@andrijapanicsb https://github.com/andrijapanicsb I have searched all the doc and goole a lot.

1. https://serverfault.com/questions/734364/add-shared-mountpoint-storage-in-cloudstack

  1. @.***/msg94600.html

But I still can not find a good way how to add a SAN/LUN storage like HP3PAR or HITACHI on the cloudstack, Could you please give some advices ?

— Reply to this email directly, view it on GitHub https://github.com/apache/cloudstack/issues/4391#issuecomment-1103573358, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARFNGAREODLVGL6E3R3W3NTVF6YODANCNFSM4SJSK5RQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Hean Seng

xuanyuanaosheng commented 2 years ago

Hi You cannot attach it to Cloudstack directly. But you can attach to a server that have NFS , and add NFS to Cloudstack. Of if you use loca storage, then attach it directly to HyberVisor as storage . … On Wed, Apr 20, 2022 at 3:42 PM xuanyuanaosheng @.> wrote: Clustered File System is required on top of SAN/LUN (whatever the storage in the background). None of those clustered file systems is something I would have an easy time recommending. @andrijapanicsb https://github.com/andrijapanicsb I have searched all the doc and goole a lot. 1. https://serverfault.com/questions/734364/add-shared-mountpoint-storage-in-cloudstack 2. @./msg94600.html But I still can not find a good way how to add a SAN/LUN storage like HP3PAR or HITACHI on the cloudstack, Could you please give some advices ? — Reply to this email directly, view it on GitHub <#4391 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARFNGAREODLVGL6E3R3W3NTVF6YODANCNFSM4SJSK5RQ . You are receiving this because you were mentioned.Message ID: @.***> -- Regards, Hean Seng

  1. I have test it on NFS, But it has performance issues.
  2. I also test the ceph RBD, But I find the cluster will auto restart when the kvm virtual machine has docker installed. I stll can not find the reason, why this happen.
  3. When using as the local storage, The highly available feature of the cluster will not be available.

So, I'm investigating new solutions, like:

  1. Sharedmountpoint
  2. CLVM: https://www.slideshare.net/MarcusLSorensen/cloud-stack-clvm
  3. Gluster.

I can not find some useful solution on these.

We have investigated another private cloud solution Zstack. It has more reosonable article. I also request two issue:

  1. https://github.com/apache/cloudstack/issues/6291
  2. https://github.com/apache/cloudstack/issues/6292
xuanyuanaosheng commented 2 years ago

@hstan77 @andrijapanicsb @DaanHoogland

As above all, Now I think using the Sharedmountpoint with glusterfs is a solution, Any idears?

rohityadavcloud commented 2 months ago

I think multiple local storage is support with KVM, is this still a valid issue.