Config Glusterfs with SSL/TLS

[drngsl]

Hi, According to guide https://gluster.readthedocs.io/en/latest/Administrator%20Guide/SSL/, I could not use glusterfs normally, so here I want to ask some questions. Env: on glusterfs host: root@minion3:/etc/ssl# ll total 72 drwxr-xr-x 4 root root 4096 Sep 29 15:52 ./ drwxr-xr-x 136 root root 12288 Sep 29 15:37 ../ drwxr-xr-x 3 root root 24576 Sep 1 22:13 certs/ -rw-r--r-- 1 root root 733 Sep 29 15:52 glusterfs.ca -rw-r--r-- 1 root root 887 Sep 29 15:52 glusterfs.key -rw-r--r-- 1 root root 733 Sep 29 15:52 glusterfs.pem ... glusterfs version root@minion3:/etc/ssl# gluster --version glusterfs 3.6.9 built on Mar 2 2016 18:21:17 Repository revision: git://git.gluster.com/glusterfs.git Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com> GlusterFS comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of GlusterFS under the terms of the GNU General Public License. gluster volume info `root@minion3:/etc/ssl# gluster volume info

Volume Name: vol-ssl Type: Distribute Volume ID: 56a6148c-6abb-4ffa-abcb-35f2f0df4d42 Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: _..**.**_:/export/vol-ssl Options Reconfigured: client.ssl: on server.ssl: on`

on client host, copy ssl/tls related file from glusterfs server host. root@minion2:/# ll /etc/ssl/ total 72 drwxr-xr-x 4 root root 4096 Sep 29 16:08 ./ drwxr-xr-x 135 root root 12288 Sep 29 19:44 ../ drwxr-xr-x 3 root root 24576 Sep 1 22:13 certs/ -rw-r--r-- 1 root root 733 Sep 29 16:05 glusterfs.ca -rw-r--r-- 1 root root 887 Sep 29 16:08 glusterfs.key -rw-r--r-- 1 root root 733 Sep 29 16:07 glusterfs.pem ... while execute cmd mount.glusterfs failed root@minion2:/# mount.glusterfs ***.***.***.***:/vol-ssl /mnt Mount failed. Please check the log file for more details.

LOG on client host: [2016-09-29 11:44:34.850226] I [rpc-clnt.c:1761:rpc_clnt_reconfig] 0-vol-ssl-client-0: changing port to 49152 (from 0) [2016-09-29 11:44:34.855100] E [socket.c:384:ssl_setup_connection] 0-vol-ssl-client-0: SSL connect error [2016-09-29 11:44:34.855160] E [socket.c:2386:socket_poller] 0-vol-ssl-client-0: client setup failed [2016-09-29 11:44:34.860610] I [fuse-bridge.c:5086:fuse_graph_setup] 0-fuse: switched to graph 0 [2016-09-29 11:44:34.861495] I [fuse-bridge.c:4015:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.23 [2016-09-29 11:44:34.861799] W [fuse-bridge.c:779:fuse_attr_cbk] 0-glusterfs-fuse: 2: LOOKUP() / => -1 (Transport endpoint is not connected) [2016-09-29 11:44:34.870410] I [fuse-bridge.c:4927:fuse_thread_proc] 0-fuse: unmounting /mnt [2016-09-29 11:44:34.870949] W [glusterfsd.c:1211:cleanup_and_exit] (--> 0-: received signum (15), shutting down [2016-09-29 11:44:34.870976] I [fuse-bridge.c:5607:fini] 0-fuse: Unmounting '/mnt'. on server host: [2016-09-29 12:03:38.884791] E [socket.c:1972:__socket_read_frag] 0-rpc: wrong MSG-TYPE (-1476197977) received from ***.***.***.***:1021

Error happened when connect to server with ssl/tls. I don't know why.

[mohit84]

Hi,

As per shared logs it confirms SSL api (SSL_Connect) is failing to connect with host. [socket.c:384:ssl_setup_connection] 0-vol-ssl-client-0: SSL connect error.

After checked with size of glusterfs.ca it seems file is same on both host (client and server),size of .ca should not be same if client and server are on different host.To correct it please follow below steps

1) For self signed CA certificates on servers, collect the .pem certificates of clients and servers, that is, /etc/ssl/glusterfs.pem files from every system. Concatenate the collected files into a single file. Place this file in /etc/ssl /g l usterfs.ca on all the servers in the trusted storage pool.

2) For self-signed CA certificates on clients, collect the .pem certificates of servers, that is, /etc/ssl/glusterfs.pem files from every server. Concatenate the collected files into a single file. Place this file in /etc/ssl/glusterfs.ca on all the clients.

Regards Mohit Agrawal

[drngsl]

@mohit84 , thanks! And about gluster volume set MYVOLUME auth.ssl-allow Zaphod what's the meaning of the user Zaphod? Is this user Zaphod the hostname of client?

If I'd like to make glusterfs volume vol-ssl only be accessed by client minon2, How to set the option auth.ssl-allow?

[mohit84]

Hi,

cli(gluster) does provide option "auth.ssl-allow " to set the list of common names of all server/client to access the volume. The common name is provides to as a argument at the time of generate private key like below

openssl req -new -x50 9 -key /etc/ssl/glusterfs.key -subj "/CN= COMMONNAME" -out /etc/ssl/glusterfs. pem

You can assign a name to every client and server at the time of private key generation and same can be configure through cli(gluster) whatever you want to give the access.

Regards Mohit Agrawal;

[drngsl]

so, every time I create an new client, and access to glusterfs volume, I need push the content of glusterfs.pem generated on client to the file /etc/ssl/glusterfs.ca of glusterfs server host?

[mohit84]

Yes, in case of SSL key transferred step is required.

[humblec]

@drngsl Is this issue resolved? If yes, please close the same .