phnmnl / cloud-deploy-kubenow

This repository contains scripts to manage the PhenoMeNal cloud CRE.
Apache License 2.0
1 stars 6 forks source link

Port for sftp access #78

Open pcm32 opened 6 years ago

pcm32 commented 6 years ago

@andersla we need an access (port) for sftp here https://github.com/kubenow/KubeNow/blob/master/aws/security_group/main.tf or at an equivalent location. Maybe port 44? Unfortunately Kubernetes ingresses don't work for low level protocols such as tcp, so you cannot redirect a name server to the sftp pod without using an additional reverse proxy that supports it; I understand that traeffik doesn't. On public/publicdev I sorted this with haproxy, but for a start I would just go for a port access.

mcapuccini commented 6 years ago

I am exporting terraform modules from KubeNow and bringing them to the terraform registry (https://github.com/mcapuccini/terraform-terreno-modules). I am thinking of a way to make the security groups generic, so we can add additional open ports.

Not sure about Traefik FTP, but we could ask them on Slack! I'm still of the idea that we should convince the users to pull the data though 🙂

andersla commented 6 years ago

I have already an almost working generic version in some development branch :-)

pcm32 commented 6 years ago

Great!

mcapuccini commented 6 years ago

@pcm32 would WebDav work as a backup plan?

pcm32 commented 6 years ago

Well, the issue is that galaxy has built in support for sftp in the newest helm chart through the use of the community setup, and it has an internal UI for that (which we didn't write) which uses the same Galaxy users from the database for auth. Using webDav would mean redoing all of that for webdav, and I don't think we have the man power. Allowing a port entry, or even setting up haproxy (which is a lot more work already than setting the port), would be far less work to do than having to support webdav on Galaxy.

Galaxy does have support for pulling files from URLs and that is being exposed, but for some users it might be more convenient to use an sftp solution which is already well supported by Galaxy. We are just trying to expose all the solutions that Galaxy has for uploads to make life easier for users.

pcm32 commented 6 years ago

I would appreciate if short term we could have the simplest solution (port access) and then maybe in the future something more elaborate like having haproxy, but I would like to avoid delaying the sftp access as it was planned already for the past release.

andersla commented 6 years ago

Fixed short term via https://github.com/phnmnl/cloud-deploy-kubenow/commit/25552911aa36d74cc175a965c3e5dcc75aced495