Request for Helm Chart for FileSender Deployment

[siddjellali]

Hello team,

I am writing to request the creation and availability of a Helm Chart for deploying FileSender on Kubernetes. A Helm Chart would greatly simplify the deployment and management of FileSender in a Kubernetes environment, making it more accessible to a broader audience.

Here are some specific reasons why a Helm Chart would be beneficial:

• Ease of Deployment: A Helm Chart would automate the setup process, allowing users to deploy FileSender with a single command, reducing the potential for human error and configuration issues.

• Configuration Management: Helm Charts allow for easy management of configuration parameters, making it straightforward to customize the deployment for various environments.

• Scalability: With Kubernetes' inherent scalability features, a Helm Chart can help users efficiently scale their FileSender instances according to their needs.

• Best Practices: Helm Charts can encapsulate Kubernetes best practices, ensuring that deployments are secure, reliable, and maintainable.

• Community Adoption: Providing a Helm Chart could increase the adoption of FileSender within the Kubernetes community, as many organizations prefer using Helm for managing their Kubernetes applications.

Thanks

[WebSpider]

We would first need to take a decision on whether we want to support a container image.

[madsi1m]

I've seen an all on one container as well as several containers serving FileSender. It gets more complex when you make its components clustered and HA.

[WebSpider]

I have made several deployments, from simple all-in-one to distributed autoscaling, and it all works. Each type of deployment has its on gotcha's, usually related to getting it to perform, or being nice to the underlying storage.

[madsi1m]

what kind of upload speeds have you seen on performant deploys?

[WebSpider]

It's a bit of difficult to slap a number on it:

• S3 based deploys are usually limited to the per-stream bandwidth that the object storage will handle
• Local/NFS are usually limited to the IOPS you can concurrently squeeze out of the storage, and then limited by the webserver

Overall, I guess a safe number to go by is 1Gbit/s per web frontend

[meijer]

@WebSpider from your perspective - what are the practical consequences of the FileSender programme supporting a container image? What kind of workload, what kind of risk, compared to the current tarball distribution?

[WebSpider]

It is mostly a one-off effort, and then keeping up with dependency changes.

Its best compared to keeping install instructions i guess: sometimes you need to change the php or simplesaml version.

Then good thing about this, is that github helps these days, with the dependabot addon, that I have proposed in PR #2022

[maxadamo]

It is mostly a one-off effort, and then keeping up with dependency changes.

Its best compared to keeping install instructions i guess: sometimes you need to change the php or simplesaml version.

Then good thing about this, is that github helps these days, with the dependabot addon, that I have proposed in PR #2022

I second the request to have an official container, maintained by this community, because the container needs to be updated along with the changes in the code. You can also setup your CI, with all the tests, and the container will be automatically created and pushed. At GÉANT we had to shutdown the service because we could not manage to reconfigure the software every time, and I believe that a container would help to streamline the upgrade process.

[WebSpider]

At GÉANT we had to shutdown the service because we could not manage to reconfigure the software every time, and I believe that a container would help to streamline the upgrade process.

I for one am sad to hear you guys had to shutdown the service for this reason and think @meijer @guidoaben and @efef should know about this.

[monkeyiq]

I am interested in how we can make the software easier to upgrade. It sounds like it might be too late for this installation but perhaps we could have some discussion on what we could do and what parts of reconfiguring the software were the most time consuming for you.

A container might be still wanting to use an external database (or the db in a or the same container itself). There would still need to be some db migration when a version updates. It may be that the /opt/filesender/config/config.php configuration file is overlayed into the container. Then the container would bring apache/php and perhaps SSP software.

One topic I have seen mentioned in other official container discussions is to what degree the container is hardened. modSecurity etc. I imagine that many folks do not want to give public details of what their defensive measures are for a deployment. Though for a container that information and configuration for that instance becomes public knowledge.

The storage could be either offered to the container from an external source or with the cloud storage that could be contacted directly from the container.

If the container is made as an artefact of the CI system then the CI system could run system tests on the container too in order to validate not just the software as normal tests but that the container has not regressed.

@maxadamo I guess container discussion aside, I am very interested in what reconfigure steps took the most time so we might look at how we can reduce that burden for other installations.

[maxadamo]

@monkeyiq matching the php version is one of the issues. My colleague is pointing out that we upgraded PHP once, and SSO broke. The version of PHP is often tied to the distribution flavour and version, and third part repositories are required. I also recall having a problem to find a library for PHP on Ubuntu. This happens very often with PHP and it's not strictly related to Filesender. For instance, to handle sessions with Redis on SimplaSAMLphp, they are using an esoteric PHP library (called predis), which is not packaged and needs to be installed with php composer. Whilst this is not related to Filesender it helps understanding that the container is a streamlined approach for PHP applications, because you take care and you match all the dependencies inside the container image. For instance I may not use containers to run a Go binary, but it comes to hand for applications with a lots of dependencies, like PHP.

[WebSpider]

I guess container discussion aside, I am very interested in what reconfigure steps took the most time so we might look at how we can reduce that burden for other installations.

From what I've seen, what takes the most time is testing everything is still working when you upgrade. There aren't many regressions I've seen in the last years, but sometimes there are interesting side-effects of browsers not supporting features, and new features being built on top of each-other.

[monkeyiq]

@maxadamo Thank you for your information. Having such feedback provides a great opportunity to address issues!

I know there has been talk on the SSP channels about providing a recommended docker image for SSP. I guess if that happens we could extend it or rely on it for FileSender to help deployments.

[monkeyiq]

@WebSpider When the update to Codeception happens running selenium tests locally will be much simpler. This should also allow sites to contribute to the main UI tests in FileSender so they can be run locally at a site when doing A/B testing. This will probably not eliminate all testing but being able to automate local acceptance testing on the base stuff with a site's deployment should help everybody.

I would also like to extend the number of browsers used in Codeception out to include Chrome, Firefox and Safari and common mobile browsers/versions.

[WebSpider]

A container might be still wanting to use an external database (or the db in a or the same container itself). There would still need to be some db migration when a version updates. It may be that the /opt/filesender/config/config.php configuration file is overlayed into the container. Then the container would bring apache/php and perhaps SSP software.

This can easily be done in one (or usually) multiple containers. There are multiple designs possible, and I know multiple implementations have been made, which could be used as a basis.

One topic I have seen mentioned in other official container discussions is to what degree the container is hardened. modSecurity etc. I imagine that many folks do not want to give public details of what their defensive measures are for a deployment. Though for a container that information and configuration for that instance becomes public knowledge.

This is the security-through-obscurity discussion the way I see it.

The storage could be either offered to the container from an external source or with the cloud storage that could be contacted directly from the container.

Yes. And here as well: Deployments are out there, and so is the experience. The matter here is, does the FileSender project want to support a containerised version of it's software?

If the container is made as an artefact of the CI system then the CI system could run system tests on the container too in order to validate not just the software as normal tests but that the container has not regressed.

Yes, this is very well possible using for instance the DIND and KIND projects, that allow to spin docker and/or k8s inside your CI pipeline, and thus make a realistic deployment.