awslabs / scale-out-computing-on-aws

Scale-Out Computing on AWS is a solution that helps customers deploy and operate a multiuser environment for computationally intensive workflows.
https://awslabs.github.io/scale-out-computing-on-aws-documentation/
Apache License 2.0
124 stars 59 forks source link

Support for Windows MyFiles and Data Persistence #141

Open mirneshalilovic opened 1 month ago

mirneshalilovic commented 1 month ago

Is your feature request related to a problem? Please describe. Yes, currently for Linux users, there is a possibility to view files inside the app and automatically attach a persistent volume. However, Windows users lack this functionality. Windows local users (not connected to Azure AD) face permission issues due to different SIDs being generated each time a new user profile is created. This leads to problems in persisting user data across sessions, and there is no option to automatically attach a volume as a D drive for user data storage.

Describe the feature you'd like

Additional context A persistent D drive for user data would enhance the user experience by allowing for data storage continuity across sessions. Automatically attaching the D drive would save manual effort and reduce configuration complexity.

mcrozes commented 1 month ago

Hello @mirneshalilovic -

Agree with your post here, and we are already planning to add native support for FSx for Netapp ONTAP (https://aws.amazon.com/fr/fsx/netapp-ontap/) in a future release. As mentioned, the goal here is to provide a seamless experience for Linux and Windows users, via a single, persistent and shared filesystem.

You can still integrate FSx for Netapp ONTAP today on SOCA, but the process is manual, the goal here is to have everything fully automated for (1) new Netapp mount (e.g: you will have the option to use Netapp for /apps or /data) as well as (2) make it easier to mount existing/on-premise Netapp volumes.

We have a beta build if you are interested to test this feature, feel free to email aligo@amazon.com.

I will keep this ticket open for visibility until this feature is GA.

mirneshalilovic commented 1 month ago

Hello @mcrozes

Thank you for the detailed response and for confirming that native FSx for NetApp ONTAP support is on the roadmap! The ability to automate the mounting of new and existing NetApp volumes will be a huge improvement for our use case

Thanks again for keeping this ticket open for visibility, and I look forward to contributing to the process!

sysdori commented 1 month ago

This feature seems very useful. I tested an environment where FSx Windows and OpenZFS file systems were automatically mounted to Windows VDI users with Windows AD, but there was a lot of manual work. I expect that the usability will greatly improve if data can be easily shared between heterogeneous OS users.

mcrozes commented 3 weeks ago

Hello all,

Just letting you know we have started working on native FSxN support for /apps & /data. Phase1 is to extend support of both shares from efs | fsx_lustre to efs | fsx_lustre | fsx_ontap.

We are also brainstorming on a new FileSystem SSM tree that will make FS automount / addition easier (e.g: you will be able to add new filesystem on-demand, post cluster deployment).

This is still in active development, feel free to email aligo@amazon.com / email me directory if you want to follow the latest updates. I expect to be able to share a preview build by end of November.