Staging file folders should allow members of the group fslg_dnasc to add their own files. Note that this requires us to loosen up our assumptions when staging and deleting files.
File operation configuration
The app is configured using the following measures so that group members can add their own files to project, dataset, and analysis directories:
main script will exit if it is not run on a process where GID is the group ID, e.g. using sg $group_name -c "python -m app". This requires that the user running the app be a member of $group_name.
value of directory file mode is 1775 (octal)
staging module sets the process umask to 0000 so that permissions can be specified explicitly
Removing files created by non-app users
In the event that a staging directory contains a file or directory that does not belong to the app's user, then the app will be unable to delete that directory. All files that do belong to the app's user should be deleted, but that's all that can be done. The directory and the remaining file(s) will just have to stay.
File operation errors
The app's tasks to create and remove files in staging folders are a source of several potential errors, described below. Note that 'file' often refers to both files and directories.
FileNotFoundError
This exception is raised when the specified file does not exist. Given that any staging file can be deleted without the app knowing about it, checking that a file exists before modifying it should occur frequently.
FileExistsError
The exception is raised when trying to create a file or directory that already exists. Similar to FileNotFoundError, this exception is relevant because we can't assume just because a file should have once existed that it still exists.
PermissionError
fslg_dnasc users are able to add and remove files and directories to and from staging directories. Attempts to delete a project or dataset directory will raise a PermissionError if any file within that directory does not belong to the app's user. Therefore, any delete operation must first check that each file involved belongs to the app's user.
More potential errors to consider
IsADirectoryError: Raised by os.rename when the source is a directory but the destination is a file.
NotADirectoryError: Raised by os.rename when the source is a file but the destination is a directory.
OSError: This is a more general exception that can be raised for a variety of reasons, such as system-related errors.
Linking $APP_USER (e.g. pacbiodnaseq) files
Use of the $APP_USER user is required to create links. This being the case, $APP_USER must have write permission to all staging directories. Therefore, $APP_USER needs to belong to $GROUP_NAME.
Staging file permissions
Staging file folders should allow members of the group
fslg_dnascto add their own files. Note that this requires us to loosen up our assumptions when staging and deleting files.File operation configuration
The app is configured using the following measures so that group members can add their own files to project, dataset, and analysis directories:
sg $group_name -c "python -m app". This requires that the user running the app be a member of$group_name.umaskto 0000 so that permissions can be specified explicitlyRemoving files created by non-app users
In the event that a staging directory contains a file or directory that does not belong to the app's user, then the app will be unable to delete that directory. All files that do belong to the app's user should be deleted, but that's all that can be done. The directory and the remaining file(s) will just have to stay.
File operation errors
The app's tasks to create and remove files in staging folders are a source of several potential errors, described below. Note that 'file' often refers to both files and directories.
FileNotFoundErrorThis exception is raised when the specified file does not exist. Given that any staging file can be deleted without the app knowing about it, checking that a file exists before modifying it should occur frequently.
FileExistsErrorThe exception is raised when trying to create a file or directory that already exists. Similar to
FileNotFoundError, this exception is relevant because we can't assume just because a file should have once existed that it still exists.PermissionErrorfslg_dnascusers are able to add and remove files and directories to and from staging directories. Attempts to delete a project or dataset directory will raise aPermissionErrorif any file within that directory does not belong to the app's user. Therefore, any delete operation must first check that each file involved belongs to the app's user.More potential errors to consider
IsADirectoryError: Raised byos.renamewhen the source is a directory but the destination is a file.NotADirectoryError: Raised byos.renamewhen the source is a file but the destination is a directory.OSError: This is a more general exception that can be raised for a variety of reasons, such as system-related errors.Linking
$APP_USER(e.g.pacbiodnaseq) filesUse of the
$APP_USERuser is required to create links. This being the case,$APP_USERmust have write permission to all staging directories. Therefore,$APP_USERneeds to belong to$GROUP_NAME.