anitsh / til

Today I Learn (til) - Github `Issues` used as daily learning management system for taking notes and storing resource links.
https://anitshrestha.com.np
MIT License
78 stars 11 forks source link

Filesystem, Linux Filesystem Hierarchy Standard (FHS), Virtual Filesystem #65

Open anitsh opened 4 years ago

anitsh commented 4 years ago

Everything is a file in Linux

Linux views all file systems from the perspective of a common set of objects.

Filesystem

A file system or filesystem (often abbreviated to fs) controls how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stops and the next begins. By separating the data into pieces and giving each piece a name, the data is easily isolated and identified. Taking its name from the way paper-based data management system is named, each group of data is called a "file." The structure and logic rules used to manage the groups of data and their names is called a "file system."

There are many different kinds of file systems. Each one has different structure and logic, properties of speed, flexibility, security, size and more. Some file systems have been designed to be used for specific applications.

File systems can be used on numerous different types of storage devices that use different kinds of media. As of 2019, hard disk drives have been key storage devices and are projected to remain so for the foreseeable future. Other kinds of media that are used include SSDs, magnetic tapes, and optical discs. In some cases, such as with tmpfs, the computer's main memory (random-access memory, RAM) is used to create a temporary file system for short-term use.

Some file systems are used on local data storage devices; others provide file access via a network protocol (for example, NFS, SMB, or 9P clients). Some file systems are "virtual", meaning that the supplied "files" (called virtual files) are computed on request (such as procfs and sysfs) or are merely a mapping into a different file system used as a backing store. The file system manages access to both the content of files and the metadata about those files. It is responsible for arranging storage space; reliability, efficiency, and tuning with regard to the physical storage medium are important design considerations.

Filesystem Hierarchy Standard (FHS)

The Filesystem Hierarchy Standard (FHS) defines the directory structure and directory contents in Linux distributions. It is maintained by the Linux Foundation.

Virtual Filesystem

The Linux kernel requires that for an entity to be a filesystem, it must also implement the open(), read(), and write() methods on persistent objects that have names associated with them. From the point of view of object-oriented programming, the kernel treats the generic filesystem as an abstract interface, and these big-three functions are "virtual," with no default definition. Accordingly, the kernel's default filesystem implementation is called a virtual filesystem (VFS).

ls command output description Character Type
- regular file
d directory
b block type special file
c character type special file
l symbolic link
p pipe
s socket

Resource


The Linux 5.10 release included a change that is expected to significantly increase the performance of the ext4 filesystem; it goes by the name "fast commits" and introduces a new, lighter-weight journaling method. Let us look into how the feature works, who can benefit from it, and when its use may be appropriate.Ext4 is a journaling filesystem.


https://en.wikipedia.org/wiki/Journaling_file_system A journaling file system is a file system that keeps track of changes not yet committed to the file system's main part by recording the intentions of such changes in a data structure known as a "journal", which is usually a circular log. In the event of a system crash or power failure, such file systems can be brought back online more quickly with a lower likelihood of becoming corrupted. Depending on the actual implementation, a journaling file system may only keep track of stored metadata, resulting in improved performance at the expense of increased possibility for data corruption. Alternatively, a journaling file system may track both stored data and related metadata, while some implementations allow selectable behavior in this regard.

anitsh commented 4 years ago

Filesystem Hierarchy Standard (FHS)

The Filesystem Hierarchy Standard defines the directory structure and directory contents in Linux distributions. It is maintained by the Linux Foundation.

image

To comply with the FSSTND the following directories, or symbolic links to directories, are required in root directory where Linux file system starts.

   /           The root directory. 
   /bin       Essential command binaries
   /boot     Static files of the boot loader
   /dev      Device files
   /etc       Host-specific system configuration
   /lib       Essential shared libraries and kernel modules
   /media     Mount point for removeable media
   /mnt       Mount point for mounting a filesystem temporarily
   /opt       Add-on application software packages
   /sbin      Essential system binaries
   /srv       Data for services provided by this system
   /tmp       Temporary files
   /usr       Secondary hierarchy
   /var       Variable data
anitsh commented 4 years ago

/bin

  • Several useful commands that are of use to both the system administrator as well as non-privileged users.
  • It contains essential system programs that must be available even if only the partition containing / is mounted.
  • There are no subdirectories in /bin and that the following commands, or symbolic links to commands, are located there.
anitsh commented 4 years ago

/boot http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/boot.html

This directory contains everything required for the boot process except for configuration files not needed at boot time (the most notable of those being those that belong to the GRUB boot-loader) and the map installer.

/boot/map Contains the location of the kernel /boot/map Contains the location of the kernel. /boot/vmlinuz, /boot/vmlinuz-kernel-version Normally the kernel or symbolic link to the kernel. /boot/grub Contains the GRUB(the GNU GRand Unified Bootloader) configuration files including boot-up images and sounds.

anitsh commented 4 years ago

/dev http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html

Is the location of special or device files.

anitsh commented 4 years ago

/etc http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/etc.html

This is the nerve center of your system, it contains all system related configuration files in here or in its sub-directories. A "configuration file" is defined as a local file used to control the operation of a program; it must be static and cannot be an executable binary. For this reason, it's a good idea to backup this directory regularly. It will definitely save you a lot of re-configuration later if you re-install or lose your current installation. Normally, no binaries should be or are located here.

anitsh commented 4 years ago

/home http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/home.html

Linux is a multi-user environment so each user is also assigned a specific directory that is accessible only to them and the system administrator.

anitsh commented 4 years ago

initrd http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/initrd.html

It provides the capability to load a RAM disk by the boot loader. This RAM disk can then be mounted as the root file system and programs can be run from it. Afterwards, a new root file system can be mounted from a different device. The previous root (from initrd) is then moved to a directory and can be subsequently unmounted. initrd is mainly designed to allow system startup to occur in two phases, where the kernel comes up with a minimum set of compiled-in drivers, and where additional modules are loaded from initrd.

anitsh commented 4 years ago

/lib http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/lib.html

It contains kernel modules and those shared library images (the C programming code library) needed to boot the system and run the commands in the root filesystem, ie. by binaries in /bin and /sbin.

anitsh commented 4 years ago

/lost+found http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/lostfound.html

In case of system crash or a power failure, the result of the recovery operation will be placed in this directory.

anitsh commented 4 years ago

/media http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/media.html

This directory contains subdirectories which are used as mount points for removeable media such as floppy disks, cdroms and zip disks.

anitsh commented 4 years ago

/mnt http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/mnt.html

This directory is provided so that the system administrator may temporarily mount a filesystem as needed. The content of this directory is a local issue and should not affect the manner in which any program is run. This directory must not be used by installation programs: a suitable temporary directory not in use by the system must be used instead.

anitsh commented 4 years ago

/opt http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/opt.html

This directory is reserved for all the software and add-on packages that are not part of the default installation.

anitsh commented 4 years ago

/proc http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/proc.html

A process information pseudo-file system. Each of the numbered directories corresponds to an actual process ID.

This is very special in that it is also a virtual filesystem. It's sometimes referred to as a process information pseudo-file system. It doesn't contain 'real' files but runtime system information (e.g. system memory, devices mounted, hardware configuration, etc). For this reason it can be regarded as a control and information centre for the kernel. In fact, quite a lot of system utilities are simply calls to files in this directory. For example, 'lsmod' is the same as 'cat /proc/modules' while 'lspci' is a synonym for 'cat /proc/pci'. By altering files located in this directory you can even read/change kernel parameters (sysctl) while the system is running.

The most distinctive thing about files in this directory is the fact that all of them have a file size of 0, with the exception of kcore, mtrr and self.

anitsh commented 4 years ago

/root http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/root.html

This is the home directory of the System Administrator, 'root'.

anitsh commented 4 years ago

/sbin http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/sbin.html

It contains binaries essential for booting, restoring, recovering, and/or repairing the system, system maintenance and/or administrative tasks.

The less important ones - in /usr/sbin and locally installed in /usr/local/sbin.

FSSTND compliance requires that the following commands, or symbolic links to commands in /sbin

   shutdown   Command to bring the system down.
   fastboot     Reboot the system without checking the disks (optional)
   fasthalt      Stop the system without checking the disks (optional)
   fdisk          Partition table manipulator (optional)
   fsck           File system check and repair utility (optional)
   fsck.*        File system check and repair utility for a specific filesystem (optional)
   getty        The getty program (optional)
   halt          Command to stop the system (optional)
   ifconfig    Configure a network interface (optional)
   init           Initial process (optional)
   mkfs       Command to build a filesystem (optional)
   mkfs.*     Command to build a specific filesystem (optional)
   mkswap     Command to set up a swap area (optional)
   reboot     Command to reboot the system (optional)
   route      IP routing table utility (optional)
   swapon     Enable paging and swapping (optional)
   swapoff    Disable paging and swapping (optional)
   update     Daemon to periodically flush filesystem buffers (optional)
anitsh commented 4 years ago

/usr http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/usr.html

User System Resources or user usable programs and data

This is one of the most important directories in the system as it contains all the user binaries, their documentation, libraries, header files, etc...

/usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere.

Large software packages must not use a direct subdirectory under the /usr hierarchy.

anitsh commented 4 years ago

/var http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/var.html

Contains variable data like system logging files, mail and printer spool directories, and transient and temporary files. Some portions of /var are not shareable between different systems. For instance, /var/log, /var/lock, and /var/run. Other portions may be shared, notably /var/mail, /var/cache/man, /var/cache/fonts, and /var/spool/news.

The following directories, or symbolic links to directories, are required in /var for FSSTND compliance: /var/cache Application cache data /var/lib Variable state information /var/local Variable data for /usr/local /var/lock Lock files /var/log Log files and directories /var/opt Variable data for /opt /var/run Data relevant to running processes /var/spool Application spool data /var/tmp Temporary files preserved between system reboots

Reserved directories /var/backups /var/cron /var/msgs /var/preserve

anitsh commented 4 years ago

/srv http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/srv.html

It contains site-specific data which is served by this system.

This main purpose of specifying this is so that users may find the location of the data files for particular service, and so that services which require a single tree for readonly data, writable data and scripts (such as cgi scripts) can be reasonably placed.

Data that is only of interest to a specific user should go in that users' home directory. The methodology used to name subdirectories of /srv is unspecified as there is currently no consensus on how this should be done. One method for structuring data under /srv is by protocol, eg. ftp, rsync, www, and cvs. On large systems it can be useful to structure /srv by administrative context, such as /srv/physics/www, /srv/compsci/cvs, etc. This setup will differ from host to host. Therefore, no program should rely on a specific subdirectory structure of /srv existing or data necessarily being stored in /srv. However /srv should always exist on FHS compliant systems and should be used as the default location for such data. Distributions must take care not to remove locally placed files in these directories without administrator permission. This is particularly important as these areas will often contain both files initially installed by the distributor, and those added by the administrator.

anitsh commented 4 years ago

/tmp http://www.tldp.org/LDP/Linux-Filesystem-Hierarchy/html/tmp.html

This directory contains mostly files that are required temporarily. Many programs use this to create lock files and for temporary storage of data. Do not remove files from this directory unless you know exactly what you are doing!

anitsh commented 3 years ago

Pseudo File System

a synthetic file system or a pseudo file system is a hierarchical interface to non-file objects that appear as if they were regular files in the tree of a disk-based or long-term-storage file system. These non-file objects may be accessed with the same system calls or utility programs as regular files and directories. The common term for both regular files and the non-file objects is node.

The benefit of synthetic file systems is that well known file system semantics can be reused for a universal and easily implementable approach to interprocess communication. Clients can use such a file system to perform simple file operations on its nodes, and do not have to implement complex message encoding and passing methods and other aspects of protocol engineering. For most operations, common file utilities can be used, so even scripting is quite easy.

This is commonly known as everything is a file and is generally regarded to have originated from Unix.

kernfs is a set of functions that contain the functionality required for creating pseudo file systems used internally by various kernel subsystems. The creation of kernfs resulted from splitting off part of the internal logic used by sysfs, which provides a set of virtual files by exporting information about hardware devices and associated device drivers from the kernel's device model to user space, into an independent and reusable functionality so other kernel subsystems can implement their own pseudo file systems more easily and consistently.

Usage Examples

Plan 9 drives this concept expansively: most operating system services, e.g. hardware access and networking stack are presented as fileservers. This way it is trivial to use these resources remotely (e.g. one host directly accessing another host's block devices or network interfaces) without the need of additional protocols.

Other implementations of the 9P file system protocol also exists for many other systems and environments.

Based on 9P, Plan 9's network filesystem, studies suggest using synthetic filesystems as universal access scheme to that information. The major benefit is that 9P is very simple and so quite easy to implement in hardware and can be easily used and over virtually any kind of network (from a serial link up to the internet).