Tuesday, 24 June 2014

                               Linux filesystem


/usr - This is one of the most important directories in the system as it
contains all the user binaries. X and its supporting libraries can be
found here. User programs like telnet, ftp etc are also placed here.
/usr/doc contains useful system documentation. /usr/src/linux contains the
source code for the Linux kernel. 
/var - This directory contains spooling data like mail and also the output
from the printer daemon. The system logs are also kept here in
/var/log/messages. You will also find the database for BIND in /var/named
and for NIS in /var/yp. 
This was a short and basic look at the Linux filesystem. You do need to
have at least this basic knowledge of the layout of the filesystem to
fully utilize its potential. One good place to read about the filesystem
is this detailed document at www.pathname.com/fhs/1.2/fsstnd-toc.htmlthat
specifies the standard structure of the Linux filesystem. 

interesting


/dev - This is a very interesting directory that highlights one important
characteristic of the Linux filesystem - everything is a file or a
directory. Look through this directory and you should see hda1, hda2 etc,
which represent the various partitions on the first master drive of the
system. /dev/cdrom and /dev/fd0 represent your CDROM drive and your floppy
drive. This may seem strange but it will make sense if you compare the
characteristics of files to that of your hardware. Both can be read from
and written to. Take /dev/dsp, for instance. This file represents your
speaker device. So any data written to this file will be re-directed to
your speaker. Try 'cat /etc/lilo.conf > /dev/dsp' and you should hear some
sound on the speaker. That's the sound of your lilo.conf file! Similarly,
sending data to and reading from /dev/ttyS0 ( COM 1 ) will allow you to
communicate with a device attached there - your modem. 
/etc - This directory contains all the configuration files for your system.
Your lilo.conf file lies in this directory as does hosts, resolv.conf and
fstab. Under this directory will be X11 sub-directory which contains the
configuration files for X. More importantly, the /etc/rc.d directory
contains the system startup scripts. This is a good directory to backup
often. It will definitely save you a lot of re-configuration later if you
re-install or lose your current installation. 
/home - Linux is a multi-user environment so each user is also assigned a
specific directory which is accessible only to them and the system
administrator. These are the user home directories, which can be found
under /home/username. This directory also contains the user specific
settings for programs like IRC, X etc. 
/lib - This contains all the shared libraries that are required by system
programs. Windows equivalent to a shared library would be a DLL file. 
/lost+found - Linux should always go through a proper shutdown. Sometimes
your system might crash or a power failure might take the machine down.
Either way, at the next boot, a lengthy filesystem check using fsck will
be done. Fsck will go through the system and try to recover any corrupt
files that it finds. The result of this recovery operation will be placed
in this directory. The files recovered are not likely to be complete or
make much sense but there always is a chance that something worthwhile is
recovered. 
/mnt - This is a generic mount point under which you mount your filesystems
or devices. Mounting is the process by which you make a filesystem
available to the system. After mounting your files will be accessible
under the mount-point. This directory usually contains mount points or
sub-directories where you mount your floppy and your CD. You can also
create additional mount-points here if you want. There is no limitation to
creating a mount-point anywhere on your system but convention says that
you do not litter your file system with mount-points. 
/opt - This directory contains all the software and add-on packages that
are not part of the default installation. Generally you will find KDE and
StarOffice here. Again, this directory is not used very often as it's
mostly a standard in Unix installations. 
/proc - This is a special directory on your system. We have a more detailed
article on this one here. 
/root - We talked about user home directories earlier and well this one is
the home directory of the user root. This is not to be confused with the
system root, which is directory at the highest level in the filesystem. 
/tmp - This directory contains mostly files that are required temporarily.
Many programs use this to create lock files and for temporary storage of

data. On some systems, this directory is cleared out at boot or at
shutdown.

Device Special Files


Device Special Files
Linux, like all versions of Unix TM presents its hardware devices as special files. So, for example, /dev/null is the null device. A device file does not use any data space in the file system, it is only an access point to the device driver. The EXT2 file system and the Linux VFS both implement device files as special types of inode. There are two types of device file; character and block special files. Within the kernel itself, the device drivers implement file semantices: you can open them, close them and so on. Character devices allow I/O operations in character mode and block devices require that all I/O is via the buffer cache. When an I/O request is made to a device file, it is forwarded to the appropriate device driver within the system. Often this is not a real device driver but a pseudo-device driver for some subsystem such as the SCSI device driver layer. Device files are referenced by a major number, which identifies the device type, and a minor type, which identifies the unit, or instance of that major type. For example, the IDE disks on the first IDE controller in the system have a major number of 3 and the first partition of an IDE disk would have a minor number of 1. So, ls -l of /dev/hda1 gives:
$ brw-rw----   1 root    disk       3,    1  Nov 24  15:09 /dev/hda1
Within the kernel, every device is uniquely described by a kdev_t data type, this is two bytes long, the first byte containing the minor device number and the second byte holding the major device number.
The IDE device above is held within the kernel as 0x0301. An EXT2 inode that represents a block or character device keeps the device's major and minor numbers in its first direct block pointer. When it is read by the VFS, the VFS inode data structure representing it has its i_rdevfield set to the correct device identifier.

Footnotes:


Footnotes:
1 Well, not knowingly, although I have been bitten by operating systems with more lawyers than Linux has developers
File translated from TEX by TTH, version 1.0.
Top of Chapter, Table of Contents, Show Frames, No Frames
© 1996-1999 David A Rusling copyright notice.



The first thing that most new users shifting from Windows will find
confusing is navigating the Linux filesystem. The Linux filesystem
does things a lot more differently than the Windows filesystem.
This article explains the differences and takes you through the
layout of the Linux filesystem. 
For starters, there is only a single hierarchal directory structure.
Everything starts from the root directory, represented by '/', and then
expands into sub-directories. Where DOS/Windows had various partitions and
then directories under those partitions, Linux places all the partitions
under the root directory by 'mounting' them under specific directories.
Closest to root under Windows would be c:. 
Under Windows, the various partitions are detected at boot and assigned a
drive letter. Under Linux, unless you mount a partition or a device, the
system does not know of the existence of that partition or device. This
might not seem to be the easiest way to provide access to your partitions
or devices but it offers great flexibility. 
This kind of layout, known as the unified filesystem, does offer several
advantages over the approach that Windows uses. Let's take the example of
the /usr directory. This directory off the root directory contains most of
the system executables. With the Linux filesystem, you can choose to mount
it off another partition or even off another machine over the network. The
underlying system will not know the difference because /usr appears to be
a local directory that is part of the local directory structure! How many
times have you wished to move around executables and data under Windows,
only to run into registry and system errors? Try moving c:windowssystem
to another partition or drive. 
Another point likely to confuse newbies is the use of the frontslash '/'
instead of the backslash '' as in DOS/Windows. So c:windowssystem would
be /c/windows/system. Well, Linux is not going against convention here.
Unix has been around a lot longer than Windows and was the standard a lot
before Windows was. Rather, DOS took the different path, using '/' for
command-line options and '' as the directory separator. 
To liven up matters even more, Linux also chooses to be case sensitive.
What this means that the case, whether in capitals or not, of the
characters becomes very important. So this is not the same as THIS or ThIs
for that matter. This one feature probably causes the most problems for
newbies. 
We now move on to the layout or the directory structure of the Linux
filesystem. Given below is the result of a 'ls -p' in the root directory. 
bin/ dev/ home/ lost+found/ proc/ sbin/ usr/
boot/ etc/ lib/ mnt/ root/ tmp/ var/ 
/sbin - This directory contains all the binaries that are essential to the
working of the system. These include system administration as well as
maintenance and hardware configuration programs. Find lilo, fdisk, init,
ifconfig etc here. These are the essential programs that are required by
all the users. Another directory that contains system binaries is /usr/sbin.
This directory contains other binaries of use to the system administrator.
This is where you will find the network daemons for your system along with
other binaries that only the system administrator has access to, but which are
not required for system maintenance, repair etc. 
/bin - In contrast to /sbin, the bin directory contains several useful
commands that are used by both the system administrator as well as
non-privileged users. This directory usually contains the shells like
bash, csh etc. as well as much used commands like cp, mv, rm, cat, ls.
There also is /usr/bin, which contains other user binaries. These binaries
on the other hand are not essential for the user. The binaries in /bin
however, a user cannot do without. 
/boot - This directory contains the system.map file as well as the Linux
kernel. Lilo places the boot sector backups in this directory. 

The /proc File System


The /proc File System
The /proc file system really shows the power of the Linux Virtual File System. It does not really exist (yet another of Linux's conjuring tricks), neither the /proc directory nor its subdirectories and its files actually exist. So how can you cat/proc/devices? The /proc file system, like a real file system, registers itself with the Virtual File System. However, when the VFS makes calls to it requesting inodes as its files and directories are opened, the /procfile system creates those files and directories from information within the kernel. For example, the kernel's /proc/devices file is generated from the kernel's data structures describing its devices.
The /proc file system presents a user readable window into the kernel's inner workings. Several Linux subsystems, such as Linux kernel modules described in chapter  modules-chapter, create entries in the the /proc file system.

The bdflush Kernel Daemon


The bdflush Kernel Daemon
The bdflush kernel daemon is a simple kernel daemon that provides a dynamic response to the system having too many dirty buffers; buffers that contain data that must be written out to disk at some time. It is started as a kernel thread at system startup time and, rather confusingly, it calls itself ``kflushd'' and that is the name that you will see if you use the pscommand to show the processes in the system. Mostly this daemon sleeps waiting for the number of dirty buffers in the system to grow too large. As buffers are allocated and discarded the number of dirty buffers in the system is checked. If there are too many as a percentage of the total number of buffers in the system then bdflushis woken up. The default threshold is 60% but, if the system is desperate for buffers, bdflushwill be woken up anyway. This value can be seen and changed using the updatecommand:

# update -d

bdflush version 1.4
0:    60 Max fraction of LRU list to examine for dirty blocks
1:   500 Max number of dirty blocks to write each time bdflush activated
2:    64 Num of clean buffers to be loaded onto free list by refill_freelist
3:   256 Dirty block threshold for activating bdflush in refill_freelist
4:    15 Percentage of cache to scan for free clusters
5:  3000 Time for data buffers to age before flushing
6:   500 Time for non-data (dir, bitmap, etc) buffers to age before flushing
7:  1884 Time buffer cache load average constant
8:     2 LAV ratio (used to determine threshold for buffer fratricide).

All of the dirty buffers are linked into the BUF_DIRTYLRU list whenever they are made dirty by having data written to them and bdflush tries to write a reasonable number of them out to their owning disks. Again this number can be seen and controlled by the update command and the default is 500 (see above).

The update Process


The update Process
The update command is more than just a command; it is also a daemon. When run as superuser (during system initialisation) it will periodically flush all of the older dirty buffers out to disk. It does this by calling a system service routine
that does more or less the same thing as bdflush. Whenever a dirty buffer is finished with, it is tagged with the system time that it should be written out to its owning disk. Every time that update runs it looks at all of the dirty buffers in the system looking for ones with an expired flush time. Every expired buffer is written out to disk.

The VFS Inode Cache


The VFS Inode Cache
As the mounted file systems are navigated, their VFS inodes are being continually read and, in some cases, written. The Virtual File System maintains an inode cache to speed up accesses to all of the mounted file systems. Every time a VFS inode is read from the inode cache the system saves an access to a physical device.
The VFS inode cache is implmented as a hash table whose entries are pointers to lists of VFS inodes that have the same hash value. The hash value of an inode is calculated from its inode number and from the device identifier for the underlying physical device containing the file system. Whenever the Virtual File System needs to access an inode, it first looks in the VFS inode cache. To find an inode in the cache, the system first calculates its hash value and then uses it as an index into the inode hash table. This gives it a pointer to a list of inodes with the same hash value. It then reads each inode in turn until it finds one with both the same inode number and the same device identifier as the one that it is searching for.
If it can find the inode in the cache, its count is incremented to show that it has another user and the file system access continues. Otherwise a free VFS inode must be found so that the file system can read the inode from memory. VFS has a number of choices about how to get a free inode. If the system may allocate more VFS inodes then this is what it does; it allocates kernel pages and breaks them up into new, free, inodes and puts them into the inode list. All of the system's VFS inodes are in a list pointed at by first_inode as well as in the inode hash table. If the system already has all of the inodes that it is allowed to have, it must find an inode that is a good candidate to be reused. Good candidates are inodes with a usage count of zero; this indicates that the system is not currently using them. Really important VFS inodes, for example the root inodes of file systems always have a usage count greater than zero and so are never candidates for reuse. Once a candidate for reuse has been located it is cleaned up. The VFS inode might be dirty and in this case it needs to be written back to the file system or it might be locked and in this case the system must wait for it to be unlocked before continuing. The candidate VFS inode must be cleaned up before it can be reused.
However the new VFS inode is found, a file system specific routine must be called to fill it out from information read from the underlying real file system. Whilst it is being filled out, the new VFS inode has a usage count of one and is locked so that nothing else accesses it until it contains valid information.
To get the VFS inode that is actually needed, the file system may need to access several other inodes. This happens when you read a directory; only the inode for the final directory is needed but the inodes for the intermediate directories must also be read. As the VFS inode cache is used and filled up, the less used inodes will be discarded and the more used inodes will remain in the cache.

The Directory Cache


The Directory Cache
To speed up accesses to commonly used directories, the VFS maintains a cache of directory entries.
As directories are looked up by the real file systems their details are added into the directory cache. The next time the same directory is looked up, for example to list it or open a file within it, then it will be found in the directory cache. Only short directory entries (up to 15 characters long) are cached but this is reasonable as the shorter directory names are the most commonly used ones. For example, /usr/X11R6/bin is very commonly accessed when the X server is running.
The directory cache consists of a hash table, each entry of which points at a list of directory cache entries that have the same hash value. The hash function uses the device number of the device holding the file system and the directory's name to calculate the offset, or index, into the hash table. It allows cached directory entries to be quickly found. It is no use having a cache when lookups within the cache take too long to find entries, or even not to find them.
In an effort to keep the caches valid and up to date the VFS keeps lists of Least Recently Used (LRU) directory cache entries. When a directory entry is first put into the cache, which is when it is first looked up, it is added onto the end of the first level LRU list. In a full cache this will displace an existing entry from the front of the LRU list. As the directory entry is accessed again it is promoted to the back of the second LRU cache list. Again, this may displace a cached level two directory entry at the front of the level two LRU cache list. This displacing of entries at the front of the level one and level two LRU lists is fine. The only reason that entries are at the front of the lists is that they have not been recently accessed. If they had, they would be nearer the back of the lists. The entries in the second level LRU cache list are safer than entries in the level one LRU cache list. This is the intention as these entries have not only been looked up but also they have been repeatedly referenced.
REVIEW NOTE: Do we need a diagram for this?

The Buffer Cache


The Buffer Cache
Figure 9.7: The Buffer Cache
As the mounted file systems are used they generate a lot of requests to the block devices to read and write data blocks. All block data read and write requests are given to the device drivers in the form of buffer_head data structures via standard kernel routine calls. These give all of the information that the block device drivers need; the device identifier uniquely identifies the device and the block number tells the driver which block to read. All block devices are viewed as linear collections of blocks of the same size. To speed up access to the physical block devices, Linux maintains a cache of block buffers. All of the block buffers in the system are kept somewhere in this buffer cache, even the new, unused buffers. This cache is shared between all of the physical block devices; at any one time there are many block buffers in the cache, belonging to any one of the system's block devices and often in many different states. If valid data is available from the buffer cache this saves the system an access to a physical device. Any block buffer that has been used to read data from a block device or to write data to it goes into the buffer cache. Over time it may be removed from the cache to make way for a more deserving buffer or it may remain in the cache as it is frequently accessed.
Block buffers within the cache are uniquely identfied by the owning device identifier and the block number of the buffer. The buffer cache is composed of two functional parts. The first part is the lists of free block buffers. There is one list per supported buffer size and the system's free block buffers are queued onto these lists when they are first created or when they have been discarded. The currently supported buffer sizes are 512, 1024, 2048, 4096 and 8192 bytes. The second functional part is the cache itself. This is a hash table which is a vector of pointers to chains of buffers that have the same hash index. The hash index is generated from the owning device identifier and the block number of the data block. Figure  9.7 shows the hash table together with a few entries. Block buffers are either in one of the free lists or they are in the buffer cache. When they are in the buffer cache they are also queued onto Least Recently Used (LRU) lists. There is an LRU list for each buffer type and these are used by the system to perform work on buffers of a type, for example, writing buffers with new data in them out to disk. The buffer's type reflects its state and Linux currently supports the following types:
clean
Unused, new buffers,
locked
Buffers that are locked, waiting to be written,
dirty
Dirty buffers. These contain new, valid data, and will be written but so far have not been scheduled to write,
shared
Shared buffers,
unshared
Buffers that were once shared but which are now not shared,
Whenever a file system needs to read a buffer from its underlying physical device, it trys to get a block from the buffer cache. If it cannot get a buffer from the buffer cache, then it will get a clean one from the appropriate sized free list and this new buffer will go into the buffer cache. If the buffer that it needed is in the buffer cache, then it may or may not be up to date. If it is not up to date or if it is a new block buffer, the file system must request that the device driver read the appropriate block of data from the disk.
Like all caches, the buffer cache must be maintained so that it runs efficiently and fairly allocates cache entries between the block devices using the buffer cache. Linux uses the bdflush
kernel daemon to perform a lot of housekeeping duties on the cache but some happen automatically as a result of the cache being used.

Finding a File in the Virtual File System


Finding a File in the Virtual File System
To find the VFS inode of a file in the Virtual File System, VFS must resolve the name a directory at a time, looking up the VFS inode representing each of the intermediate directories in the name. Each directory lookup involves calling the file system specific lookup whose address is held in the VFS inode representing the parent directory. This works because we always have the VFS inode of the root of each file system available and pointed at by the VFS superblock for that system. Each time an inode is looked up by the real file system it checks the directory cache for the directory. If there is no entry in the directory cache, the real file system gets the VFS inode either from the underlying file system or from the inode cache.

Creating a File in the Virtual File System


Creating a File in the Virtual File System
9.2.7  Unmounting a File System
The workshop manual for my MG usually describes assembly as the reverse of disassembly and the reverse is more or less true for unmounting a file system.
A file system cannot be unmounted if something in the system is using one of its files. So, for example, you cannot umount/mnt/cdromif a process is using that directory or any of its children. If anything is using the file system to be unmounted there may be VFS inodes from it in the VFS inode cache, and the code checks for this by looking through the list of inodes looking for inodes owned by the device that this file system occupies. If the VFS superblock for the mounted file system is dirty, that is it has been modified, then it must be written back to the file system on disk. Once it has been written to disk, the memory occupied by the VFS superblock is returned to the kernel's free pool of memory. Finally the vfsmount data structure for this mount is unlinked from vfsmntlistand freed.

Registering the File Systems


Registering the File Systems
Figure 9.5: Registered File Systems
When you build the Linux kernel you are asked if you want each of the supported file systems. When the kernel is built, the file system startup code contains calls to the initialisation routines of all of the built in file systems.
Linux file systems may also be built as modules and, in this case, they may be demand loaded as they are needed or loaded by hand using insmod. Whenever a file system module is loaded it registers itself with the kernel and unregisters itself when it is unloaded. Each file system's initialisation routine registers itself with the Virtual File System and is represented by afile_system_type data structure which contains the name of the file system and a pointer to its VFS superblock read routine. Figure  9.5 shows that the file_system_type data structures are put into a list pointed at by thefile_systems pointer. Each file_system_type data structure contains the following information:
Superblock read routine
This routine is called by the VFS when an instance of the file system is mounted,
File System name
The name of this file system, for example ext2,
Device needed
Does this file system need a device to support? Not all file system need a device to hold them. The /procfile system, for example, does not require a block device,
You can see which file systems are registered by looking in at /proc/filesystems. For example:
      ext2
nodev proc
      iso9660

Mounting a File System


Mounting a File System
When the superuser attempts to mount a file system, the Linux kernel must first validate the arguments passed in the system call. Although mount does some basic checking, it does not know which file systems this kernel has been built to support or that the proposed mount point actually exists. Consider the following mount command:
$ mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
This mount command will pass the kernel three pieces of information; the name of the file system, the physical block device that contains the file system and, thirdly, where in the existing file system topology the new file system is to be mounted.
The first thing that the Virtual File System must do is to find the file system.
To do this it searches through the list of known file systems by looking at each file_system_type data structure in the list pointed at by file_systems.
If it finds a matching name it now knows that this file system type is supported by this kernel and it has the address of the file system specific routine for reading this file system's superblock. If it cannot find a matching file system name then all is not lost if the kernel is built to demand load kernel modules (see Chapter  modules-chapter). In this case the kernel will request that the kernel daemon loads the appropriate file system module before continuing as before.
Next if the physical device passed by mountis not already mounted, it must find the VFS inode of the directory that is to be the new file system's mount point. This VFS inode may be in the inode cache or it might have to be read from the block device supporting the file system of the mount point. Once the inode has been found it is checked to see that it is a directory and that there is not already some other file system mounted there. The same directory cannot be used as a mount point for more than one file system.
At this point the VFS mount code must allocate a VFS superblock and pass it the mount information to the superblock read routine for this file system. All of the system's VFS superblocks are kept in the super_blocks vector of super_blockdata structures and one must be allocated for this mount. The superblock read routine must fill out the VFS superblock fields based on information that it reads from the physical device. For the EXT2 file system this mapping or translation of information is quite easy, it simply reads the EXT2 superblock and fills out the VFS superblock from there. For other file systems, such as the MS DOS file system, it is not quite such an easy task. Whatever the file system, filling out the VFS superblock means that the file system must read whatever describes it from the block device that supports it. If the block device cannot be read from or if it does not contain this type of file system then the mount command will fail.
Figure 9.6: A Mounted File System
Each mounted file system is described by a vfsmount data structure; see figure  9.6. These are queued on a list pointed at byvfsmntlist.
Another pointer, vfsmnttail points at the last entry in the list and the mru_vfsmntpointer points at the most recently used file system. Each vfsmount structure contains the device number of the block device holding the file system, the directory where this file system is mounted and a pointer to the VFS superblock allocated when this file system was mounted. In turn the VFS superblock points at the file_system_type data structure for this sort of file system and to the root inode for this file system. This inode is kept resident in the VFS inode cache all of the time that this file system is loaded.

The Virtual File System (VFS)


The Virtual File System (VFS)
Figure 9.4: A Logical Diagram of the Virtual File System
Figure  9.4 shows the relationship between the Linux kernel's Virtual File System and it's real file systems. The virtual file system must manage all of the different file systems that are mounted at any given time. To do this it maintains data structures that describe the whole (virtual) file system and the real, mounted, file systems.
Rather confusingly, the VFS describes the system's files in terms of superblocks and inodes in much the same way as the EXT2 file system uses superblocks and inodes. Like the EXT2 inodes, the VFS inodes describe files and directories within the system; the contents and topology of the Virtual File System. From now on, to avoid confusion, I will write about VFS inodes and VFS superblocks to distinquish them from EXT2 inodes and superblocks.
As each file system is initialised, it registers itself with the VFS. This happens as the operating system initialises itself at system boot time. The real file systems are either built into the kernel itself or are built as loadable modules. File System modules are loaded as the system needs them, so, for example, if the VFAT file system is implemented as a kernel module, then it is only loaded when a VFAT file system is mounted. When a block device based file system is mounted, and this includes the root file system, the VFS must read its superblock. Each file system type's superblock read routine must work out the file system's topology and map that information onto a VFS superblock data structure. The VFS keeps a list of the mounted file systems in the system together with their VFS superblocks. Each VFS superblock contains information and pointers to routines that perform particular functions. So, for example, the superblock representing a mounted EXT2 file system contains a pointer to the EXT2 specific inode reading routine. This EXT2 inode read routine, like all of the file system specific inode read routines, fills out the fields in a VFS inode. Each VFS superblock contains a pointer to the first VFS inode on the file system. For the root file system, this is the inode that represents the ``/''directory. This mapping of information is very efficient for the EXT2 file system but moderately less so for other file systems.
As the system's processes access directories and files, system routines are called that traverse the VFS inodes in the system.
For example, typing ls for a directory or cat for a file cause the the Virtual File System to search through the VFS inodes that represent the file system. As every file and directory on the system is represented by a VFS inode, then a number of inodes will be being repeatedly accessed. These inodes are kept in the inode cache which makes access to them quicker. If an inode is not in the inode cache, then a file system specific routine must be called in order to read the appropriate inode. The action of reading the inode causes it to be put into the inode cache and further accesses to the inode keep it in the cache. The less used VFS inodes get removed from the cache.
All of the Linux file systems use a common buffer cache to cache data buffers from the underlying devices to help speed up access by all of the file systems to the physical devices holding the file systems.
This buffer cache is independent of the file systems and is integrated into the mechanisms that the Linux kernel uses to allocate and read and write data buffers. It has the distinct advantage of making the Linux file systems independent from the underlying media and from the device drivers that support them. All block structured devices register themselves with the Linux kernel and present a uniform, block based, usually asynchronous interface. Even relatively complex block devices such as SCSI devices do this. As the real file systems read data from the underlying physical disks, this results in requests to the block device drivers to read physical blocks from the device that they control. Integrated into this block device interface is the buffer cache. As blocks are read by the file systems they are saved in the global buffer cache shared by all of the file systems and the Linux kernel. Buffers within it are identified by their block number and a unique identifier for the device that read it. So, if the same data is needed often, it will be retrieved from the buffer cache rather than read from the disk, which would take somewhat longer. Some devices support read ahead where data blocks are speculatively read just in case they are needed.
The VFS also keeps a cache of directory lookups so that the inodes for frequently used directories can be quickly found.
As an experiment, try listing a directory that you have not listed recently. The first time you list it, you may notice a slight pause but the second time you list its contents the result is immediate. The directory cache does not store the inodes for the directories itself; these should be in the inode cache, the directory cache simply stores the mapping between the full directory names and their inode numbers.

Finding a File in an EXT2 File System


5  Finding a File in an EXT2 File System
A Linux filename has the same format as all Unix TM filenames have. It is a series of directory names separated by forward slashes (``/'') and ending in the file's name. One example filename would be /home/rusling/.cshrc where /home and/rusling are directory names and the file's name is .cshrc. Like all other Unix TM systems, Linux does not care about the format of the filename itself; it can be any length and consist of any of the printable characters. To find the inode representing this file within an EXT2file system the system must parse the filename a directory at a time until we get to the file itself.
The first inode we need is the inode for the root of the file system and we find its number in the file system's superblock. To read an EXT2 inode we must look for it in the inode table of the appropriate Block Group. If, for example, the root inode number is 42, then we need the 42nd inode from the inode table of Block Group 0. The root inode is for an EXT2 directory, in other words the mode of the root inode describes it as a directory and it's data blocks contain EXT2 directory entries.
home is just one of the many directory entries and this directory entry gives us the number of the inode describing the/homedirectory. We have to read this directory (by first reading its inode and then reading the directory entries from the data blocks described by its inode) to find the ruslingentry which gives us the number of the inode describing the/home/rusling directory. Finally we read the directory entries pointed at by the inode describing the /home/ruslingdirectory to find the inode number of the .cshrcfile and from this we get the data blocks containing the information in the file.

Changing the Size of a File in an EXT2 File System


Changing the Size of a File in an EXT2 File System
One common problem with a file system is its tendency to fragment. The blocks that hold the file's data get spread all over the file system and this makes sequentially accessing the data blocks of a file more and more inefficient the further apart the data blocks are. The EXT2 file system tries to overcome this by allocating the new blocks for a file physically close to its current data blocks or at least in the same Block Group as its current data blocks. Only when this fails does it allocate data blocks in another Block Group.
Whenever a process attempts to write data into a file the Linux file system checks to see if the data has gone off the end of the file's last allocated block. If it has, then it must allocate a new data block for this file. Until the allocation is complete, the process cannot run; it must wait for the file system to allocate a new data block and write the rest of the data to it before it can continue. The first thing that the EXT2 block allocation routines do is to lock the EXT2 Superblock for this file system. Allocating and deallocating changes fields within the superblock, and the Linux file system cannot allow more than one process to do this at the same time. If another process needs to allocate more data blocks, it will have to wait until this process has finished. Processes waiting for the superblock are suspended, unable to run, until control of the superblock is relinquished by its current user. Access to the superblock is granted on a first come, first served basis and once a process has control of the superblock, it keeps control until it has finished. Having locked the superblock, the process checks that there are enough free blocks left in this file system. If there are not enough free blocks, then this attempt to allocate more will fail and the process will relinquish control of this file system's superblock.
If there are enough free blocks in the file system, the process tries to allocate one.
If the EXT2 file system has been built to preallocate data blocks then we may be able to take one of those. The preallocated blocks do not actually exist, they are just reserved within the allocated block bitmap. The VFS inode representing the file that we are trying to allocate a new data block for has two EXT2 specific fields, prealloc_block and prealloc_count, which are the block number of the first preallocated data block and how many of them there are, respectively. If there were no preallocated blocks or block preallocation is not enabled, the EXT2 file system must allocate a new block. The EXT2 file system first looks to see if the data block after the last data block in the file is free. Logically, this is the most efficient block to allocate as it makes sequential accesses much quicker. If this block is not free, then the search widens and it looks for a data block within 64 blocks of the of the ideal block. This block, although not ideal is at least fairly close and within the same Block Group as the other data blocks belonging to this file.
If even that block is not free, the process starts looking in all of the other Block Groups in turn until it finds some free blocks. The block allocation code looks for a cluster of eight free data blocks somewhere in one of the Block Groups. If it cannot find eight together, it will settle for less. If block preallocation is wanted and enabled it will update prealloc_blockandprealloc_countaccordingly.
Wherever it finds the free block, the block allocation code updates the Block Group's block bitmap and allocates a data buffer in the buffer cache. That data buffer is uniquely identified by the file system's supporting device identifier and the block number of the allocated block. The data in the buffer is zero'd and the buffer is marked as ``dirty'' to show that it's contents have not been written to the physical disk. Finally, the superblock itself is marked as ``dirty'' to show that it has been changed and it is unlocked. If there were any processes waiting for the superblock, the first one in the queue is allowed to run again and will gain exclusive control of the superblock for its file operations. The process's data is written to the new data block and, if that data block is filled, the entire process is repeated and another data block allocated.

The EXT2 Group Descriptor


The EXT2 Group Descriptor
Each Block Group has a data structure describing it. Like the Superblock, all the group descriptors for all of the Block Groups are duplicated in each Block Group in case of file system corruption.
Each Group Descriptor contains the following information:
Blocks Bitmap
The block number of the block allocation bitmap for this Block Group. This is used during block allocation and deallocation,
Inode Bitmap
The block number of the inode allocation bitmap for this Block Group. This is used during inode allocation and deallocation,
Inode Table
The block number of the starting block for the inode table for this Block Group. Each inode is represented by the EXT2 inode data structure described below.
Free blocks count, Free Inodes count, Used directory count
The group descriptors are placed on after another and together they make the group descriptor table. Each Blocks Group contains the entire table of group descriptors after its copy of the Superblock. Only the first copy (in Block Group 0) is actually used by the EXT2 file system. The other copies are there, like the copies of the Superblock, in case the main copy is corrupted.

EXT2 Directories


EXT2 Directories
Figure 9.3: EXT2 Directory
In the EXT2 file system, directories are special files that are used to create and hold access paths to the files in the file system. Figure  9.3 shows the layout of a directory entry in memory.
A directory file is a list of directory entries, each one containing the following information:
inode
The inode for this directory entry. This is an index into the array of inodes held in the Inode Table of the Block Group. In figure  9.3, the directory entry for the file called file has a reference to inode number i1,
name length
The length of this directory entry in bytes,
name
The name of this directory entry.
The first two entries for every directory are always the standard ``.'' and ``..'' entries meaning ``this directory'' and ``the parent directory'' respectively.

The Second Extended File system (EXT2)


9.1  The Second Extended File system (EXT2)
Figure 9.1: Physical Layout of the EXT2 File system
The Second Extended File system was devised (by Rémy Card) as an extensible and powerful file system for Linux. It is also the most successful file system so far in the Linux community and is the basis for all of the currently shipping Linux distributions.
The EXT2 file system, like a lot of the file systems, is built on the premise that the data held in files is kept in data blocks. These data blocks are all of the same length and, although that length can vary between different EXT2 file systems the block size of a particular EXT2 file system is set when it is created (using mke2fs). Every file's size is rounded up to an integral number of blocks. If the block size is 1024 bytes, then a file of 1025 bytes will occupy two 1024 byte blocks. Unfortunately this means that on average you waste half a block per file. Usually in computing you trade off CPU usage for memory and disk space utilisation. In this case Linux, along with most operating systems, trades off a relatively inefficient disk usage in order to reduce the workload on the CPU. Not all of the blocks in the file system hold data, some must be used to contain the information that describes the structure of the file system. EXT2 defines the file system topology by describing each file in the system with an inode data structure. An inode describes which blocks the data within a file occupies as well as the access rights of the file, the file's modification times and the type of the file. Every file in the EXT2 file system is described by a single inode and each inode has a single unique number identifying it. The inodes for the file system are all kept together in inode tables. EXT2 directories are simply special files (themselves described by inodes) which contain pointers to the inodes of their directory entries.
Figure  9.1 shows the layout of the EXT2 file system as occupying a series of blocks in a block structured device. So far as each file system is concerned, block devices are just a series of blocks that can be read and written. A file system does not need to concern itself with where on the physical media a block should be put, that is the job of the device's driver. Whenever a file system needs to read information or data from the block device containing it, it requests that its supporting device driver reads an integral number of blocks. The EXT2 file system divides the logical partition that it occupies into Block Groups.
Each group duplicates information critical to the integrity of the file system as well as holding real files and directories as blocks of information and data. This duplication is neccessary should a disaster occur and the file system need recovering. The subsections describe in more detail the contents of each Block Group.

The file system in reality


The file system in reality
For most users and for most common system administration tasks, it is enough to accept that files and directories are ordered in a tree-like structure. The computer, however, doesn't understand a thing about trees or tree-structures.
Every partition has its own file system. By imagining all those file systems together, we can form an idea of the tree-structure of the entire system, but it is not as simple as that. In a file system, a file is represented by an inode, a kind of serial number containing information about the actual data that makes up the file: to whom this file belongs, and where is it located on the hard disk.
Every partition has its own set of inodes; throughout a system with multiple partitions, files with the same inode number can exist.
Each inode describes a data structure on the hard disk, storing the properties of a file, including the physical location of the file data. When a hard disk is initialized to accept data storage, usually during the initial system installation process or when adding extra disks to an existing system, a fixed number of inodes per partition is created. This number will be the maximum amount of files, of all types (including directories, special files, links etc.) that can exist at the same time on the partition. We typically count on having 1 inode per 2 to 8 kilobytes of storage.
At the time a new file is created, it gets a free inode. In that inode is the following information:
  • Owner and group owner of the file.
  • File type (regular, directory, ...)
  • Permissions on the file Section 3.4.1
  • Date and time of creation, last read and change.
  • Date and time this information has been changed in the inode.
  • Number of links to this file (see later in this chapter).
  • File size
  • An address defining the actual location of the file data.
The only information not included in an inode, is the file name and directory. These are stored in the special directory files. By comparing file names and inode numbers, the system can make up a tree-structure that the user understands. Users can display inode numbers using the -i option to ls. The inodes have their own separate space on the disk.


Chapter 9 The File system


Chapter 9
The File system

This chapter describes how the Linux kernel maintains the files in the file systems that it supports. It describes the Virtual File System (VFS) and explains how the Linux kernel's real file systems are supported.
One of the most important features of Linux is its support for many different file systems. This makes it very flexible and well able to coexist with many other operating systems. At the time of writing, Linux supports 15 file systems; ext, ext2, xia,minix, umsdos, msdos, vfat, proc, smb, ncp, iso9660, sysv, hpfs, affs and ufs, and no doubt, over time more will be added.
In Linux, as it is for Unix TM, the separate file systems the system may use are not accessed by device identifiers (such as a drive number or a drive name) but instead they are combined into a single hierarchical tree structure that represents the file system as one whole single entity. Linux adds each new file system into this single file system tree as it is mounted. All file systems, of whatever type, are mounted onto a directory and the files of the mounted file system cover up the existing contents of that directory. This directory is known as the mount directory or mount point. When the file system is unmounted, the mount directory's own files are once again revealed.
When disks are initialized (using fdisk, say) they have a partition structure imposed on them that divides the physical disk into a number of logical partitions. Each partition may hold a single file system, for example an EXT2file system. File systems organize files into logical hierarchical structures with directories, soft links and so on held in blocks on physical devices. Devices that can contain file systems are known as block devices. The IDE disk partition /dev/hda1, the first partition of the first IDE disk drive in the system, is a block device. The Linux file systems regard these block devices as simply linear collections of blocks, they do not know or care about the underlying physical disk's geometry. It is the task of each block device driver to map a request to read a particular block of its device into terms meaningful to its device; the particular track, sector and cylinder of its hard disk where the block is kept. A file system has to look, feel and operate in the same way no matter what device is holding it. Moreover, using Linux's file systems, it does not matter (at least to the system user) that these different file systems are on different physical media controlled by different hardware controllers. The file system might not even be on the local system, it could just as well be a disk remotely mounted over a network link. Consider the following example where a Linux system has its root file system on a SCSI disk:
A         E         boot      etc       lib       opt       tmp       usr
C         F         cdrom     fd        proc      root      var       sbin
D         bin       dev       home      mnt       lost+found
Neither the users nor the programs that operate on the files themselves need know that /C is in fact a mounted VFAT file system that is on the first IDE disk in the system. In the example (which is actually my home Linux system), /E is the master IDE disk on the second IDE controller. It does not matter either that the first IDE controller is a PCI controller and that the second is an ISA controller which also controls the IDE CDROM. I can dial into the network where I work using a modem and the PPP network protocol using a modem and in this case I can remotely mount my Alpha AXP Linux system's file systems on/mnt/remote.
The files in a file system are collections of data; the file holding the sources to this chapter is an ASCII file calledfilesystems.tex. A file system not only holds the data that is contained within the files of the file system but also the structure of the file system. It holds all of the information that Linux users and processes see as files, directories soft links, file protection information and so on. Moreover it must hold that information safely and securely, the basic integrity of the operating system depends on its file systems. Nobody would use an operating system that randomly lost data and files1.
Minix, the first file system that Linux had is rather restrictive and lacking in performance.
Its filenames cannot be longer than 14 characters (which is still better than 8.3 filenames) and the maximum file size is 64MBytes. 64Mbytes might at first glance seem large enough but large file sizes are necessary to hold even modest databases. The first file system designed specifically for Linux, the Extended File system, or EXT, was introduced in April 1992 and cured a lot of the problems but it was still felt to lack performance.
So, in 1993, the Second Extended File system, or EXT2, was added.
It is this file system that is described in detail later on in this chapter.
An important development took place when the EXT file system was added into Linux. The real file systems were separated from the operating system and system services by an interface layer known as the Virtual File system, or VFS.
VFS allows Linux to support many, often very different, file systems, each presenting a common software interface to the VFS. All of the details of the Linux file systems are translated by software so that all file systems appear identical to the rest of the Linux kernel and to programs running in the system. Linux's Virtual File system layer allows you to transparently mount the many different file systems at the same time.
The Linux Virtual File system is implemented so that access to its files is as fast and efficient as possible. It must also make sure that the files and their data are kept correctly. These two requirements can be at odds with each other. The Linux VFS caches information in memory from each file system as it is mounted and used. A lot of care must be taken to update the file system correctly as data within these caches is modified as files and directories are created, written to and deleted. If you could see the file system's data structures within the running kernel, you would be able to see data blocks being read and written by the file system. Data structures, describing the files and directories being accessed would be created and destroyed and all the time the device drivers would be working away, fetching and saving data. The most important of these caches is the Buffer Cache, which is integrated into the way that the individual file systems access their underlying block devices. As blocks are accessed they are put into the Buffer Cache and kept on various queues depending on their states. The Buffer Cache not only caches data buffers, it also helps manage the asynchronous interface with the block device drivers.

More file system layout


More file system layout
3.1.3.1. Visual
For convenience, the Linux file system is usually thought of in a tree structure. On a standard Linux system you will find the layout generally follows the scheme presented below.
Figure 3-1. Linux file system layout
This is a layout from a RedHat system. Depending on the system admin, the operating system and the mission of the UNIX machine, the structure may vary, and directories may be left out or added at will. The names are not even required; they are only a convention.
The tree of the file system starts at the trunk or slash, indicated by a forward slash (/). This directory, containing all underlying directories and files, is also called the root directoryor "the root" of the file system.
Directories that are only one level below the root directory are often preceded by a slash, to indicate their position and prevent confusion with other directories that could have the same name. When starting with a new system, it is always a good idea to take a look in the root directory. Let's see what you could run into:
emmy:~>cd /
emmy:/>ls
bin/   dev/  home/    lib/         misc/  opt/     root/  tmp/  var/
boot/  etc/  initrd/  lost+found/  mnt/   proc/    sbin/  usr/

Subdirectories of the root directory


Table 3-2. Subdirectories of the root directory
Directory
Content
/bin
Common programs, shared by the system, the system administrator and the users.
/boot
The startup files and the kernel, vmlinuz. In recent distributions also grub data. Grub is the GRand Unified Boot loader and is an attempt to get rid of the many different boot-loaders we know today.
/dev
Contains references to all the CPU peripheral hardware, which are represented as files with special properties.
/etc
Most important system configuration files are in /etc, this directory contains data similar to those in the Control Panel in Windows
/home
Home directories of the common users.
/initrd
(on some distributions) Information for booting. Do not remove!
/lib
Library files, includes files for all kinds of programs needed by the system and the users.
/lost+found
Every partition has a lost+found in its upper directory. Files that were saved during failures are here.
/misc
For miscellaneous purposes.
/mnt
Standard mount point for external file systems, e.g. a CD-ROM or a digital camera.
/net
Standard mount point for entire remote file systems
/opt
Typically contains extra and third party software.
/proc
A virtual file system containing information about system resources. More information about the meaning of the files in proc is obtained by entering the command man proc in a terminal window. The file proc.txt discusses the virtual file system in detail.
/root
The administrative user's home directory. Mind the difference between /, the root directory and /root, the home directory of the root user.
/sbin
Programs for use by the system and the system administrator.
/tmp
Temporary space for use by the system.
/usr
Programs, libraries, documentation etc. for all user-related programs.
/var
Storage for all variable files and temporary files created by users, such as log files, the mail queue, the print spooler area, space for temporary storage of files downloaded from the Internet, or to keep an image of a CD before burning it.
How can you find out which partition a directory is on? Using the df command with a dot (.) as an option shows the partition the current directory belongs to, and informs about the amount of space used on this partition:
sandra:/lib>df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda7             980M  163M  767M  18% /
As a general rule, every directory under the root directory is on the root partition, unless it has a separate entry in the full listing from df (or df -h with no other options).

Partition layout and types


Partition layout and types
There are two kinds of major partitions on a Linux system:
  • data partition: normal Linux system data, including the root partition containing all the data to start up and run the system; and
  • swap partition: expansion of the computer's physical memory, extra memory on hard disk.
Most systems contain a root partition, one or more data partitions and one or more swap partitions. Systems in mixed environments may contain partitions for other system data, such as a partition with a FAT or VFAT file system for MS Windows data.
Most Linux systems use fdisk at installation time to set the partition type. As you may have noticed during the exercise from Chapter 1, this usually happens automatically. On some occasions, however, you may not be so lucky. In such cases, you will need to select the partition type manually and even manually do the actual partitioning. The standard Linux partitions have number 82 for swap and 83 for data, which can be journaled (ext3) or normal (ext2, on older systems). The fdisk utility has built-in help, should you forget these values.
The standard root partition is about 100-500 MB, and contains the system configuration files, most basic commands and server programs, system libraries, some temporary space and the home directory of the administrative user. A standard installation requires about 250 MB for the root partition.
Swap space is only accessible for the system itself, and is hidden from view during normal operation. Swap is the system that ensures, like on normal UNIX systems, that you can keep on working, whatever happens. On Linux, you will never see irritating messages like Out of memory, please close some applications first and try again, because of this extra memory. Using memory on a hard disk is naturally slower than using the real memory chips of a computer, but having this little extra is a great comfort. We will learn more about swap when we discuss Processes in Chapter 4.
Linux generally counts on having twice the amount of physical memory in the form of swap space on the hard disk. When installing a system, you have to know how you are going to do this. An example on a system with 512 MB of RAM:
  • 1st possibility: one swap partition of 1 GB
  • 2nd possibility: two swap partitions of 512 MB
  • 3rd possibility: with two hard disks: 1 partition of 512 MB on each disk.
The last option will give the best results when a lot of I/O is to be expected.
Read the software documentation for specific guidelines. Some applications, such as databases, might require more swap space. Others, such as some handheld systems, might not have any swap at all by lack of a hard disk. Swap space may also depend on your kernel version.
The rest of the hard disk(s) is generally divided in data partitions, although it may be that all of the non-system critical data resides on one partition, for example when you perform a standard workstation installation. When non-critical data is separated on different partitions, it usually happens following a set pattern:
  • a partition for user programs
  • a partition containing the users' personal data
  • a partition to store temporary data like print- and mail-queues
  • a partition for third party and extra software
Once the partitions are made, you can only add more. Changing sizes or properties of existing partitions is possible but not advisable.
The division of hard disks into partitions is determined by the system administrator. On larger systems, he or she may even spread one partition over several hard disks, using the appropriate software. Most distributions allow for standard setups optimized for workstations (average users) and for general server purposes, but also accept customized partitions. During the installation process you can define your own partition layout using either Disk Druid, a straight forward graphical tool, orfdisk, a text-based tool for creating partitions and setting their properties.
A workstation or client installation is for use by mainly one and the same person. The selected software for installation reflects this and the stress is on common user packages, such as nice desktop themes, development tools, client programs for E-mail, multimedia software, web and other services. Everything is put together on one large partition, swap space twice the amount of RAM is added and your generic workstation is complete, providing the largest amount of disk space possible for personal use, but with the disadvantage of possible data integrity loss during problem situations.
On a server, system data tends to be separate from user data. Programs that offer services are kept in a different place than the data handled by this service. Different partitions will be created on such systems:
  • a partition with all data necessary to boot the machine
  • a partition with configuration data and server programs
  • one or more partitions containing the server data such as database tables, user mails, an ftp archive etc.
  • a partition with user programs and application
  • one or more partitions for the user specific files (home directories)
  • one or more swap partitions (virtual memory)
Servers usually have more memory and thus more swap space. Certain server processes, such as databases, may require more swap space than usual; see the specific documentation for detailed information. For better performance, swap is often divided into different swap partitions.

The df command


3.1.2.3. The df command
On a running system, information about the partitions can be displayed using the df command (which stands for disk full). In Linux, df is the GNU version, and supports the -h or human readable option which greatly improves readability. Note that commercial UNIX machines commonly have their own versions of df and many other commands. Their behavior is usually the same, though GNU versions of common tools often have more and better features.
The df command only displays information about active non-swap partitions. These can include partitions from other networked systems, like in the example below where the home directories are mounted from a file server on the network, a situation often encountered in corporate environments.
freddy:~>df -h
Filesystem          Size  Used Avail Use% Mounted on
/dev/hda8           496M  183M  288M  39% /
/dev/hda1           124M  8.4M  109M   8% /boot
/dev/hda5            19G   15G  2.7G  85% /opt
/dev/hda6           7.0G  5.4G  1.2G  81% /usr
/dev/hda7           3.7G  2.7G  867M  77% /var
fs1:/home           8.9G  3.7G  4.7G  44% /.automount/fs1/root/home
Partitions are mountedon a mount point, which can be almost any directory in the system. In the next section, we'll take a closer look at all those directories.

Table 3-1. File types


Table 3-1. File types
Symbol
Meaning
-
Regular file
d
Directory
l
Link
c
Special file
s
Socket
p
Named pipe
Before we look at the important files and directories, we need to know more about partitions.

About partitioning


3.1.2. About partitioning
3.1.2.1. Why partition?
Most people have a vague knowledge of what partitions are, since almost every operating system has the ability to create or remove them. Since Linux uses more than one partition on the same disk, even when using the standard installation procedure, this may seem strange at first.
The goal of having different partitions is primarily to achieve higher data security in case of disaster. By dividing the hard disk in partitions, data can be grouped and separated. When an accident occurs, only the data in the partition that got the hit will be damaged, while the data on the other partitions will most likely survive.
This principle dates from the days when Linux didn't have journaled file systems and power failures might have lead to disaster. The use of partitions remains for security and robustness reasons, so a breach on one part of the system doesn't automatically mean that the whole computer is in danger. This is currently the most important reason for partitioning.
Mind that having a journaled file system only provides data security in case of power failure and sudden disconnection of storage devices. This does not protect your data against bad blocks and logical errors in the file system. In those cases, you should use a RAID (Redundant Array of Inexpensive Disks) solution.

3.1.1.2. Sorts of files


3.1.1.2. Sorts of files
Most files are just files, called regular files; they contain normal data, for example text files, executable files or programs, input for or output from a program and so on.
While it is reasonably safe to suppose that everything you encounter on a Linux system is a file, there are some exceptions.
  • Directories: files that are lists of other files.
  • Special files: the mechanism used for input and output. Most special files are in /dev, we will discuss them later.
  • Links: a system to make a file or directory visible in multiple parts of the system's file tree. We will talk about links in detail.
  • (Domain) sockets: a special file type, similar to TCP/IP sockets, providing inter-process networking protected by the file system's access control.
  • Named pipes: act more or less like sockets and form a way for processes to communicate with each other, without using network socket semantics.
The -l option to ls displays the file type, using the first character of each input line:
jaime:~/Documents>ls -l
total 80
-rw-rw-r--   1 jaime   jaime   31744 Feb 21 17:56 intro Linux.doc
-rw-rw-r--   1 jaime   jaime   41472 Feb 21 17:56 Linux.doc
drwxrwxr-x   2 jaime   jaime    4096 Feb 25 11:50 course/
This table gives an overview of the characters determining the file type:

Longevity


Longevity
The open source nature of Linux guarantees it is here to stay, and the amazing growth of Linux over the past years bears that out. Best of all, as long as you stick with a truely free/open source operating system like Linux and truely free/open source applications, you can never get locked into depending on any particular vendor. Linux puts you in control of what you do with your software, how, when and if you choose to change or upgrade it.

General overview of the Linux file system 3.1.1. Files


General overview of the Linux file system
3.1.1. Files
3.1.1.1. General
A simple description of the UNIX system, also applicable to Linux, is this:
"On a UNIX system, everything is a file; if something is not a file, it is a process."
This statement is true because there are special files that are more than just files (named pipes and sockets, for instance), but to keep things simple, saying that everything is a file is an acceptable generalization. A Linux system, just like UNIX, makes no difference between a file and a directory, since a directory is just a file containing names of other files. Programs, services, texts, images, and so forth, are all files. Input and output devices, and generally all devices, are considered to be files, according to the system.
In order to manage all those files in an orderly fashion, man likes to think of them in an ordered tree-like structure on the hard disk, as we know from MS-DOS (Disk Operating System) for instance. The large branches contain more branches, and the branches at the end contain the tree's leaves or normal files. For now we will use this image of the tree, but we will find out later why this is not a fully accurate image.

Software Development


Software Development
Programmers often find that the Linux development environment is second to none--a good thing for end users who depend on these software developers to provide free software. Nearly all development software for Linux is free and covered under the GNU Public License, which guarantees that it will always remain free. Linux systems come standard with C and C++ compilers and an assembler, and usually include Pascal, FORTRAN, compiled Java, Perl, Python, and BASIC implementations as well. In addition, modern languages like Ruby and classic languages like LISP are all available, fully functional and completely free.
Linux runs two of the most popular development environments, Eclipse and KDevelop, and you can use these environments to with just about any programming language available. These two development tools support web application development, but there are additional free/open source highly sophisticated development tools dedicated to building web applications.
In addition, the source code for nearly any Linux program is freely available (and often included by default). This not only means that bugs are discovered and corrected almost immediately, but development of software proceeds at a much faster pace than one finds even at extremely successful commercial software houses. This phenomenon is called Open Source and is the subject of much discussion and amazement in the business world, the computer world, and the press.
The Open Source nature of Linux also makes it ideal for embedded and specialized systems (routers, cell phones, multimedia entertainment centers, point-of-sale systems), because there's no limit to what you can do to customize Linux for your special needs.

Networking


Networking
Networking comes naturally to Linux. Probably all networking protocols in use on the Internet are native to UNIX and/or Linux, so one can expect that UNIX and Linux would network better than any other platforms. Setting up a network on a Linux machine is surprisingly simple, because Linux handles most of the work.
A large part of the Web is running on Linux boxes, especially because of the Apache Web Server which dramatically defeated its commercial competitors, proving the effectiveness and viability of the Open Source approach

Productivity


Productivity
Productivity software availability has exploded in recent years, and commercial developers have been producing excellent software for the Linux platform. The Firefox browser, Opera, and Mozilla are freely available (with some licensing restrictions) as well as the OpenOffice productivity suite, KOffice and a host of others, which often come standard on Linux distributions. Many distributors package commercial software with their distributions, and many commercial producers offer free downloads for Linux. Linux productivity packages can usually read and write files from productivity packages on other platforms; Linux has always been at the leading edge of compatibility and openness.
Linux happily coexists on the same machine as other operating systems including Windows or Mac OSX, and Linux easily accesses the files stored by other operating systems. You can use one of many virtualization techniques to run Linux and Windows or any other operating system (even another version of Linux) on the same machine, simultaneously. You can run many Windows programs on Linux via Wine, or commercial helper products such as Crossover Office or Cedega, both of which even support the popular game Word of Warcraft! There are countless Linux distributions which run beautifully from a CD or DVD without the need to install the operating system. This makes it possible for new Linux users to see if they like Linux without erasing their old OS or having to buy another computer.

Speed


Speed
So much of the web is built on Linux that the acronym LAMP has emerged. LAMP represents Linux, Apache (web server), MySQL (database) and PHP (web application language). This acronym may need to be changed eventually due to the rapid growth of PostgreSQL, Ruby, and Java on Linux web servers.
Unlike some commercial operating systems, no free Linux distributions impose any artificial constraints on how you use the operating system. There are no arbitrary limits to the number of user accounts you can create, the number of simultaneous connections your Linux-based web server can handle, or arbitrary limits any other Linux resources.
Linux machines are known to be extremely fast, because the operating system is very efficient at managing resources such as memory, CPU power, and disk space. NASA, Sandia, Fermilabs and many others have built very powerful yet inexpensive supercomputers by creating clusters of Linux boxes running in parallel. Clusters of Linux systems have been responsible for rendering the graphics for movies like Shrek, Titanic, and many others.
Many high-profile organizations have adopted Linux. For example, visit the NOAA (the National Weather Service at www.srh.noaa.gov) and you can thank Linux for the weather reports you will see online.

Stability


Stability
Linux has long been praised for its stability--Linux boxes are known for running months or even years at a time without crashing, freezing, or having to be rebooted. Linux users sometimes poke fun at other, less stable operating systems, by way of screensavers like BSOD (Blue Screen of Death, which displays crash screens from various other platforms).
Linux is extremely secure compared to other platforms. Viruses and Trojan Horse programs are practically non-existent. Linux servers practically run the World Wide Web, so one cannot argue that there are so few malicious programs for Linux because it represents an insignificant number of target machines.

Monday, 21 May 2012

The Linux System


The Linux System
The central nervous system of Linux is the kernel, the operating system code which runs the whole computer. The kernel is under constant development and is always available in both the latest stable release and the latest experimental release. Progress on development is very fast, and the recent 2.6-series kernels are simply amazing on all counts. The kernel design is modular, so that the actual OS code is very small yet able to load whatever functionality it needs when it needs it. Because of this, the kernel remains small and fast yet highly extensible, in comparison to other operating systems which slow down the computer and waste memory by loading everything all the time, whether it is needed or not.
Linux systems excel in many areas, ranging from end-user concerns such as stability, speed, and ease of use, to serious concerns such as development and networking. Nowadays, Linux even offers a wide variety of free and commercial productivity packages such as the OpenOffice suite which can import and export files from other platforms, including Windows and MacOS.

What is Linux


What is Linux?
Linux, also known as GNU/Linux, is a free, UNIX-like operating system, developed originally for home PCs, but which now runs on practically every hardware platform available including PowerPC, Macintosh, DEC Alpha, Sun Sparc, ARM, Mainframes, and many others. Linux aims for POSIX compliancy to maintain maximum compatibility with other UNIX-like systems. With millions of users worldwide, Linux is probably the most popular UNIX-like OS in the world.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)