Files and Directory Manipulation
Chapter 04
Now, it's time to dive into the actual tasks! In this chapter, we will acquaint you with the subsequent commands:
cp– Copy files and directoriesmv– Move/rename files and directoriesmkdir– Create directoriesrm– Remove files and directoriesln– Create hard and symbolic links
Among the commonly employed Linux commands, these five hold significant importance. They serve the purpose of manipulating both files and directories.
To be honest, certain tasks accomplished by these commands can be more conveniently executed using a graphical file manager. With a file manager, we have the ability to effortlessly drag and drop files between directories, perform cut and paste operations, delete files, and more. Hence, one might question the rationale behind utilizing these traditional command line programs.
The key lies in the advantages of power and flexibility. Although performing basic file manipulations is effortless using a graphical file manager, complex tasks become more manageable with command line programs. Consider the scenario where we need to copy all HTML files from one directory to another, but only those that are not already present in the destination directory or are newer than the existing versions. Achieving this becomes quite challenging using a file manager, but remarkably straightforward through the command line:
Wildcards
Prior to diving into the utilization of our commands, it is crucial to discuss a shell feature that amplifies the power of these commands. Given the frequent usage of filenames within the shell, it offers special characters known as wildcards, designed to expedite the specification of groups of filenames. By employing wildcards, also referred to as globbing, you can conveniently select filenames based on patterns of characters. The table provided below outlines the various wildcards and their corresponding selections:
*
Matches any characters
?
Matches any single character
[characters]
Matches any character that is a member of the set characters
[!characters]
Matches any character that is not a member of the set characters
[[:class:]]
Matches any character that is a member of the specified class
Following are the most commonly used character classes:
[:alnum:]
Matches any alphanumeric character
[:alpha:]
Matches any alphabetic character
[:digit:]
Matches any numeral
[:lower:]
Matches any lowercase letter
[:upper:]
Matches any uppercase letter
The utilization of wildcards enables the creation of intricate selection criteria for filenames. Presented below are a few examples of patterns and the corresponding matches they produce:
*
All files
g*
Any file beginning with “g”
b*.txt
Any file beginning with “b” followed by any characters and ending with “.txt”
Data???
Any file beginning with “Data” followed by exactly three characters
[abc]*
Any file beginning with either an “a”, a “b”, or a “c”
BACKUP.[0-9][0-9][0-9]
Any file beginning with “BACKUP.” followed by exactly three numerals
[[:upper:]]*
Any file beginning with an uppercase letter
[![:digit:]]*
Any file not beginning with a numeral
*[[:lower:]123]
Any file ending with a lowercase letter or the numerals “1”, “2”, or “3”
Wildcards can be applied with any command that accepts filenames as arguments, but we will delve deeper into this topic in Chapter 7.
Character Ranges
If you have experience with other Unix-like environments or have explored other literature on this topic, you might have come across the [A-Z] or [a-z] character range notations. These notations are traditional Unix conventions and were functional in earlier versions of Linux as well. While they can still be employed, caution must be exercised as they may not yield the anticipated outcomes unless appropriately configured. At present, it is advisable to refrain from using them and instead utilize character classes as a suitable alternative.
mkdir – Create Directories
mkdir – Create DirectoriesTo create directories, the mkdir command is employed. Its functionality operates in the following manner:
Regarding notation, when an argument in the command description is followed by three periods (as mentioned above), it indicates that the argument can be repeated in the following manner:
would create a single directory named dir1, while
would create three directories named dir1, dir2, and dir3.
cp – Copy Files And Directories
cp – Copy Files And DirectoriesThe cp command is utilized for copying files or directories and can be employed in two distinct ways:
To copy a single file or directory named item1 to another file or directory named item2, you can utilize the following approach:
to copy multiple items (either files or directories) into a directory.
Useful Options And Examples
Listed below are several commonly employed options (both the short option and the equivalent long option) for the cp command:
-a, --archive
Copy the files and directories and all of their attributes, including ownerships and permissions. Normally, copies take on the default attributes of the user performing the copy.
-i, --interactive
Before overwriting an existing file, prompt the user for confirmation. If this option is not specified, cp will silently overwrite files.
-r, --recursive
Recursively copy directories and their contents. This option (or the -a option) is required when copying directories.
-u, --update
When copying files from one directory to another, only copy files that either don't exist, or are newer than the existing corresponding files, in the destination directory.
-v, --verbose
Display informative messages as the copy is performed.
Some examples:
cp file1 file2
Copy file1 to file2. If file2 exists, it is overwritten with the contents of file1. If file2 does not exist, it is created.
cp -i file1 file2
Same as above, except that if file2 exists, the user is prompted before it is overwritten.
cp file1 file2 dir1
Copy file1 and file2 into directory dir1. dir1 must already exist.
cp dir1/* dir2
Using a wildcard, all the files in dir1 are copied into dir2. dir2 must already exist.
cp -r dir1 dir2
Copy the contents of directory dir1 to directory dir2. If directory dir2 does not exist, it is created and, after the copy, will contain the same contents as directory dir1.
If directory dir2 does exist, then directory dir1 (and its contents) will be copied into dir2.
mv – Move And Rename Files
mv – Move And Rename FilesThe mv command serves the dual purpose of moving files and renaming files, depending on its usage. In either scenario, the original filename ceases to exist after the operation. The mv command is employed in a similar manner to cp:
to move or rename file or directory item1 to item2 or:
to move one or more items from one directory to another.
Useful Options And Examples
mv shares many of the same options as cp:
-i, --interactive
Before overwriting an existing file, prompt the user for confirmation. If this option is not specified, mv will silently overwrite files.
-u, --update
When moving files from one directory to another, only move files that either don't exist, or are newer than the existing corresponding files in the destination directory.
-v, --verbose
Display informative messages as the move is performed.
Some Examples
mv file1 file2
Move file1 to file2. If file2 exists, it is overwritten with the contents of file1. If file2 does not exist, it is created. In either case, file1 ceases to exist.
mv -i file1 file2
Same as above, except that if file2 exists, the user is prompted before it is overwritten.
mv file1 file2 dir1
Move file1 and file2 into directory dir1. dir1 must already exist.
mv dir1 dir2
If directory dir2 does not exist, create directory dir2 and move the contents of directory dir1 into dir2 and delete directory dir1.
If directory dir2 does exist, move directory dir1 (and its contents) into directory dir2.
rm – Remove Files And Directories
rm – Remove Files And DirectoriesThe rm command is used to remove (delete) files and directories:
where item is one or more files or directories.
Useful Options And Examples
Here are some of the common options for rm:
-i, --interactive
Before deleting an existing file, prompt the user for confirmation. If this option is not specified, rm will silently delete files.
-r, --recursive
Recursively delete directories. This means that if a directory being deleted has subdirectories, delete them too. To delete a directory, this option must be specified.
-f, --force
Ignore nonexistent files and do not prompt. This overrides the --interactive option.
-v, --verbose
Display informative messages as the deletion is performed.
Some Examples
rm file1
Delete file1 silently.
rm -i file1
Same as above, except that the user is prompted for confirmation before the deletion is performed.
rm -r file1 dir1
Delete file1 and dir1 and its contents.
rm -rf file1 dir1
Same as above, except that if either file1 or dir1 do not exist, rm will continue silently.
Be Careful With rm!
rm!Unix-like operating systems, including Linux, do not feature an undelete command. When an item is deleted using the rm command, it is permanently removed. Linux operates under the assumption that users are knowledgeable and intentional with their actions.
Exercise caution when working with wildcards, as a small mistake can have unintended consequences. Let's consider a classic example.
Suppose you intend to delete only the HTML files within a directory. To accomplish this, you would enter the command:
rm *.html
which is correct. However, if you inadvertently insert a space between the "*" and ".html" like this:
rm * .html
the rm command will proceed to delete all files in the directory and subsequently report an error indicating the absence of a file named ".html".
Here's a valuable tip: Whenever you intend to use wildcards with the rm command (aside from verifying your typing accuracy), it's advisable to first test the wildcard with ls. This allows you to preview the files that will be deleted. Afterward, you can press the up arrow key to recall the command and replace the ls portion with rm. This approach ensures that you have a clear understanding of the files that will be affected before proceeding with the deletion.
ln – Create Links
ln – Create LinksThe ln command is utilized to create either hard or symbolic links and can be employed in one of two manners:
to create a hard link, and:
to create a symbolic link where item is either a file or a directory.
Hard Links
Hard links are the traditional method in Unix for creating links, while symbolic links are a more recent addition. By default, every file possesses a single hard link that corresponds to its name. When a hard link is created, an additional directory entry for the file is generated. However, hard links come with two significant limitations:
A hard link is restricted to referencing files within its own file system. In other words, it is incapable of referencing a file that exists on a different disk partition than the link itself.
A hard link may not reference a directory.
A hard link bears no distinguishing characteristics from the file itself. Unlike a symbolic link, when you view a directory containing a hard link, there will be no specific indication of the link's presence. When a hard link is deleted, the link itself is removed, but the file's contents persist (i.e., the allocated space is not deallocated) until all links to the file are eliminated.
It is crucial to have an awareness of hard links as you may come across them occasionally. However, contemporary practices tend to favor symbolic links, which we will discuss in the following section.
Symbolic Links
Symbolic links were developed to surpass the limitations of hard links. Symbolic links function by generating a distinct file type that includes a textual pointer to the referenced file or directory. In this aspect, they operate similarly to a Windows shortcut, although it's important to note that symbolic links predates the Windows feature by many years.
A file that is referenced by a symbolic link and the symbolic link itself appear almost identical. For instance, if you write something to the symbolic link, the changes are applied to the referenced file. However, when you delete a symbolic link, only the link itself is removed, while the file it points to remains unaffected. If the file is deleted prior to the symbolic link, the link will persist but will no longer have a valid target. In such cases, the link is considered broken. Some implementations of the ls command display broken links in a distinct color, such as red, to highlight their existence.
The concept of links may initially appear perplexing, but don't worry. We will delve into practical examples and explore these concepts together, with the aim of providing clarity along the way. So, hang in there, and hopefully, everything will become clear as we proceed.
Let's Play
As we are about to engage in practical file manipulation, let's establish a secure location for experimenting with our file manipulation commands. To begin, we require a directory to serve as our working space. We will create this directory, named "playground," within our home directory.
Creating Directories
The mkdir command is utilized for directory creation. To establish our "playground" directory, we will confirm that we are currently in our home directory and proceed to create the new directory as follows:
To add some intrigue to our playground, let's create a couple of directories within it named dir1 and dir2. To achieve this, we will change our current working directory to "playground" and execute another mkdir command:
It's worth noting that the mkdir command allows us to provide multiple arguments, enabling the creation of multiple directories with a single command.
Copying Files
Now, let's introduce some data into our playground. We can accomplish this by copying a file. With the cp command, we will duplicate the passwd file from the /etc directory to our current working directory:
Observe how we utilized the shorthand notation for the current working directory, denoted by a single trailing period. Consequently, if we execute an ls command, we will witness the presence of our file:
Now, for a touch of amusement, let's repeat the copy operation while employing the -v option (verbose) to observe its effects:
The cp command executed the copy operation once more, while displaying a succinct message that indicates the performed action. It's important to note that cp overwrote the initial copy without providing a warning. Once again, this exemplifies cp's assumption that you are aware of what you are doing. However, if you desire a warning, we can include the -i option (interactive):
When prompted, responding with a y will result in the file being overwritten, while any other character (such as n) will cause cp to preserve the file without making any changes.
Moving And Renaming Files
Now, considering that the name passwd may not align with the playful nature of our playground, let's modify it to something more fitting:
Let's add some fun by circulating our renamed file through each of the directories and then returning it back again:
to move it first to directory dir1, then:
to move it from dir1 to dir2, then:
To ultimately bring it back to the current working directory, let's proceed by moving our renamed file. Furthermore, let's explore the impact of the mv command on directories. Initially, we will relocate our data file into dir1 once again:
then move dir1 into dir2 and confirm it with ls:
Please note that since dir2 was already present, the mv command moved dir1 into dir2. If dir2 had not existed, mv would have instead renamed dir1 to dir2. Finally, let's restore everything to their original state:
Creating Hard Links
Now, let's delve into creating some links, starting with hard links. We will generate a few links to our data file using the following method:
As a result, we now have four instances of the file named fun. Let's examine the contents of our playground directory:
Upon closer inspection, you may observe that both fun and fun-hard in the listing share a 4 in the second field. This number represents the count of hard links currently associated with the file. It is crucial to remember that a file always possesses at least one link since its name is established through a link.
However, how can we ascertain that fun and fun-hard are indeed referring to the same file? In this scenario, relying solely on the ls command does not provide a definitive answer. Although we can observe that fun and fun-hard share the same size (field 5) in the listing, this information alone does not guarantee their identity. To resolve this uncertainty, we must delve deeper into the matter.
When contemplating hard links, it proves beneficial to envision files as comprising two components: the data part, containing the actual file contents, and the name part, holding the file's name. By creating hard links, we essentially generate additional name parts that all point to the same data part. The system allocates a series of disk blocks to form what is known as an "inode", which is subsequently linked to the name part. Each hard link thus refers to a specific inode, encapsulating the file's contents.
To unveil this information, the ls command provides a solution. By utilizing the -i option, we can invoke the desired functionality:
In this listing version, the first field corresponds to the inode number. Upon inspection, we observe that both fun and fun-hard possess identical inode numbers, thereby confirming their identity as the same file.
Creating Symbolic Links
Symbolic links were developed as a solution to overcome the limitations of hard links. Hard links are unable to extend across physical devices and are limited to referencing files rather than directories. Symbolic links, on the other hand, serve as a distinct file type that includes a textual pointer to the target file or directory.
Creating symbolic links is similar to creating hard links:
The initial example is quite simple: we merely include the -s option to generate a symbolic link instead of a hard link. However, how do we approach the following two examples? Recall that when we establish a symbolic link, we essentially create a textual representation indicating the relative location of the target file in relation to the symbolic link. This becomes clearer when we examine the output of the ls command:
In the directory listing for fun-sym in dir1, we can observe that it is a symbolic link indicated by the leading l in the first field. Furthermore, it points to ../fun, which is accurate. In relation to the location of fun-sym, the file fun is situated in the directory above it. It's worth noting that the length of the symbolic link file is 6, corresponding to the number of characters in the string ../fun, rather than the length of the file it is referencing.
When creating symbolic links, you can either use absolute pathnames:
or relative pathnames, as demonstrated in our previous example. Utilizing relative pathnames is preferable since it enables the renaming or moving of a directory containing symbolic links without causing them to break.
In addition to regular files, symbolic links can also reference directories:
Removing Files And Directories
As mentioned previously, the rm command is employed for deleting files and directories. We will now utilize it to tidy up our playground. To begin, let's remove one of our hard links:
The deletion process went as anticipated. The file fun-hard has been successfully removed, and the link count for fun has decreased from four to three, as indicated in the second field of the directory listing. Now, let's proceed with deleting the file fun. For some added enjoyment, we'll include the -i option to observe its behavior:
Upon confirming with a y at the prompt, the file has been successfully deleted. However, let's examine the output of the ls command now. Did you notice what happened to fun-sym? As it is a symbolic link pointing to a file that no longer exists, the link has become broken:
In most Linux distributions, the ls command is configured to highlight broken links. For example, on a Fedora system, broken links are displayed in blinking red text! While the presence of a broken link is not inherently dangerous, it can make things look messy. If we attempt to use a broken link, we will encounter the following situation:
Let's clean up a little. We'll delete the symbolic links:
It is important to keep in mind that when working with symbolic links, most file operations are performed on the target of the link rather than the link itself. However, there is an exception with the rm command. When you delete a symbolic link, it is the link itself that gets deleted, not the target file or directory.
To conclude, we will delete our playground. To accomplish this, we'll navigate back to our home directory and employ the rm command with the recursive option (-r) to delete the playground directory along with all its contents, including any subdirectories.
Summary
We've covered a significant amount of material, and it may require some time for everything to fully sink in. I encourage you to repeat the playground exercise multiple times until you feel comfortable with it. Developing a solid understanding of fundamental file manipulation commands and wildcards is crucial. Don't hesitate to enhance the playground exercise by adding additional files and directories, and experimenting with wildcards to perform various operations. While the concept of links might seem perplexing initially, invest the time to grasp how they function because they can prove invaluable in certain situations.
Last updated