Shell Command Language
This chapter contains the definition of the XSI Shell Command Language. See the Shell Command Language Index for an overview. Shell Introduction
The shell is a command language interpreter. This chapter describes the syntax of that command language as it is used by the sh utility and the system() and popen() functions in the XSH specification.The shell operates according to the following general overview of operations. The specific details are included in the cited sections of this chapter.
- The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions in the XSH specification. If the first line of a file of shell commands starts with the characters #!, the results are unspecified.
The construct #! is reserved for implementations wishing to provide that extension. A portable application cannot use #! as the first line of a shell script; it might not be interpreted as a comment.
- The shell breaks the input into tokens: words and operators. (See
Token Recognition .)
- The shell parses the input into simple commands (see
Simple Commands ) and compound commands (seeCompound Commands ).
- The shell performs various expansions (separately) on different parts of each command, resulting in a list of pathnames and fields to be treated as a command and arguments (see
Word Expansions ).
- The shell performs redirection (see
Redirection ) and removes redirection operators and their operands from the parameter list.
- The shell executes a function (see
Function Definition Command ), built-in (seeSpecial Built-in Utilities ), executable file or script, giving the names of the arguments as positional parameters numbered 1 to n, and the name of the command (or in the case of a function within a script, the name of the script) as the positional parameter numbered 0 (seeCommand Search and Execution ).
- The shell optionally waits for the command to complete and collects the exit status (see
Exit Status for Commands ).
Quoting
Quoting is used to remove the special meaning of certain characters or words to the shell. Quoting can be used to preserve the literal meaning of the special characters in the next paragraph; prevent reserved words from being recognised as such; and prevent parameter expansion and command substitution within here-document processing (see Here-document ).The following characters must be quoted if they are to represent themselves:
and the following may need to be quoted under certain circumstances. That is, these characters may be special depending on conditions described elsewhere in this specification:| & ; < > ( ) $ ` \ " ' <space> <tab> <newline>
* ? [ # ~ = %
The various quoting mechanisms are the escape character, single-quotes and double-quotes. The here-document represents another form of quoting; see
Here-document .Escape Character (Backslash)
A backslash that is not quoted preserves the literal value of the following character, with the exception of a newline character. If a newline character follows the backslash, the shell will interpret this as line continuation. The backslash and newline characters will be removed before splitting the input into tokens. Since the escaped newline character is removed entirely from the input and is not replaced by any white space, it cannot serve as a token separator. Single-quotes
Enclosing characters in single-quotes (' ') preserves the literal value of each character within the single-quotes. A single-quote cannot occur within single-quotes.A backslash cannot be used to escape a single-quote in a single-quoted string. An embedded quote can be created by writing, for example: 'a'\"b', which yields a'b. (See
Field Splitting for a better understanding of how portions of words are either split into fields or remain concatenated.) A single token can be made up of concatenated partial strings containing all three kinds of quoting or escaping, thus permitting any combination of characters.Double-quotes
Enclosing characters in double-quotes (" ") preserves the literal value of all characters within the double-quotes, with the exception of the characters dollar-sign, backquote and backslash, as follows:
- $
- The dollar-sign retains its special meaning introducing parameter expansion (see
Parameter Expansion ), a form of command substitution (seeCommand Substitution ), and arithmetic expansion (seeArithmetic Expansion ). The input characters within the quoted string that are also enclosed between "$(" and the matching "(" will not be affected by the double-quotes, but rather define that command whose output replaces the $(...) when the word is expanded. The tokenising rules inToken Recognition are applied recursively to find the matching ")". Within the string of characters from an enclosed ${ to the matching "}", an even number of unescaped double-quotes or single-quotes, if any, must occur. A preceding backslash character must be used to escape a literal "{" or "}". The rule inParameter Expansion will be used to determine the matching "}".- `
- The backquote retains its special meaning introducing the other form of command substitution (see
Command Substitution ). The portion of the quoted string from the initial backquote and the characters up to the next backquote that is not preceded by a backslash, having escape characters removed, defines that command whose output replaces `...` when the word is expanded. Either of the following cases produces undefined results:
- a single- or double-quoted string that begins, but does not end, within the `...` sequence
- a `...` sequence that begins, but does not end, within the same double-quoted string.
- \
- The backslash retains its special meaning as an escape character (see
Escape Character (Backslash) ) only when followed by one of the characters:$ ` " \ <newline>
A double-quote must be preceded by a backslash to be included within double-quotes. The parameter @ has special meaning inside double-quotes and is described in
Special Parameters .In double-quoting, if a backslash is immediately followed by a character that would be interpreted as having a special meaning, the backslash is deleted and the subsequent character is taken literally. If a backslash does not precede a character that would have a special meaning, it is left in place unmodified and the character immediately following it is also left unmodified. Thus, for example:
The requirement that double-quotes be matched inside ${...} within double-quotes and the rule for finding the matching "}" in"\$" -> $ "\a" -> \a
Parameter Expansion eliminate several subtle inconsistencies in expansion for historical shells in rare cases; for example:yields bar when foo is not defined, and is an invalid substitution when foo is defined, in many historical shells. The differences in processing the ${...} form have led to inconsistencies between historical systems. A consequence of this rule is that single-quotes cannot be used to quote the "}" within ${...}; for example:"${foo-bar"}
is invalid because the ${...} substitution contains an unpaired unescaped single-quote. The backslash can be used to escape the "}" in this example to achieve the desired result:unset bar foo="${bar-'}'}"
unset bar foo="${bar-\}}"
Some systems have allowed the end of the word to terminate the backquoted command substitution, such as in:
"`echo hello"
This usage is undefined; the matching backquote is required by this specification. The other undefined usage can be illustrated by the example:
The description of the recursive actions involving command substitution can be illustrated with an example. Upon recognising the introduction of command substitution, the shell must parse input (in a new context), gathering the source for the command substitution until an unbalanced ")" or ` is located. For example, in the following:sh -c '` echo "foo`'
the double-quote following the echo does not terminate the first double-quote; it is part of the command substitution script. Similarly, in:echo "$(date; echo " one" )"
the asterisk is not quoted since it is inside command substitution; however:echo "$(echo *)"
is quoted (and represents the asterisk character itself).echo "$(echo "*")"
Token Recognition
The shell reads its input in terms of lines from a file, from a terminal in the case of an interactive shell or from a string in the case of sh -c or system(). The input lines can be of unlimited length. These lines are parsed using two major modes: ordinary token recognition and processing of here-documents. When an io_here token has been recognised by the grammar (see
Shell Grammar ), one or more of the subsequent lines immediately following the next NEWLINE token form the body of one or more here-documents and are parsed according to the rules ofHere-document .When it is not processing an io_here, the shell will break its input into tokens by applying the first applicable rule below to the next character in its input. The token will be from the current position in the input until a token is delimited according to one of the rules below; the characters forming the token are exactly those in the input, including any quoting characters. If it is indicated that a token is delimited, and no characters have been included in a token, processing will continue until an actual token is delimited.
- If the end of input is recognised, the current token will be delimited. If there is no current token, the end-of-input indicator will be returned as the token.
- If the previous character was used as part of an operator and the current character is not quoted and can be used with the current characters to form an operator, it will be used as part of that (operator) token.
Note that certain combinations of characters are invalid in portable scripts, as shown in the grammar, and that some systems have assigned these combinations (such as |&) as valid control operators. Portable scripts cannot rely on receiving errors in all cases where this specification indicates that a syntax is invalid.
- If the previous character was used as part of an operator and the current character cannot be used with the current characters to form an operator, the operator containing the previous character will be delimited.
- If the current character is backslash, single-quote or double-quote (\, ' or ") and it is not quoted, it will affect quoting for subsequent characters up to the end of the quoted text. The rules for quoting are as described in
Quoting . During token recognition no substitutions will be actually performed, and the result token will contain exactly the characters that appear in the input (except for newline character joining), unmodified, including any embedded or enclosing quotes or substitution operators, between the quote mark and the end of the quoted text. The token will not be delimited by the end of the quoted field.
- If the current character is an unquoted "$" or `, the shell will identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: "$" or ${, $( or `, and $((, respectively. The shell will read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell will recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognise embedded constructs, will be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token will not be delimited by the end of the substitution.
- If the current character is not quoted and can be used as the first character of a new operator, the current token (if any) will be delimited. The current character will be used as the beginning of the next (operator) token.
- If the current character is an unquoted newline character, the current token will be delimited.
- If the current character is an unquoted blank character, any token containing the previous character is delimited and the current character will be discarded.
- If the previous character was part of a word, the current character will be appended to that word.
- If the current character is a "#", it and all subsequent characters up to, but excluding, the next newline character will be discarded as a comment. The newline character that ends the line is not considered part of the comment. The "#" starts a comment only when it is at the beginning of a token. Since the search for the end-of-comment does not consider an escaped newline character specially, a comment cannot be continued to the next line.
- The current character will be used as the start of a new word.
Once a token is delimited, it will be categorised as required by the grammar in
Shell Grammar .Alias Substitution
The processing of aliases is supported on all XSI-conformant systems. After a token has been delimited, but before applying the grammatical rules in
Shell Grammar , a resulting word that is identified to be the command name word of a simple command is examined to determine if it is an unquoted, valid alias name. However, reserved words in correct grammatical context are not candidates for alias substitution. A valid alias name (see the term alias name in the XBD specification, Glossary ) is one that has been defined by the alias utility and not subsequently undefined using unalias. Implementations also may provide predefined valid aliases that are in effect when the shell is invoked. To prevent infinite loops in recursive aliasing, if the shell is not currently processing an alias of the same name, the word will be replaced by the value of the alias; otherwise, it will not be replaced.If the value of the alias replacing the word ends in a blank character, the shell will check the next command word for alias substitution; this process continues until a word is found that is not a valid alias or an alias value does not end in a blank character.
When used as specified by this specification, alias definitions will not be inherited by separate invocations of the shell or by the utility execution environments invoked by the shell; see
Shell Execution Environment .The definition of alias name precludes an alias name containing a slash character. Since the text applies to the command words of simple commands, reserved words (in their proper places) cannot be confused with aliases.
An example concerning trailing blank characters and reserved words follows. If the user types:
$ alias foo="/bin/ls " $ alias while="/"
The effect of executing:
is a never-ending sequence of Hello, World strings to the screen. However, if the user types:$ while true > do > echo "Hello, World" > done
the result will be an ls listing of /. Since the alias substitution for foo ends in a space character, the next word is checked for alias substitution. The next word, while, has also been aliased, so it is substituted as well. Since it is not in the proper position as a command word, it is not recognised as a reserved word.$ foo while
If the user types:
while retains its normal reserved-word properties.$ foo; while
Reserved Words
Reserved words are words that have special meaning to the shell. (SeeShell Commands .) The following words will be recognised as reserved words:(*) In some historical systems, the curly braces are treated as control operators. To assist in future standardisation activities, portable applications should avoid using unquoted braces to represent the characters themselves. It is possible that a future version of the ISO/IEC 9945-2:1993 standard may require that { and } be treated individually as control operators, although the token {} will probably be a special-case exemption from this because of the often-used find{} construct.
! elif fi in while case else for then {* do esac if until } done This recognition will occur only when none of the characters are quoted and when the word is used as:
- the first word of a command
- the first word following one of the reserved words other than case, for, or in
- the third word in a case or for command (only in is valid in this case).
See the grammar in
Shell Grammar .The following words may be recognised as reserved words on some systems (when none of the characters are quoted), causing unspecified results:
function select [[ ]] This list of unspecified reserved words is from the KornShell, so portable applications cannot use them in places a reserved word would be recognised without quoting some or all of their characters.
Words that are the concatenation of a name and a colon (:) are reserved; their use produces unspecified results. This reservation is to allow future implementations that support named labels for flow control.
Reserved words are recognised only when they are delimited (that is, meet the definition of word in the XSH specification), whereas operators are themselves delimiters. For instance, "(" and ")" are control operators, so that no space character is needed in (list). However, "{" and "}" are reserved words in { list;}, so that in this case the leading space character and semicolon are required.
Parameters and Variables
A parameter can be denoted by a name, a number or one of the special characters listed in Special Parameters . A variable is a parameter denoted by a name.A parameter is set if it has an assigned value (null is a valid value). Once a variable is set, it can only be unset by using the unset special built-in command.
Positional Parameters
A positional parameter is a parameter denoted by the decimal value represented by one or more digits, other than the single digit 0. The digits denoting the positional parameters are always interpreted as a decimal value, even if there is a leading zero. When a positional parameter with more than one digit is specified, the application must enclose the digits in braces (seeParameter Expansion ). Positional parameters are initially assigned when the shell is invoked (see sh), temporarily replaced when a shell function is invoked (seeFunction Definition Command ), and can be reassigned with the set special built-in command.Special Parameters
Listed below are the special parameters and the values to which they will expand. Only the values of the special parameters are listed; see Word Expansions for a detailed summary of all the stages involved in expanding words.
- *
- Expands to the positional parameters, starting from one. When the expansion occurs within a double-quoted string (see
Double-quotes ), it expands to a single field with the value of each parameter separated by the first character of the variable, or by a space character if is unset. If is set to a null string, this is not equivalent to unsetting it; its first character will not exist, so the parameter values are concatenated. For example:$ IFS='' $ set foo bar bam $ echo "$@" foo bar bam $ echo "$*" foobarbam $ unset IFS $ echo "$*" foo bar bam
- @
- Expands to the positional parameters, starting from one. When the expansion occurs within double-quotes, and where field splitting (see
Field Splitting ) is performed, each positional parameter expands as a separate field, with the provision that the expansion of the first parameter is still joined with the beginning part of the original word (assuming that the expanded parameter was embedded within a word), and the expansion of the last parameter is still joined with the last part of the original word. If there are no positional parameters, the expansion of "@" generates zero fields, even when "@" is double-quoted.- #
- Expands to the decimal number of positional parameters. The command name (parameter 0) is not counted in the number given by "#" because it is a special parameter, not a positional parameter.
- ?
- Expands to the decimal exit status of the most recent pipeline (see
Pipelines ).- -
- (Hyphen.) Expands to the current option flags (the single-letter option names concatenated into a string) as specified on invocation, by the set special built-in command or implicitly by the shell. The $- special parameter can be used to save and restore set options:
The three options are removed using sed in the example because they may appear in the value of $- (from the sh command line), but are not valid options to set.Save=$(echo $- | sed 's/[ics]//g') ... set +aCefnuvx if [ -n "$Save" ]; then set -$Save fi
- $
- Expands to the decimal process ID of the invoked shell. In a subshell (see
Shell Execution Environment ), "$" expands to the same value as that of the current shell. Most historical implementations implement subshells by forking; thus, the special parameter "$" does not necessarily represent the process ID of the shell process executing the commands since the subshell execution environment preserves the value of "$".- !
- Expands to the decimal process ID of the most recent background command (see
Lists ) executed from the current shell. (For example, background commands executed from subshells do not affect the value of $! in the current shell environment.) For a pipeline, the process ID is that of the last command in the pipeline.- 0
- (Zero.) Expands to the name of the shell or shell script. See sh for a detailed description of how this name is derived.
See the description of the IFS variable in
Shell Variables .The descriptions of parameters "*" and "@" assume the reader is familiar with the field splitting discussion in
Field Splitting and understands that portions of the word will remain concatenated unless there is some reason to split them into separate fields.Some examples of the "*" and "@" properties, including the concatenation aspects:
set "abc" "def ghi" "jkl" echo $* => "abc" "def" "ghi" "jkl" echo "$*" => "abc def ghi jkl" echo $@ => "abc" "def" "ghi" "jkl"
but:
In the preceding examples, the double-quote characters that appear after the => do not appear in the output and are used only to illustrate word boundaries.echo "$@" => "abc" "def ghi" "jkl" echo "xx$@yy" => "xxabc" "def ghi" "jklyy" echo "$@$@" => "abc" "def ghi" "jklabc" "def ghi" "jkl"
Shell Variables
Variables are initialised from the environment (as defined by the XSH specification) and can be given new values with variable assignment commands. If a variable is initialised from the environment, it is marked for export immediately; see the export special built-in. New variables can be defined and initialised with variable assignments, with the read or getopts utilities, with the name parameter in a for loop, with the ${name=word} expansion or with other mechanisms provided as implementation extensions. The following variables affect the execution of the shell:
- ENV
- This variable, when the shell is invoked, is subjected to parameter expansion (see
Parameter Expansion ) by the shell and the resulting value is used as a pathname of a file containing shell commands to execute in the current environment. The file need not be executable. If the expanded value of ENV is not an absolute pathname, the results are unspecified. ENV will be ignored if the user's real and effective user IDs or real and effective group IDs are different. This variable can be used to set aliases and other items local to the invocation of a shell. The file referred to by ENV differs from $HOME/.profile in that .profile is typically executed at session startup, whereas the file is executed at the beginning of each shell invocation. The ENV value is interpreted in a manner similar to a dot script, in that the commands are executed in the current environment and the file needs to be readable, but not executable. However, unlike dot scripts, no PATH searching is performed. This is used as a guard against Trojan Horse security breaches.- HOME
- This variable is interpreted as the pathname of the user's home directory. The contents of HOME are used in tilde expansion (see
Tilde Expansion ).- IFS
- Input field separators: a string treated as a list of characters that is used for field splitting and to split lines into fields with the read command. If IFS is not set, the shell will behave as if the value of IFS were the space, tab and newline characters. (See
Field Splitting .)- LANG
- Provide a default value for the internationalisation variables that are unset or null. If LANG is unset or null, the corresponding value from the implementation-dependent default locale will be used. If any of the internationalisation variables contains an invalid setting, the utility will behave as if none of the variables had been defined.
- LC_ALL
- This variable provides a default value for the variables, as described in the XBD specification, Environment Variables .
- LC_COLLATE
- This variable determines the behaviour of range expressions, equivalence classes and multi-character collating elements within pattern matching.
- LC_CTYPE
- This variable determines the interpretation of sequences of bytes of text data as characters (for example, single- as opposed to multi-byte characters), which characters are defined as letters (character class alpha) and blank characters (character class blank), and the behaviour of character classes within pattern matching. Changing the value of LC_CTYPE after the shell has started does not affect the lexical processing of shell commands in the current shell execution environment or its subshells. Invoking a shell script or performing exec sh subjects the new shell to the changes in LC_CTYPE.
- LC_MESSAGES
- This variable determines the language in which messages should be written.
- LINENO
- This variable is set by the shell to a decimal number representing the current sequential line number (numbered starting with 1) within a script or function before it executes each command. If the user unsets or resets LINENO the variable may lose its special meaning for the life of the shell. If the shell is not currently executing a script or function, the value of LINENO is unspecified.
- NLSPATH
- Determine the location of message catalogues for the processing of LC_MESSAGES.
- PATH
- This variable represents a string formatted as described in the XBD specification, Chapter 6, Environment Variables, used to effect command interpretation. See
Command Search and Execution .- PPID
- This variable is set by the shell to the decimal process ID of the process that invoked this shell. In a subshell (see
Shell Execution Environment ), PPID will be set to the same value as that of the parent of the current shell. For example, echo $PPID and (echo $PPID) would produce the same value. Without this variable, there is no way for a utility to signal its parent or to find its parent process. This is also useful to know if the shell has been orphaned.- PS1
- Each time an interactive shell is ready to read a command, the value of this variable is subjected to parameter expansion and written to standard error. The default value is "$ ". For users who have specific additional implementation-dependent privileges, the default may be another, implementation-dependent, value. (Historically, the superuser has had a prompt of "# ".) The shell replaces each instance of the character "!" in PS1 with the history file number of the next command to be typed. Escaping the "!" with another "!" (that is, !!) places the literal character "!" in the prompt.
- PS2
- Each time the user enters a newline character prior to completing a command line in an interactive shell, the value of this variable is subjected to parameter expansion and written to standard error. The default value is "> ".
- PS4
- When an execution trace (set -x) is being performed in an interactive shell, before each line in the execution trace, the value of this variable is subjected to parameter expansion and written to standard error. The default value is "+ ". For example, the following script:
writes the following to standard error:PS4='[${LINENO}]+ ' set -x echo Hello
[3]+ echo Hello
Tilde expansion for components of the in an assignment such as:
is a feature of some historical shells and is allowed by the wording ofPATH=~hlj/bin:~dwc/bin:$PATH
Tilde Expansion . Note that the tildes are expanded during the assignment to PATH not when PATH is accessed during command search.
Word Expansions
This section describes the various expansions that are performed on words. Not all expansions are performed on every word, as explained in the following sections. Tilde expansions, parameter expansions, command substitutions, arithmetic expansions and quote removals that occur within a single word expand to a single field. It is only field splitting or pathname expansion that can create multiple fields from a single word. The single exception to this rule is the expansion of the special parameter "@" within double-quotes, as described in
Special Parameters .The order of word expansion is as follows:
- Tilde expansion (see
Tilde Expansion ), parameter expansion (seeParameter Expansion ), command substitution (seeCommand Substitution ), and arithmetic expansion (seeArithmetic Expansion ) are performed, beginning to end. See item 5 inToken Recognition .
- Field splitting (see
Field Splitting ) is performed on the portions of the fields generated by step 1, unless is null.
- Pathname expansion (see
Pathname Expansion ) is performed, unless set -f is in effect.
- Quote removal (see
Quote Removal ) always is performed last.
The expansions described in this section will occur in the same shell environment as that in which the command is executed.
If the complete expansion appropriate for a word results in an empty field, that empty field will be deleted from the list of fields that form the completely expanded command, unless the original word contained single-quote or double-quote characters.
The "$" character is used to introduce parameter expansion, command substitution or arithmetic evaluation. If an unquoted "$" is followed by a character that is either not numeric, the name of one of the special parameters (see
Special Parameters ), a valid first character of a variable name, a left curly brace ({) or a left parenthesis, the result is unspecified.IFS is used for performing field splitting on the results of parameter and command substitution; it is not used for splitting all fields. Previous versions of the shell used it for splitting all fields during field splitting, but this has severe problems because the shell can no longer parse its own script. There are also important security implications caused by this behaviour. All useful applications of IFS use it for parsing input of the read utility and for splitting the results of parameter and command substitution.
The rule concerning expansion to a single field requires that if foo=abc and bar=def, that:
expands to the single field:"$foo""$bar"
abcdef
The rule concerning empty fields can be illustrated by:
$ unset foo $ set $foo bar '' xyz "$foo" abc $ for i > do > echo "-$i-" > done -bar- -- -xyz- -- -abc-
Step 2 indicates that parameter expansion, command substitution and arithmetic expansion are all processed simultaneously as they are scanned. For example, the following is valid arithmetic:
x=1 echo $(( $(echo 3)+$x ))
Tilde Expansion
A tilde-prefix consists of an unquoted tilde character at the beginning of a word, followed by all of the characters preceding the first unquoted slash in the word, or all the characters in the word if there is no slash. In an assignment (see variable assignment in the XBD specification, Glossary ) multiple tilde-prefixes can be used: at the beginning of the word (that is, following the equal sign of the assignment), following any unquoted colon or both. A tilde-prefix in an assignment is terminated by the first unquoted colon or slash. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name from the user database. A portable login name cannot contain characters outside the set given in the description of the LOGNAME environment variable in the XSH specification. If the login name is null (that is, the tilde-prefix contains only the tilde), the tilde-prefix will be replaced by the value of the variable HOME. If HOME is unset, the results are unspecified. Otherwise, the tilde-prefix will be replaced by a pathname of the home directory associated with the login name obtained using the XSH specification getpwnam() function. If the system does not recognise the login name, the results are undefined. Tilde expansion generally occurs only at the beginning of words, but an exception based on historical practice has been included:
is eligible for tilde expansion because tilde follows a colon and none of the relevant characters is quoted. Consideration was given to prohibiting this behaviour because any of the following are reasonable substitutes:PATH=/posix/bin:~dgk/bin
PATH=$(printf %s: ~rms/bin ~bfox/bin ...) PATH=$(printf %s ~karels/bin : ~bostic/bin) for Dir in ~maart/bin ~srb/bin ... do PATH=${PATH:+$PATH:}$Dir done
In the first command, any number of directory names are concatenated and separated with colons, but it may be undesirable to end the variable with a colon because this is an obsolescent means to include dot at the end of the PATH. In the second, explicit colons are used for each directory. In all cases, the shell performs tilde expansion on each directory because all are separate words to the shell.
Note that expressions in operands such as:
do not qualify as shell variable assignments and tilde expansion is not performed (unless the command does so itself, which make does not).make -k mumble LIBDIR=~chet/lib
The special sequence $~ has been designated for future implementations to evaluate as a means of forcing tilde expansion in any word.
Because of the requirement that the word is not quoted, the following are not equivalent; only the last will cause tilde expansion:
\~hlj/ ~h\lj/ ~"hlj"/ ~hlj\/ ~hlj/
The results of giving tilde with an unknown login name are undefined because the KornShell ~+ and ~- constructs make use of this condition, but, in general it is an error to give an incorrect login name with tilde. The results of having HOME unset are unspecified because some historical shells treat this as an error.
Parameter Expansion
The format for parameter expansion is as follows: where expression consists of all characters until the matching "}". Any "}" escaped by a backslash or within a quoted string, and characters in embedded arithmetic expansions, command substitutions and variable expansions, are not examined in determining the matching "}".${expression}
The simplest form for parameter expansion is:
${parameter}
The value, if any, of parameter will be substituted.
The parameter name or symbol can be enclosed in braces, which are optional except for positional parameters with more than one digit or when parameter is followed by a character that could be interpreted as part of the name. The matching closing brace will be determined by counting brace levels, skipping over enclosed quoted strings and command substitutions.
If the parameter name or symbol is not enclosed in braces, the expansion will use the longest valid name (see name in the XBD specification, Glossary ), whether or not the symbol represented by that name exists. When the shell is scanning its input to determine the boundaries of a name, it is not bound by its knowledge of what names are already defined. For example, if F is a defined shell variable, the command:
does not echo the value of $F followed by red; it selects the longest possible valid name, Fred, which in this case might be unset.echo $Fred
If a parameter expansion occurs inside double-quotes:
- Pathname expansion will not be performed on the results of the expansion.
- Field splitting will not be performed on the results of the expansion, with the exception of "@"; see
Special Parameters .
In addition, a parameter expansion can be modified by using one of the following formats. In each case that a value of word is needed (based on the state of parameter, as described below), word will be subjected to tilde expansion, parameter expansion, command substitution and arithmetic expansion. If word is not needed, it will not be expanded. The "}" character that delimits the following parameter expansion modifications is determined as described previously in this section and in
Double-quotes . (For example, ${foo-bar}xyz} would result in the expansion of foo followed by the string xyz} if foo is set, else the string barxyz}).
- ${parameter:-word}
- Use Default Values. If parameter is unset or null, the expansion of word will be substituted; otherwise, the value of parameter will be substituted.
- ${parameter:=word}
- Assign Default Values. If parameter is unset or null, the expansion of word will be assigned to parameter. In all cases, the final value of parameter will be substituted. Only variables, not positional parameters or special parameters, can be assigned in this way.
- ${parameter:?[word]}
- Indicate Error if Null or Unset. If parameter is unset or null, the expansion of word (or a message indicating it is unset if word is omitted) will be written to standard error and the shell will exit with a non-zero exit status. Otherwise, the value of parameter will be substituted. An interactive shell need not exit.
- ${parameter:+word}
- Use Alternative Value. If parameter is unset or null, null will be substituted; otherwise, the expansion of word will be substituted.
In the parameter expansions shown previously, use of the colon in the format results in a test for a parameter that is unset or null; omission of the colon results in a test for a parameter that is only unset. The following table summarises the effect of the colon:
parameter
set and not nullparameter
set but nullparameter
unset${parameter:-word} substitute parameter substitute word substitute word ${parameter-word} substitute parameter substitute null substitute word ${parameter:=word} substitute parameter assign word assign word ${parameter=word} substitute parameter substitute parameter assign null ${parameter:?word} substitute parameter error, exit error, exit ${parameter?word} substitute parameter substitute null error, exit ${parameter:+word} substitute word substitute null substitute null ${parameter+word} substitute word substitute word substitute null In all cases shown with "substitute", the expression is replaced with the value shown. In all cases shown with "assign", parameter is assigned that value, which also replaces the expression.
- ${#parameter}
- String Length. The length in characters of the value of parameter. If parameter is "*" or "@", the result of the expansion is unspecified.
The following four varieties of parameter expansion provide for substring processing. In each case, pattern matching notation (see
Pattern Matching Notation ), rather than regular expression notation, will be used to evaluate the patterns. If parameter is "*" or "@", the result of the expansion is unspecified. Enclosing the full parameter expansion string in double-quotes will not cause the following four varieties of pattern characters to be quoted, whereas quoting characters within the braces will have this effect.
- ${parameter%word}
- Remove Smallest Suffix Pattern. The word will be expanded to produce a pattern. The parameter expansion then will result in parameter, with the smallest portion of the suffix matched by the pattern deleted.
- ${parameter%%word}
- Remove Largest Suffix Pattern. The word will be expanded to produce a pattern. The parameter expansion then will result in parameter, with the largest portion of the suffix matched by the pattern deleted.
- ${parameter#word}
- Remove Smallest Prefix Pattern. The word will be expanded to produce a pattern. The parameter expansion then will result in parameter, with the smallest portion of the prefix matched by the pattern deleted.
- ${parameter##word}
- Remove Largest Prefix Pattern. The word will be expanded to produce a pattern. The parameter expansion then will result in parameter, with the largest portion of the prefix matched by the pattern deleted.
Examples
- ${parameter:-word}
- In this example, ls is executed only if x is null or unset. (The $(ls) command substitution notation is explained in
Command Substitution .)${x:-$(ls)}
- ${parameter:=word}
unset X echo ${X:=abc} abc- ${parameter:?word}
unset posix echo ${posix:?} sh: posix: parameter null or not set- ${parameter:word}
set a b c echo ${3:+posix} posix- ${#parameter}
HOME=/usr/posix echo ${#HOME} 10- ${parameter%word}
x=file.c echo ${x%.c}.o file.o- ${parameter%%word}
x=posix/src/std echo ${x%%/*} posix- ${parameter#word}
x=$HOME/src/cmd echo ${x#$HOME} /src/cmd- ${parameter##word}
x=/one/two/three echo ${x##*/} threeThe double-quoting of patterns is different depending on where the double-quotes are placed:
- ${x#*}
- The asterisk is a pattern character.
- ${x#"*"}
- The literal asterisk is quoted and not special.
Command Substitution
Command substitution allows the output of a command to be substituted in place of the command name itself. Command substitution occurs when the command is enclosed as follows: or (backquoted version):$(command)
`command`
The shell will expand the command substitution by executing command in a subshell environment (see
Shell Execution Environment ) and replacing the command substitution (the text of command plus the enclosing $() or backquotes) with the standard output of the command, removing sequences of one or more newline characters at the end of the substitution. Embedded newline characters before the end of the output will not be removed; however, they may be treated as field delimiters and eliminated during field splitting, depending on the value of and quoting that is in effect.Within the backquoted style of command substitution, backslash shall retain its literal meaning, except when followed by:
$ ` \
(dollar-sign, backquote, backslash). The search for the matching backquote is satisfied by the first backquote found without a preceding backslash; during this search, if a non-escaped backquote is encountered within a shell comment, a here-document, an embedded command substitution of the $(command) form, or a quoted string, undefined results occur. A single- or double-quoted string that begins, but does not end, within the `...` sequence produces undefined results.
With the $(command) form, all characters following the open parenthesis to the matching closing parenthesis constitute the command. Any valid shell script can be used for command, except:
- a script consisting solely of redirections produces unspecified results
- see the restriction on single subshells described below.
The results of command substitution will not be field splitting and pathname expansion processed for further tilde expansion, parameter expansion, command substitution or arithmetic expansion. If a command substitution occurs inside double-quotes, it will not be performed on the results of the substitution.
Command substitution can be nested. To specify nesting within the backquoted version, the application must precede the inner backquotes with backslashes; for example:
\`command\`
The $() form of command substitution solves a problem of inconsistent behaviour when using backquotes. For example:
Command Output echo '\$x' \$x echo `echo '\$x'` $x echo $(echo '\$x') \$x Additionally, the backquoted syntax has historical restrictions on the contents of the embedded command. While the new $() form can process any kind of valid embedded script, the backquoted form cannot handle some valid scripts that include backquotes. For example, these otherwise valid embedded scripts do not work in the left column, but do work on the right:
echo ` echo $( cat <<\eof cat <<\eof a here-doc with ` a here-doc with ) eof eof ` ) echo ` echo $( echo abc # a comment with ` echo abc # a comment with ) ` ) echo ` echo $( echo '`' echo ')' ` )
Because of these inconsistent behaviours, the backquoted variety of command substitution is not recommended for new applications that nest command substitutions or attempt to embed complex scripts.
If the command substitution consists of a single subshell, such as:
a portable application must separate the $( and "(" into two tokens (that is, separate them with white space). This is required to avoid any ambiguities with arithmetic expansion.$( (command) )
Arithmetic Expansion
Arithmetic expansion provides a mechanism for evaluating an arithmetic expression and substituting its value. The format for arithmetic expansion is as follows: $((expression))
The expression is treated as if it were in double-quotes, except that a double-quote inside the expression is not treated specially. The shell will expand all tokens in the expression for parameter expansion, command substitution and quote removal.
Next, the shell will treat this as an arithmetic expression and substitute the value of the expression. The arithmetic expression will be processed according to the rules of the ISO C standard, with the following exceptions:
- Only integer arithmetic is required.
- The sizeof() operator and the prefix and postfix ++ and -- operators are not required.
- Selection, iteration and jump statements are not supported.
As an extension, the shell may recognise arithmetic expressions beyond those listed. If the expression is invalid, the expansion will fail and the shell will write a message to standard error indicating the failure.
A simple example using arithmetic expansion:
# repeat a command 100 times x=100 while [ $x -gt 0 ] do command x=$(($x-1)) done
Field Splitting
After parameter expansion, command substitution, and arithmetic expansion the shell will scan the results of expansions and substitutions that did not occur in double-quotes for field splitting and multiple fields can result. The shell will treat each character of the IFS as a delimiter and use the delimiters to split the results of parameter expansion and command substitution into fields.
- If the value of IFS is a space, tab and newline character, or if it is unset, any sequence of space, tab or newline characters at the beginning or end of the input will be ignored and any sequence of those characters within the input will delimit a field. For example, the input:
yields two fields, foo and bar.<newline><space><tab>foo<tab><tab>bar<space>
- If the value of IFS is null, no field splitting will be performed.
- Otherwise, the following rules will be applied in sequence. The term "IFS white space" is used to mean any sequence (zero or more instances) of white-space characters that are in the IFS value (for example, if IFS contains space/comma/tab, any sequence of space and tab characters is considered IFS white space).
- IFS white space is ignored at the beginning and end of the input.
- Each occurrence in the input of an IFS character that is not IFS white space, along with any adjacent IFS white space, will delimit a field, as described previously.
- Non-zero-length IFS white space will delimit a field.
The last rule can be summarised as a pseudo-ERE:
where s is an white-space character and n is a character in the that is not white space. Any string matching that ERE delimits a field, except that the s+ form does not delimit fields at the beginning or the end of a line. For example, if IFS is space/comma/tab, the string:(s*ns*|s+)
yields the three colours as the delimited fields.<space><space>red<space><space>,<space>white<space>blue
Pathname Expansion
After field splitting, if set -f is not in effect, each field in the resulting command line will be expanded using the algorithm described in Pattern Matching Notation , qualified by the rules inPatterns Used for Filename Expansion .Quote Removal
The quote characters: \ ' "
(backslash, single-quote, double-quote) that were present in the original word will be removed unless they have themselves been quoted.
Redirection
Redirection is used to open and close files for the current shell execution environment (see Shell Execution Environment ) or for any command. Redirection operators can be used with numbers representing file descriptors (see the definition in the ISO POSIX-1 standard) as described below.The overall format used for redirection is:
[n]redir-op word
The number n is an optional decimal number designating the file descriptor number; it must be delimited from any preceding text and immediately precede the redirection operator redir-op. If n is quoted, the number will not be recognised as part of the redirection expression. For example:
writes the character 2 into file a. If any part of redir-op is quoted, no redirection expression will be recognised. For example:echo \2>a
writes the characters 2>a to standard output. The optional number, redirection operator and word will not appear in the arguments provided to the command to be executed (if any).echo 2\>a
Open files are represented by decimal numbers starting with zero. The largest possible value is implementation-dependent; however, all implementations support at least 0 to 9, inclusive, for use by the application. These numbers are called file descriptors. The values 0, 1 and 2 have special meaning and conventional uses and are implied by certain redirection operations; they are referred to as standard input, standard output and standard error, respectively. Programs usually take their input from standard input, and write output on standard output. Error messages are usually written on standard error. The redirection operators can be preceded by one or more digits (with no intervening blank characters allowed) to designate the file descriptor number.
If the redirection operator is << or <<-, the word that follows the redirection operator will be subjected to quote removal; it is unspecified whether any of the other expansions occur. For the other redirection operators, the word that follows the redirection operator will be subjected to tilde expansion, parameter expansion, command substitution, arithmetic expansion and quote removal. Pathname expansion will not be performed on the word by a non-interactive shell; an interactive shell may perform it, but will do so only when the expansion would result in one word.
If more than one redirection operator is specified with a command, the order of evaluation is from beginning to end.
A failure to open or create a file will cause the redirection to fail.
Redirecting Input
Input redirection will cause the file whose name results from the expansion of word to be opened for reading on the designated file descriptor, or standard input if the file descriptor is not specified.The general format for redirecting input is:
where the optional n represents the file descriptor number. If the number is omitted, the redirection will refer to standard input (file descriptor 0).[n]<word
Redirecting Output
The two general formats for redirecting output are: where the optional n represents the file descriptor number. If the number is omitted, the redirection will refer to standard output (file descriptor 1).[n]>word [n]>|word
Output redirection using the ">" format will fail if the noclobber option is set (see the description of set -C) and the file named by the expansion of word exists and is a regular file. Otherwise, redirection using the ">" or >| formats will cause the file whose name results from the expansion of word to be created and opened for output on the designated file descriptor, or standard output if none is specified. If the file does not exist, it will be created; otherwise, it will be truncated to be an empty file after being opened.
Appending Redirected Output
Appended output redirection will cause the file whose name results from the expansion of word to be opened for output on the designated file descriptor. The file is opened as if the XSH specification open() function was called with the O_APPEND flag. If the file does not exist, it will be created.The general format for appending redirected output is as follows:
where the optional n represents the file descriptor number.[n]>>word
Here-document
The redirection operators << and <<- both allow redirection of lines contained in a shell input file, known as a here-document, to the standard input of a command. The here-document is treated as a single word that begins after the next newline character and continues until there is a line containing only the delimiter, with no trailing blank characters. Then the next here-document starts, if there is one. The format is as follows:
[n]<<word here-document delimiter
If any character in word is quoted, the delimiter is formed by performing quote removal on word, and the here-document lines will not be expanded. Otherwise, the delimiter is the word itself.
If no characters in word are quoted, all lines of the here-document will be expanded for parameter expansion, command substitution and arithmetic expansion. In this case, the backslash in the input will behave as the backslash inside double-quotes (see
Double-quotes ). However, the double-quote character (") will not be treated specially within a here-document, except when the double-quote appears within $( ), ` ` or ${ }.If the redirection symbol is <<-, all leading tab characters will be stripped from input lines and the line containing the trailing delimiter. If more than one << or <<- operator is specified on a line, the here-document associated with the first operator will be supplied first by the application and will be read first by the shell. For example:
cat <<eof1; cat <<eof2 Hi, eof1 Helene. eof2
The case of a missing delimiter at the end of a here-document is not specified. This is considered an error in the script (one that sometimes can be difficult to diagnose), although some systems have treated end-of-file as an implicit delimiter.
Duplicating an Input File Descriptor
The redirection operator:is used to duplicate one input file descriptor from another, or to close one. If word evaluates to one or more digits, the file descriptor denoted by n, or standard input if n is not specified, will be made to be a copy of the file descriptor denoted by word; if the digits in word do not represent a file descriptor already open for input, a redirection error will result (see[n]<&word
Consequences of Shell Errors ). If word evaluates to "-", file descriptor n, or standard input if n is not specified, will be closed. If word evaluates to something else, the behaviour is unspecified.Duplicating an Output File Descriptor
The redirection operator:is used to duplicate one output file descriptor from another, or to close one. If word evaluates to one or more digits, the file descriptor denoted by n, or standard output if n is not specified, will be made to be a copy of the file descriptor denoted by word; if the digits in word do not represent a file descriptor already open for output, a redirection error will result (see[n]>&word
Consequences of Shell Errors ). If word evaluates to "-", file descriptor n, or standard output if n is not specified, will be closed. If word evaluates to something else, the behaviour is unspecified.The construct 2>&1 is often used to redirect standard error to the same file as standard output. Since the redirections take place beginning to end, the order of redirections is significant. For example:
directs both standard output and standard error to file foo. However:ls > foo 2>&1
only directs standard output to file foo because standard error was duplicated as standard output before standard output was directed to file foo.ls 2>&1 > foo
Open File Descriptors for Reading and Writing
The redirection operator:will cause the file whose name is the expansion of word to be opened for both reading and writing on the file descriptor denoted by n, or standard input if n is not specified. If the file does not exist, it will be created.[n]<>word
The <> operator could be useful in writing an application that worked with several terminals, and occasionally wanted to start up a shell. That shell would in turn be unable to run applications that run from an ordinary controlling terminal unless it could make use of <> redirection. The specific example is a historical version of the pager more, which reads from standard error to get its commands, so standard input and standard output are both available for their usual usage. There is no way of saying the following in the shell without <>:
cat food | more - >/dev/tty03 2<>/dev/tty03
Another example of <> is one that opens /dev/tty on file descriptor 3 for reading and writing:
exec 3<> /dev/tty
An example of creating a lock file for a critical code region:
set -C until 2> /dev/null > lockfile do sleep 30 done set +C perform critical function rm lockfile
Since /dev/null is not a regular file, no error is generated by redirecting to it in noclobber mode.
Exit Status and Errors
Consequences of Shell Errors
For a non-interactive shell, an error condition encountered by a special built-in (see Special Built-in Utilities ) or other type of utility will cause the shell to write a diagnostic message to standard error and exit as shown in the following table:An expansion error is one that occurs when the shell expansions defined in
Error Special Built-in Other Utilities Shell language syntax error will exit will exit Utility syntax error (option or operand error) will exit will not exit Redirection error will exit will not exit Variable assignment error will exit will not exit Expansion error will exit will exit Command not found n/a may exit Dot script not found will exit n/a Word Expansions are carried out (for example, ${x!y}, because "!" is not a valid operator); an implementation may treat these as syntax errors if it is able to detect them during tokenisation, rather than during expansion.If any of the errors shown as "will (may) exit" occur in a subshell, the subshell will (may) exit with a non-zero status, but the script containing the subshell will not exit because of the error.
In all of the cases shown in the table, an interactive shell will write a diagnostic message to standard error without exiting.
Exit Status for Commands
Each command has an exit status that can influence the behaviour of other shell commands. The exit status of commands that are not utilities is documented in this section. The exit status of the standard utilities is documented in their respective sections. If a command is not found, the exit status will be 127. If the command name is found, but it is not an executable utility, the exit status will be 126. Applications that invoke utilities without using the shell should use these exit status values to report similar errors.
If a command fails during word expansion or redirection, its exit status will be greater than zero.
Internally, for purposes of deciding if a command exits with a non-zero exit status, the shell will recognise the entire status value retrieved for the command by the equivalent of the XSH specification wait() function WEXITSTATUS macro. When reporting the exit status with the special parameter ?, the shell will report the full eight bits of exit status available. The exit status of a command that terminated because it received a signal will be reported as greater than 128.
Shell Commands
This section describes the basic structure of shell commands. The following command descriptions each describe a format of the command that is only used to aid the reader in recognising the command type, and does not formally represent the syntax. Each description discusses the semantics of the command; for a formal definition of the command language, consult Shell Grammar .A command is one of the following:
- simple command (see
Simple Commands )
- pipeline (see
Pipelines )
- list or compound-list (see
Lists )
- compound command (see
Compound Commands )
- function definition (see
Function Definition Command ).
Unless otherwise stated, the exit status of a command is that of the last simple command executed by the command. There is no limit on the size of any shell command other than that imposed by the underlying system (memory constraints, {ARG_MAX}, and so on).
Simple Commands
A simple command is a sequence of optional variable assignments and redirections, in any sequence, optionally followed by words and redirections, terminated by a control operator. When a given simple command is required to be executed (that is, when any conditional construct such as an AND-OR list or a case statement has not bypassed the simple command), the following expansions, assignments and redirections will all be performed from the beginning of the command text to the end.
- The words that are recognised as variable assignments or redirections according to
Shell Grammar Rules are saved for processing in steps 3 and 4.
- The words that are not variable assignments or redirections will be expanded. If any fields remain following their expansion, the first field will be considered the command name and remaining fields will be the arguments for the command.
- Redirections will be performed as described in
Redirection .
- Each variable assignment will be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion and quote removal prior to assigning the value.
In the preceding list, the order of steps 3 and 4 may be reversed for the processing of special built-in utilities. See
Special Built-in Utilities .If no command name results, variable assignments will affect the current execution environment. Otherwise, the variable assignments will be exported for the execution environment of the command and will not affect the current execution environment (except for special built-ins). If any of the variable assignments attempt to assign a value to a read-only variable, a variable assignment error will occur. See
Consequences of Shell Errors for the consequences of these errors.If there is no command name, any redirections will be performed in a subshell environment; it is unspecified whether this subshell environment is the same one as that used for a command substitution within the command. (To affect the current execution environment, see the exec special built-in.) If any of the redirections performed in the current shell execution environment fail, the command will immediately fail with an exit status greater than zero, and the shell will write an error message indicating the failure. See
Consequences of Shell Errors for the consequences of these failures on interactive and non-interactive shells.If there is a command name, execution will continue as described in
Command Search and Execution . If there is no command name, but the command contained a command substitution, the command will complete with the exit status of the last command substitution performed. Otherwise, the command will complete with a zero exit status.The following example illustrates both how a variable assignment without a command name affects the current execution environment, and how an assignment with a command name only affects the execution environment of the command.
$ x=red $ echo $x red $ export x $ sh -c 'echo $x' red $ x=blue sh -c 'echo $x' blue $ echo $x red
This next example illustrates that redirections without a command name are still performed.
$ ls foo ls: foo: no such file or directory $ > foo $ ls foo foo
A command without a command name, but one that includes a command substitution, has an exit status of the last command substitution that the shell performed. For example:
if x=$(command) then ... fi
An example of redirections without a command name being performed in a subshell shows that the here-document does not disrupt the standard input of the while loop:
IFS=: while read a b do echo $a <<-eof Hello eof done </etc/passwd
Some examples of commands without command names in AND-OR lists:
> foo || { echo "error: foo cannot be created" >&2 exit 1 } # set saved if /vmunix.save exists test -f /vmunix.save && saved=1
Command substitution and redirections without command names both occur in subshells, but they are not necessarily the same ones. For example, in:
it is unspecified whether foo will be echoed to the file or to standard output.exec 3> file var=$(echo foo >&3) 3>&1
Command Search and Execution
If a simple command results in a command name and an optional list of arguments, the following actions will be performed.
- If the command name does not contain any slashes, the first successful step in the following sequence will occur:
- If the command name matches the name of a special built-in utility, that special built-in utility will be invoked.
- If the command name matches the name of a function known to this shell, the function will be invoked as described in
Function Definition Command . If the implementation has provided a standard utility in the form of a function, it will not be recognised at this point. It will be invoked in conjunction with the path search in step 1d.
- If the command name matches the name of a utility listed in the following table, that utility will be invoked.
alias false jobs true bg fc kill umask cd fg newgrp unalias command getopts read wait
- Otherwise, the command will be searched for using the PATH environment variable as described in the XBD specification, Chapter 6, Environment Variables:
- If the search is successful:
- If the system has implemented the utility as a regular built-in or as a shell function, it will be invoked at this point in the path search.
- Otherwise, the shell will execute the utility in a separate utility environment (see
Shell Execution Environment ) with actions equivalent to calling the XSH specification execve() function with the path argument set to the pathname resulting from the search, arg0 set to the command name, and the remaining arguments set to the operands, if any.If the execve() function fails due to an error equivalent to the XSH specification error [ENOEXEC], the shell will execute a command equivalent to having a shell invoked with the command name as its first operand, along with any remaining arguments passed along. If the executable file is not a text file, the shell may bypass this command execution, write an error message, and return an exit status of 126.
Once a utility has been searched for and found (either as a result of this specific search or as part of an unspecified shell startup activity), an implementation may remember its location and need not search for the utility again unless the PATH variable has been the subject of an assignment. If the remembered location fails for a subsequent invocation, the shell will repeat the search to find the new location for the utility, if any.
- If the search is unsuccessful, the command will fail with an exit status of 127 and the shell will write an error message.
If the command name contains at least one slash, the shell will execute the utility in a separate utility environment with actions equivalent to calling the XSH specification execve() function with the path and arg0 arguments set to the command name, and the remaining arguments set to the operands, if any. If the execve() function fails due to an error equivalent to the XSH specification error [ENOEXEC], the shell will execute a command equivalent to having a shell invoked with the command name as its first operand, along with any remaining arguments passed along. If the executable file is not a text file, the shell may bypass this command execution, write an error message and return an exit status of 126.
>
This description requires that the shell can execute shell scripts directly, even if the underlying system does not support the common #! interpreter convention. That is, if file foo contains shell commands and is executable, the following will execute foo:
./foo
The sequence selected for the ISO/IEC 9945-2:1993 standard acknowledges that special built-ins cannot be overridden, but gives the programmer full control over which versions of other utilities are executed. It provides a means of suppressing function lookup (via the command utility) for the user's own functions and ensures that any regular built-ins or functions provided by the implementation are under the control of the path search. The mechanisms for associating built-ins or functions with executable files in the path are not specified by this specification, but the wording requires that if either is implemented, the application will not be able to distinguish a function or built-in from an executable (other than in terms of performance, presumably). The implementation will ensure that all effects specified by this specification resulting from the invocation of the regular built-in or function (interaction with the environment, variables, traps, and so on) are identical to those resulting from the invocation of an executable file.
Example: Consider three versions of the ls utility:
- The application includes a shell function named ls.
- The user writes a utility named ls and puts it in /fred/bin.
- The example implementation provides ls as a regular shell built-in that will be invoked (either by the shell or directly by exec when the path search reaches the directory /posix/bin.
If PATH=/posix/bin, various invocations yield different versions of ls:
Invocation Version of ls ls (from within application script) (1) function command ls (from within application script) (3) built-in ls (from within makefile called by application) (3) built-in system("ls") (3) built-in PATH="/fred/bin:$PATH" ls (2) user's version Pipelines
A pipeline is a sequence of one or more commands separated by the control operator "|". The standard output of all but the last command will be connected to the standard input of the next command. The format for a pipeline is:
[!] command1 [ | command2 ...]
The standard output of command1 will be connected to the standard input of command2. The standard input, standard output or both of a command will be considered to be assigned by the pipeline before any redirection specified by redirection operators that are part of the command (see
Redirection ).If the pipeline is not in the background (see
Lists ), the shell will wait for the last command specified in the pipeline to complete, and may also wait for all commands to complete.Exit Status
If the reserved word "!" does not precede the pipeline, the exit status will be the exit status of the last command specified in the pipeline. Otherwise, the exit status is the logical NOT of the exit status of the last command. That is, if the last command returns zero, the exit status will be 1; if the last command returns greater than zero, the exit status will be zero.Because pipeline assignment of standard input or standard output or both takes place before redirection, it can be modified by redirection. For example:
sends both the standard output and standard error of command1 to the standard input of command2.$ command1 2>&1 | command2
The reserved word ! allows more flexible testing using AND and OR lists.
Lists
An AND-OR-list is a sequence of one or more pipelines separated by the operators: && ||
A list is a sequence of one or more AND-OR-lists separated by the operators:
; &
and optionally terminated by:
; & <newline>
The operators && and || have equal precedence and will be evaluated from beginning to end. For example, both of the following commands write solely bar to standard output:false && echo foo || echo bar true || echo foo && echo bar
A ";" or newline character terminator will cause the preceding AND-OR-list to be executed sequentially; an "&" will cause asynchronous execution of the preceding AND-OR-list.
The term compound-list is derived from the grammar in
Shell Grammar ; it is equivalent to a sequence of lists, separated by newline characters, that can be preceded or followed by an arbitrary number of newline characters.The following is an example that illustrates newline characters in compound-lists:
while # a couple of newlines # a list date && who || ls; cat file # a couple of newlines # another list wc file > output & true do # 2 lists ls cat file done
Asynchronous Lists
If a command is terminated by the control operator ampersand (&), the shell will execute the command asynchronously in a subshell. This means that the shell does not wait for the command to finish before executing the next command. The format for running a command in the background is:
command1 & [command2 & ...]
The standard input for an asynchronous list, before any explicit redirections are performed, will be considered to be assigned to a file that has the same properties as /dev/null. If it is an interactive shell, this need not happen. In all cases, explicit redirection of standard input will override this activity.
Since the connection of the input to the equivalent of /dev/null. is considered to occur before redirections, the following script would produce no output:
exec < /etc/passwd cat <&0 & wait
When an element of an asynchronous list (the portion of the list ended by an ampersand, such as command1, above) is started by the shell, the process ID of the last command in the asynchronous list element will become known in the current shell execution environment; see
Shell Execution Environment . This process ID will remain known until:
- The command terminates and the application waits for the process ID.
- Another asynchronous list invoked before $! (corresponding to the previous asynchronous list) is expanded in the current execution environment.
The implementation need not retain more than the {CHILD_MAX} most recent entries in its list of known process IDs in the current shell execution environment.
Exit Status: The exit status of an asynchronous list is zero.
Sequential Lists
Commands that are separated by a semicolon (;) will be executed sequentially.The format for executing commands sequentially is:
command1 [; command2] ...
Each command will be expanded and executed in the order specified.
Exit Status: The exit status of a sequential list will be the exit status of the last command in the list.
AND Lists
The control operator && denotes an AND list. The format is:command1 [ && command2] ...
First command1 will be executed. If its exit status is zero, command2 will be executed, and so on until a command has a non-zero exit status or there are no more commands left to execute. The commands will be expanded only if they are executed.
Exit Status: The exit status of an AND list will be the exit status of the last command that is executed in the list.
OR Lists
The control operator || denotes an OR List. The format is:command1 [ || command2] ...
First, command1 will be executed. If its exit status is non-zero, command2 will be executed, and so on until a command has a zero exit status or there are no more commands left to execute.
Exit Status: The exit status of an OR list will be the exit status of the last command that is executed in the list.
Compound Commands
The shell has several programming constructs that are compound commands, which provide control flow for commands. Each of these compound commands has a reserved word or control operator at the beginning, and a corresponding terminator reserved word or operator at the end. In addition, each can be followed by redirections on the same line as the terminator. Each redirection will apply to all the commands within the compound command that do not explicitly override that redirection. Grouping Commands
The format for grouping commands is as follows:
- (compound-list)
- Execute compound-list in a subshell environment; see
Shell Execution Environment . Variable assignments and built-in commands that affect the environment will not remain in effect after the list finishes.- { compound-list;}
- Execute compound-list in the current process environment. The semicolon shown here is an example of a control operator delimiting the "}" reserved word. Other delimiters are possible, as shown in
Shell Grammar ; a newline character is frequently used.Exit Status: The exit status of a grouping command will be the exit status of list.
For Loop
The for loop will execute a sequence of commands for each member in a list of items. The for loop requires that the reserved words do and done be used to delimit the sequence of commands.The format for the for loop is as follows:
First, the list of words following in will be expanded to generate a list of items. Then, the variable name will be set to each item, in turn, and the compound-list executed each time. If no items result from the expansion, the compound-list will not be executed. Omitting:for name [ in word ... ] do compound-list done
is equivalent to:in word ...
in "$@"
The format is shown with generous usage of newline characters. See the grammar in
Shell Grammar for a precise description of where newline characters and semicolons can be interchanged.Exit Status: The exit status of a for command will be the exit status of the last command that executes. If there are no items, the exit status will be zero.
Case Conditional Construct
The conditional construct case will execute the compound-list corresponding to the first one of several patterns (seePattern Matching Notation ) that is matched by the string resulting from the tilde expansion, parameter expansion, command substitution, and arithmetic expansion and quote removal of the given word. The reserved word in will denote the beginning of the patterns to be matched. Multiple patterns with the same compound-list are delimited by the "|" symbol. The control operator ")" terminates a list of patterns corresponding to a given action. The compound-list for each list of patterns is terminated with ;;. The case construct terminates with the reserved word esac (case reversed).The format for the case construct is as follows:
case word in [(]pattern1) compound-list;; [(]pattern2|pattern3) compound-list;; ... esac
The ;; is optional for the last compound-list.
In order from the beginning to the end of the case statement, each pattern that labels a compound-list is subjected to tilde expansion, parameter expansion, command substitution and arithmetic expansion, and the result of these expansions is compared against the expansion of word, according to the rules described in
Pattern Matching Notation (which also describes the effect of quoting parts of the pattern). After the first match, no more patterns are expanded, and the compound-list is executed. The order of expansion and comparison of multiple patterns that label a compound-list statement is unspecified.Exit Status: The exit status of case is zero if no patterns are matched. Otherwise, the exit status will be the exit status of the last command executed in the compound-list.
The pattern *, given as the last pattern in a case construct, is equivalent to the default case in a C-language switch statement.
The grammar shows that reserved words can be used as patterns, even if one is the first word on a line. Obviously, the reserved word esac cannot be used in this manner.
If Conditional Construct
The if command will execute a compound-list and use its exit status to determine whether to execute another compound-list.The format for the if construct is as follows:
if compound-list then compound-list [elif compound-list then compound-list] ... [else compound-list] fi
The if compound-list is executed; if its exit status is zero, the then compound-list is executed and the command will complete. Otherwise, each elif compound-list is executed, in turn, and if its exit status is zero, the then compound-list is executed and the command will complete. Otherwise, the else compound-list is executed.
Exit Status: The exit status of the if command will be the exit status of the then or else compound-list that was executed, or zero, if none was executed.
While Loop
The while loop continuously will execute one compound-list as long as another compound-list has a zero exit status.The format of the while loop is as follows:
while compound-list-1 do compound-list-2 done
The compound-list-1 will be executed, and if it has a non-zero exit status, the while command will complete. Otherwise, the compound-list-2 will be executed, and the process will repeat.
Exit Status: The exit status of the while loop will be the exit status of the last compound-list-2 executed, or zero if none was executed.
Until Loop
The until loop continuously will execute one compound-list as long as another compound-list has a non-zero exit status.The format of the until loop is as follows:
until compound-list-1 do compound-list-2 done
The compound-list-1 will be executed, and if it has a zero exit status, the until command will complete. Otherwise, the compound-list-2 will be executed, and the process will repeat.
Exit Status: The exit status of the until loop will be the exit status of the last compound-list-2 executed, or zero if none was executed.
Function Definition Command
A function is a user-defined name that is used as a simple command to call a compound command with new positional parameters. A function is defined with a function definition command. The format of a function definition command is as follows:
The function is named fname; it must be a name (see name in the XBD specification, Glossary ). An implementation may allow other characters in a function name as an extension. The implementation will maintain separate name spaces for functions and variables.fname() compound-command[io-redirect ...]
The () in the function definition command consists of two operators. Therefore, intermixing blank characters with the fname, "(" and ")" is allowed, but unnecessary.
The argument compound-command represents a compound command, as described in
Compound Commands .When the function is declared, none of the expansions in
Word Expansions will be performed on the text in compound-command or io-redirect; all expansions will be performed as normal each time the function is called. Similarly, the optional io-redirect redirections and any variable assignments within compound-command will be performed during the execution of the function itself, not the function definition. SeeConsequences of Shell Errors for the consequences of failures of these operations on interactive and non-interactive shells.When a function is executed, it will have the syntax-error and variable-assignment properties described for special built-in utilities in the enumerated list at the beginning of
Special Built-in Utilities .The compound-command will be executed whenever the function name is specified as the name of a simple command (see
Command Search and Execution ). The operands to the command temporarily will become the positional parameters during the execution of the compound-command; the special parameter "#" will also be changed to reflect the number of operands. The special parameter 0 will be unchanged. When the function completes, the values of the positional parameters and the special parameter "#" will be restored to the values they had before the function was executed. If the special built-in return is executed in the compound-command, the function will complete and execution will resume with the next command after the function call.An example of how a function definition can be used wherever a simple command is allowed:
# If variable i is equal to "yes", # define function foo to be ls -l # [ "$i" = yes ] && foo() { ls -l }
Exit Status
The exit status of a function definition will be zero if the function was declared successfully; otherwise, it will be greater than zero. The exit status of a function invocation will be the exit status of the last command executed by the function.
Shell Grammar
The following grammar defines the Shell Command Language. This formal syntax takes precedence over the preceding text syntax description. Shell Grammar Lexical Conventions
The input language to the shell must be first recognised at the character level. The resulting tokens will be classified by their immediate context according to the following rules (applied in order). These rules are used to determine what a "token" that is subject to parsing at the token level is. The rules for token recognition inToken Recognition will apply.
- A newline character will be returned as the token identifier NEWLINE.
- If the token is an operator, the token identifier for that operator will result.
- If the string consists solely of digits and the delimiter character is one of < or >, the token identifier IO_NUMBER will be returned.
- Otherwise, the token identifier TOKEN will result.
Further distinction on TOKEN is context-dependent. It may be that the same TOKEN yields WORD, a NAME, an ASSIGNMENT, or one of the reserved words below, dependent upon the context. Some of the productions in the grammar below are annotated with a rule number from the following list. When a TOKEN is seen where one of those annotated productions could be used to reduce the symbol, the applicable rule will be applied to convert the token identifier type of the TOKEN to a token identifier acceptable at that point in the grammar. The reduction will then proceed based upon the token identifier type yielded by the rule applied. When more than one rule applies, the highest numbered rule will apply (which in turn may refer to another rule). (Note that except in rule 7, the presence of an = in the token has no effect.)
The WORD tokens will have the word expansion rules applied to them immediately before the associated command is executed, not at the time the command is parsed.
Shell Grammar Rules
- [Command Name]
When the TOKEN is exactly a reserved word, the token identifier for that reserved word will result. Otherwise, the token WORD will be returned. Also, if the parser is in any state where only a reserved word could be the next correct token, proceed as above. This rule applies rather narrowly: when a compound list is terminated by some clear delimiter (such as the closing fi of an inner if_clause) then it would apply; where the compound list might continue (as in after a ;), rule 7a (and consequently the first sentence of this rule) would apply. In many instances the two conditions are identical, but this part of this rule does not give licence to treating a WORD as a reserved word unless it is in a place where a reserved word must appear.
- Note:
- Because at this point quote marks are retained in the token, quoted strings cannot be recognised as reserved words. This rule also implies that reserved words will not be recognised except in certain positions in the input, such as after a newline character or semicolon; the grammar presumes that if the reserved word is intended, it will be properly delimited by the user, and does not attempt to reflect that requirement directly. Also note that line joining is done before tokenisation, as described in
Escape Character (Backslash) , so escaped newlines are already removed at this point.Rule 1 is not directly referenced in the grammar, but is referred to by other rules, or applies globally.
- [Redirection to or from filename]
The expansions specified inRedirection will occur. As specified there, exactly one field can result (or the result is unspecified), and there are additional requirements on pathname expansion.
- [Redirection from here-document]
Quote removal will be applied to the word to determine the delimiter that will be used to find the end of the here-document that begins after the next newline character.
- [Case statement termination]
When the TOKEN is exactly the reserved word Esac, the token identifier for Esac will result. Otherwise, the token WORD will be returned.
- [NAME in for]
When the TOKEN meets the requirements for a name (see name in the XBD specification, Glossary ), the token identifier NAME will result. Otherwise, the token WORD will be returned.
- [Third word of for and case]
When the TOKEN is exactly the reserved word In, the token identifier for In will result. Otherwise, the token WORD will be returned. (As indicated in the grammar, a linebreak precedes the token In. If newline characters are present at the indicated location, it is the token after them that is treated in this fashion.)
- [Assignment preceding command name]
- [When the first word]
If the TOKEN does not contain the character "=", rule 1 will be applied. Otherwise, 7b will be applied.
- [Not the first word]
If the TOKEN contains the equal sign character:
- If it begins with =, the token WORD will be returned.
- If all the characters preceding = form a valid name (see name in the XSH specification), the token ASSIGNMENT_WORD will be returned. (Quoted characters cannot participate in forming a valid name.)
- Otherwise, it is unspecified whether it is ASSIGNMENT_WORD or WORD that is returned.
Assignment to the NAME will occur as specified in
Simple Commands .
- [NAME in function]
When the TOKEN is exactly a reserved word, the token identifier for that reserved word will result. Otherwise, when the TOKEN meets the requirements for a name (see Name), the token identifier NAME will result. Otherwise, rule 7 will apply.
- [Body of function]
Word expansion and assignment will never occur, even when required by the rules above, when this rule is being parsed. Each TOKEN that might either be expanded or have assignment applied to it will instead be returned as a single WORD consisting only of characters that are exactly the token described inToken Recognition .
/* ------------------------------------------------------- The grammar symbols ------------------------------------------------------- */ %token WORD %token ASSIGNMENT_WORD %token NAME %token NEWLINE %token IO_NUMBER /* The following are the operators mentioned above. */ %token AND_IF OR_IF DSEMI /* '&&' '||' ';;' */ %token DLESS DGREAT LESSAND GREATAND LESSGREAT DLESSDASH /* '<<' '>>' '<&' '>&' '<>' '<<-' */ %token CLOBBER /* '>|' */ /* The following are the reserved words. */ %token If Then Else Elif Fi Do Done /* 'if' 'then' 'else' 'elif' 'fi' 'do' 'done' */ %token Case Esac While Until For /* 'case' 'esac' 'while' 'until' 'for' */ /* These are reserved words, not operator tokens, and are recognised when reserved words are recognised. */ %token Lbrace Rbrace Bang /* '{' '}' '!' */ %token In /* 'in' */ /* ------------------------------------------------------- The Grammar ------------------------------------------------------- */ %start complete_command %% complete_command : list separator | list ; list : list separator_op and_or | and_or ; and_or : pipeline | and_or AND_IF linebreak pipeline | and_or OR_IF linebreak pipeline ; pipeline : pipe_sequence | Bang pipe_sequence ; pipe_sequence : command | pipe_sequence '|' linebreak command ; command : simple_command | compound_command | compound_command redirect_list | function_definition ; compound_command : brace_group | subshell | for_clause | case_clause | if_clause | while_clause | until_clause ; subshell : '(' compound_list ')' ; compound_list : term | newline_list term | term separator | newline_list term separator ; term : term separator and_or | and_or ; for_clause : For name linebreak do_group | For name linebreak in wordlist sequential_sep do_group ; name : NAME /* Apply rule 5 */ ; in : In /* Apply rule 6 */ ; wordlist : wordlist WORD | WORD ; case_clause : Case WORD linebreak in linebreak case_list Esac | Case WORD linebreak in linebreak Esac ; case_list : case_list case_item | case_item ; case_item : pattern ')' linebreak DSEMI linebreak | pattern ')' compound_list DSEMI linebreak | '(' pattern ')' linebreak DSEMI linebreak | '(' pattern ')' compound_list DSEMI linebreak ; pattern : WORD /* Apply rule 4 */ | pattern '|' WORD /* Do not apply rule (4) */ ; if_clause : If compound_list Then compound_list else_part Fi | If compound_list Then compound_list Fi ; else_part : Elif compound_list Then else_part | Else compound_list ; while_clause : While compound_list do_group ; until_clause : Until compound_list do_group ; function_definition : fname '(' ')' linebreak function_body ; function_body : compound_command /* Apply rule 9 */ | compound_command redirect_list /* Apply rule 9 */ ; fname : NAME /* Apply rule 8 */ ; brace_group : Lbrace compound_list Rbrace ; do_group : Do compound_list Done ; simple_command : cmd_prefix cmd_word cmd_suffix | cmd_prefix cmd_word | cmd_prefix | cmd_name cmd_suffix | cmd_name ; cmd_name : WORD /* Apply rule 7a */ ; cmd_word : WORD /* Apply rule 7b */ ; cmd_prefix : io_redirect | cmd_prefix io_redirect | ASSIGNMENT_WORD | cmd_prefix ASSIGNMENT_WORD ; cmd_suffix : io_redirect | cmd_suffix io_redirect | WORD | cmd_suffix WORD ; redirect_list : io_redirect | redirect_list io_redirect ; io_redirect : io_file | IO_NUMBER io_file | io_here | IO_NUMBER io_here ; io_file : '<' filename | LESSAND filename | '>' filename | GREATAND filename | DGREAT filename | LESSGREAT filename | CLOBBER filename ; filename : WORD /* Apply rule 2 */ ; io_here : DLESS here_end | DLESSDASH here_end ; here_end : WORD /* Apply rule 3 */ ; newline_list : NEWLINE | newline_list NEWLINE ; linebreak : newline_list | /* empty */ ; separator_op : '&' | ';' ; separator : separator_op linebreak | newline_list ; sequential_sep : ';' linebreak | newline_list ;
There are several subtle aspects of this grammar where conventional usage implies rules about the grammar that in fact are not true.
For compound_list, only the forms that end in a separator allow a reserved word to be recognised, so usually only a separator can be used where a compound list precedes a reserved word (such as Then, Else, Do and Rbrace). Explicitly requiring a separator would disallow such valid (if rare) statements as:
if (false) then (echo x) else (echo y) fi
See the Note under special grammar rule 1.
Note that the bodies of here-documents are handled by token recognition (see
Token Recognition ) and do not appear in the grammar directly. (However, the here-document I/O redirection operator is handled as part of the grammar.)The start symbol of the grammar (complete_command) represents either input from the command line or a shell script. It is repeatedly applied by the interpreter to its input, and represents a single chunk of that input as seen by the interpreter.
Signals and Error Handling
When a command is in an asynchronous list, the shell will prevent SIGQUIT and SIGINT signals from the keyboard from interrupting the command. Otherwise, signals will have the values inherited by the shell from its parent (see also the trap special built-in).When a signal for which a trap has been set is received while the shell is waiting for the completion of a utility executing a foreground command, the trap associated with that signal will not be executed until after the foreground command has completed. When the shell is waiting, by means of the wait utility, for asynchronous commands to complete, the reception of a signal for which a trap has been set will cause the wait utility to return immediately with an exit status >128, immediately after which the trap associated with that signal will be taken.
If multiple signals are pending for the shell for which there are associated trap actions, the order of execution of trap actions is unspecified.
Shell Execution Environment
A shell execution environment consists of the following:
- open files inherited upon invocation of the shell, plus open files controlled by exec
- working directory as set by cd
- file creation mask set by umask
- current traps set by trap
- shell parameters that are set by variable assignment (see the set special built-in) or from the XSH specification environment inherited by the shell when it begins (see the export special built-in)
- shell functions (see
Function Definition Command )
- options turned on at invocation or by set
- process IDs of the last commands in asynchronous lists known to this shell environment; see
Lists
- shell aliases (see
Alias Substitution ).
Utilities other than the special built-ins (see
Special Built-in Utilities ) will be invoked in a separate environment that consists of the following. The initial value of these objects will be the same as that for the parent shell, except as noted below.
- open files inherited on invocation of the shell, open files controlled by the exec special built-in plus any modifications and additions specified by any redirections to the utility
- current working directory
- file creation mask
- if the utility is a shell script, traps caught by the shell will be set to the default values and traps ignored by the shell will be set to be ignored by the utility; if the utility is not a shell script, the trap actions (default or ignore) will be mapped into the appropriate signal handling actions for the utility
- variables with the export attribute, along with those explicitly exported for the duration of the command, will be passed to the utility as XSH specification environment variables.
The environment of the shell process will not be changed by the utility unless explicitly specified by the utility description (for example, cd and umask).
A subshell environment will be created as a duplicate of the shell environment, except that signal traps set by that shell environment will be set to the default values. Changes made to the subshell environment will not affect the shell environment. Command substitution, commands that are grouped with parentheses and asynchronous lists will be executed in a subshell environment. Additionally, each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment. All other commands will be executed in the current shell environment.
Some systems have implemented the last stage of a pipeline in the current environment so that commands such as:
set variable foo in the current environment. This extension is allowed, but not required; therefore, a shell programmer should consider a pipeline to be in a subshell environment, but not depend on it.command | read foo
Pattern Matching Notation
The pattern matching notation described in this section is used to specify patterns for matching strings in the shell. Historically, pattern matching notation is related to, but slightly different from, the regular expression notation described in the XBD specification, Chapter 7, Regular Expressions. For this reason, the description of the rules for this pattern matching notation are based on the description of regular expression notation. Patterns Matching a Single Character
The following patterns matching a single character match a single character: ordinary characters, special pattern characters and pattern bracket expressions. The pattern bracket expression will also match a single collating element. An ordinary character is a pattern that matches itself. It can be any character in the supported character set except for NUL, those special shell characters in
Quoting that require quoting, and the following three special pattern characters. Matching is based on the bit pattern used for encoding the character, not on the graphic representation of the character. If any character (ordinary, shell special or pattern special) is quoted, that pattern will match the character itself. The shell special characters always require quoting.When unquoted and outside a bracket expression, the following three characters will have special meaning in the specification of patterns:
- ?
- A question-mark is a pattern that will match any character.
- *
- An asterisk is a pattern that will match multiple characters, as described in
Patterns Matching Multiple Characters .- [
- The open bracket will introduce a pattern bracket expression.
The description of basic regular expression bracket expressions in the XBD specification, Section 7.3.5, RE Bracket Expression also applies to the pattern bracket expression, except that the exclamation-mark character (!) replaces the circumflex character (^) in its role in a non-matching list in the regular expression notation. A bracket expression starting with an unquoted circumflex character produces unspecified results.
The restriction on a circumflex in a bracket expression is to allow implementations that support pattern matching using the circumflex as the negation character in addition to the exclamation-mark. A portable application must use something like [\^!] to match either character.
When pattern matching is used where shell quote removal is not performed (such as in the argument to the find -name primary when find is being called using one of the XSH specification exec functions, or in the pattern argument to the fnmatch() function), special characters can be escaped to remove their special meaning by preceding them with a backslash character. This escaping backslash will be discarded. The sequence \\ represents one literal backslash. All of the requirements and effects of quoting on ordinary, shell special and special pattern characters will apply to escaping in this context.
Both quoting and escaping are described here because pattern matching must work in three separate circumstances:
- Calling directly upon the shell, such as in pathname expansion or in a case statement. All of the following will match the string or file abc:
The following will not:abc "abc" a"b"c a\bc a[b]c a["b"]c a[\b]c a["\b"]c a?c a*c
"a?c" a\*c a\[b]c
- Calling a utility or function without going through a shell, as described for find and the XSH specification function fnmatch().
- Calling utilities such as find, cpio, tar or pax through the shell command line. In this case, shell quote removal is performed before the utility sees the argument. For example, in:
after quote removal, the backslashes are presented to find and it treats them as escape characters. Both precede ordinary characters, so the c and h represent themselves and echo would be found on many historical systems (that have it in /bin). To find a filename that contained shell special characters or pattern characters, both quoting and escaping are required, such as:find /bin -name "e\c[\h]o" -print
to extract a filename ending with a(?.pax -r ... "*a\(\?"
Conforming applications are required to quote or escape the shell special characters (sometimes called metacharacters). If used without this protection, syntax errors can result or implementation extensions can be triggered. For example, the KornShell supports a series of extensions based on parentheses in patterns.
Patterns Matching Multiple Characters
The following rules are used to construct patterns matching multiple characters from patterns matching a single character:
- The asterisk (*) is a pattern that will match any string, including the null string.
- The concatenation of patterns matching a single character is a valid pattern that will match the concatenation of the single characters or collating elements matched by each of the concatenated patterns.
- The concatenation of one or more patterns matching a single character with one or more asterisks is a valid pattern. In such patterns, each asterisk will match a string of zero or more characters, matching the greatest possible number of characters that still allows the remainder of the pattern to match the string.
Since each asterisk matches zero or more occurrences, the patterns a*b and a**b have identical functionality.
Examples
- a[bc]
- matches the strings ab and ac.
- a*d
- matches the strings ad, abd and abcd, but not the string abc.
- a*d*
- matches the strings ad, abcd, abcdef, aaaad and adddd.
- *a*d
- matches the strings ad, abcd, efabcd, aaaad and adddd.
Patterns Used for Filename Expansion
The rules described so far in Patterns Matching a Single Character andPatterns Matching Multiple Characters are qualified by the following rules that apply when pattern matching notation is used for filename expansion.
- The slash character in a pathname must be explicitly matched by using one or more slashes in the pattern; it cannot be matched by the asterisk or question-mark special characters or by a bracket expression. Slashes in the pattern are identified before bracket expressions; thus, a slash cannot be included in a pattern bracket expression used for filename expansion. For example, the pattern a[b/c]d will not match such pathnames as abd or a/d. It will only match a pathname of literally a[b/c]d.
- If a filename begins with a period (.) the period must be explicitly matched by using a period as the first character of the pattern or immediately following a slash character. The leading period will not be matched by:
- the asterisk or question-mark special characters
- a bracket expression containing a non-matching list, such as:
a range expression, such as:[!a]
or a character class expression, such as:[%-0]
[[:punct:]]
It is unspecified whether an explicit period in a bracket expression matching list, such as:
can match a leading period in a filename.[.abc]
- Specified patterns are matched against existing filenames and pathnames, as appropriate. Each component that contains a pattern character requires read permission in the directory containing that component. Any component, except the last, that does not contain a pattern character requires search permission. For example, given the pattern:
search permission is needed for directories / and foo, search and read permissions are needed for directory bar, and search permission is needed for each x* directory. If the pattern matches any existing filenames or pathnames, the pattern will be replaced with those filenames and pathnames, sorted according to the collating sequence in effect in the current locale. If the pattern contains an invalid bracket expression or does not match any existing filenames or pathnames, the pattern string is left unchanged./foo/bar/x*/bam
Special Built-in Utilities
The following special built-in utilities will be supported in the shell command language. The output of each command, if any, will be written to standard output, subject to the normal redirection and piping possible with all commands. The term built-in implies that the shell can execute the utility directly and does not need to search for it. An implementation can choose to make any utility a built-in; however, the special built-in utilities described here differ from regular built-in utilities in two respects:
- A syntax error in a special built-in utility may cause a shell executing that utility to abort, while a syntax error in a regular built-in utility will not cause a shell executing that utility to abort. (See
Consequences of Shell Errors for the consequences of errors on interactive and non-interactive shells.) If a special built-in utility encountering a syntax error does not abort the shell, its exit value will be non-zero.
- Variable assignments specified with special built-in utilities will remain in effect after the built-in completes; this is not the case with a regular built-in or other utility.
The special built-in utilities in this section need not be provided in a manner accessible via the XSH specification exec family of functions.
Some of the special built-ins are described as conforming to the XBD specification, Utility Syntax Guidelines . For those that are not, the requirement in Utility Description Defaults
that -- be recognised as a first argument to be discarded does not apply and a portable application must not use that argument.
break - Exit From for, while or until Loop
SYNOPSIS
break [n]
DESCRIPTION
Exit from the smallest enclosing for, while or until loop, if any; or from the nth enclosing loop if n is specified. The value of n is an unsigned decimal integer greater than or equal to 1. The default is equivalent to n=1. If n is greater than the number of enclosing loops, the last enclosing loop is exited from. Execution will continue with the command immediately following the loop.EXAMPLES
for i in * do if test -d "$i" then break fi done
EXIT STATUS
- 0
- Successful completion.
- >0
- The n value was not an unsigned decimal integer greater than or equal to 1.
colon - Null Utility
SYNOPSIS
: [argument ...]
DESCRIPTION
This utility will only expand command arguments. It is used when a command is needed, as in the then condition of an if command, but nothing is to be done by the command.EXAMPLES
: ${X=abc} if false then : else echo $X fi abc
As with any of the special built-ins, the null utility can also have variable assignments and redirections associated with it, such as:
which sets variable x to the value y (so that it persists after the null utility completes) and creates or truncates file z.x=y : > z
EXIT STATUS
Zero.
continue - Continue for, while or until Loop
SYNOPSIS
continue [n]
DESCRIPTION
The continue utility will return to the top of the smallest enclosing for, while or until loop, or to the top of the nth enclosing loop, if n is specified. This involves repeating the condition list of a while or until loop or performing the next assignment of a for loop, and reexecuting the loop if appropriate.The value of n is a decimal integer greater than or equal to 1. The default is equivalent to n=1. If n is greater than the number of enclosing loops, the last enclosing loop is used.
EXAMPLES
for i in * do if test -d "$i" then continue fi done
EXIT STATUS
- 0
- Successful completion.
- >0
- The n value was not an unsigned decimal integer greater than or equal to 1.
dot - Execute Commands in Current Environment
SYNOPSIS
. file
DESCRIPTION
The shell will execute commands from the file in the current environment.If file does not contain a slash, the shell will use the search path specified by to find the directory containing file. Unlike normal command search, however, the file searched for by the dot utility need not be executable. If no readable file is found, a non-interactive shell will abort; an interactive shell will write a diagnostic message to standard error, but this condition will not be considered a syntax error.
EXAMPLES
cat foobar foo=hello bar=world . foobar echo $foo $bar hello world
EXIT STATUS
Returns the value of the last command executed, or a zero exit status if no command is executed.
eval - Construct Command by Concatenating Arguments
SYNOPSIS
eval [argument ...]
DESCRIPTION
The eval utility will construct a command by concatenating arguments together, separating each with a space character. The constructed command will be read and executed by the shell.EXAMPLES
foo=10 x=foo y='$'$x echo $y $foo eval y='$'$x echo $y 10
EXIT STATUS
If there are no arguments, or only null arguments, eval will return a zero exit status; otherwise, it will return the exit status of the command defined by the string of concatenated arguments separated by spaces.
exec - Execute Commands and Open, Close or Copy File Descriptors
SYNOPSIS
exec [command [argument ...]]
DESCRIPTION
The exec utility will open, close or copy file descriptors as specified by any redirections as part of the command.If exec is specified without command or arguments, and any file descriptors with numbers > 2 are opened with associated redirection statements, it is unspecified whether those file descriptors remain open when the shell invokes another utility. Scripts concerned that child shells could misuse open file descriptors can always close them explicitly, as shown in one of the following examples.
If exec is specified with command, it will replace the shell with command without creating a new process. If arguments are specified, they are arguments to command. Redirection will affect the current shell execution environment.
EXAMPLES
Open readfile as file descriptor 3 for reading:
exec 3< readfile
Open writefile as file descriptor 4 for writing:
exec 4> writefile
Make unit 5 a copy of unit 0:
exec 5<&0
Close file unit 3:
exec 3<&-
Cat the file maggie by replacing the current shell with the cat utility:
exec cat maggie
EXIT STATUS
If command is specified, exec will not return to the shell; rather, the exit status of the process will be the exit status of the program implementing command, which overlaid the shell. If command is not found, the exit status will be 127. If command is found, but it is not an executable utility, the exit status will be 126. If a redirection error occurs (seeConsequences of Shell Errors ), the shell will exit with a value in the range 1-125. Otherwise, exec will return a zero exit status.
exit - Cause the Shell to Exit
SYNOPSIS
exit [n]
DESCRIPTION
The exit utility causes the shell to exit with the exit status specified by the unsigned decimal integer n. If n is specified, but its value is not between 0 and 255 inclusively, the exit status is undefined.As explained in other sections, certain exit status values have been reserved for special uses and should be used by applications only for those purposes:
- 126
- A command to be executed was found, but the file was not an executable utility.
- 127
- A command to be executed was not found.
- >128
- A command was interrupted by a signal.
A trap on EXIT will be executed before the shell terminates, except when the exit utility is invoked in that trap itself, in which case the shell will exit immediately.
EXAMPLES
Exit with a true value:
exit 0
Exit with a false value:
exit 1
EXIT STATUS
The exit status will be n, if specified. Otherwise, the value will be the exit value of the last command executed, or zero if no command was executed. When exit is executed in a trap action, the last command is considered to be the command that executed immediately preceding the trap action.
export - Set Export Attribute for Variables
SYNOPSIS
export name[=word]...
export -pDESCRIPTION
The shell will give the export attribute to the variables corresponding to the specified names, which will cause them to be in the environment of subsequently executed commands.The export special built-in supports the XBD specification, Utility Syntax Guidelines .
When -p is specified, export will write to the standard output the names and values of all exported variables, in the following format:
"export %s=%s\n", <name>, <value>
The -p option allows portable access to the values that can be saved and then later restored using, for instance, a dot script.
The shell will format the output, including the proper use of quoting, so that it is suitable for reinput to the shell as commands that achieve the same exporting results.
When no arguments are given, the results are unspecified.
EXAMPLES
Export and variables:
export PWD HOME
Set and export the variable:
export PATH=/local/bin:$PATH
Save and restore all exported variables:
export -p > temp-file unset a lot of variables ... processing . temp-file
EXIT STATUS
Zero.
readonly - Set Read-only Attribute for Variables
SYNOPSIS
readonly name[=word]...
readonly -pDESCRIPTION
The variables whose names are specified will be given the readonly attribute. The values of variables with the read-only attribute cannot be changed by subsequent assignment, nor can those variables be unset by the unset utility.Some versions of the shell exist that preserve the read-only attribute across separate invocations. This specification allows this behaviour, but does not require it.
The readonly special built-in supports the XBD specification, Utility Syntax Guidelines .
When -p is specified, readonly will write to the standard output the names and values of all read-only variables, in the following format:
"readonly %s=%s\n", <name>, <value>
The shell will format the output, including the proper use of quoting, so that it is suitable for reinput to the shell as commands that achieve the same attribute-setting results.
The -p option allows portable access to the values that can be saved and then later restored using, for instance, a dot script. (See a related example under export.)
EXAMPLES
readonly HOME PWD
EXIT STATUS
Zero.
return - Return from a Function
SYNOPSIS
return [n]
DESCRIPTION
The return utility will cause the shell to stop executing the current function or dot script. If the shell is not currently executing a function or dot script, the results are unspecified.The results of returning a number greater than 255 are undefined because of differing practices in the various historical implementations. Some shells AND out all but the low-order 8 bits; others allow larger values, but not of unlimited size.
See the discussion of appropriate exit status values under exit.
EXIT STATUS
The value of the special parameter ? will be set to n, an unsigned decimal integer, or to the exit status of the last command executed if n is not specified. If the value of n is greater than 255, the results are undefined. When return is executed in a trap action, the last command is considered to be the command that executed immediately preceding the trap action.
set - Set or Unset Options and Positional Parameters
SYNOPSIS
set [-abCefmnuvx][-h][-o option][argument...] set [+abCefmnuvx][+h][-o option][argument...] set --[argument...] set -[argument...]
DESCRIPTION
If no options or arguments are specified, set will write the names and values of all shell variables in the collation sequence of the current locale. Each name will start on a separate line, using the format:
"%s=%s\n", <name>, <value>
The value string will be written with appropriate quoting so that it is suitable for reinput to the shell, setting or resetting, as far as possible, the variables that are currently set. Read-only variables cannot be reset. See the description of shell quoting in
Quoting .When options are specified, they will set or unset attributes of the shell, as described below. When arguments are specified, they will cause positional parameters to be set or unset, as described below. Setting or unsetting attributes and positional parameters are not necessarily related actions, but they can be combined in a single invocation of set.
The set special built-in supports the XBD specification, Utility Syntax Guidelines except that options can be specified with either a leading hyphen (meaning enable the option) or plus sign (meaning disable it).
Implementations support the options in the following list in both their hyphen and plus-sign forms. These options can also be specified as options to sh.
- -a
- When this option is on, the export attribute will be set for each variable to which an assignment is performed. (See Assignment in the XBD specification, Glossary .) If the assignment precedes a utility name in a command, the export attribute will not persist in the current execution environment after the utility completes, with the exception that preceding one of the special built-in utilities will cause the export attribute to persist after the built-in has completed. If the assignment does not precede a utility name in the command, or if the assignment is a result of the operation of the getopts or read utilities, the export attribute will persist until the variable is unset.
- -b
- Cause the shell to notify the user asynchronously of background job completions. The following message will be written to standard error:
where the fields are as follows:"[%d]%c %s%s\n", <job-number>, <current>, <status>, <job-name>
- <current>
- The character "+" identifies the job that would be used as a default for the fg or bg utilities; this job can also be specified using the job_id %+ or %%. The character "-" identifies the job that would become the default if the current default job were to exit; this job can also be specified using the job_id %-. For other jobs, this field is a space character. At most one job can be identified with "+" and at most one job can be identified with "-". If there is any suspended job, then the current job will be a suspended job. If there are at least two suspended jobs, then the previous job will also be a suspended job.
- <job-number>
- A number that can be used to identify the process group to the wait, fg, bg and kill utilities. Using these utilities, the job can be identified by prefixing the job number with "%".
- <status>
- Unspecified.
- <job-name>
- Unspecified.
When the shell notifies the user a job has been completed, it may remove the job's process ID from the list of those known in the current shell execution environment; see
Lists . Asynchronous notification will not be enabled by default.
- -C
- (Upper-case C.) Prevent existing files from being overwritten by the shell's ">" redirection operator (see
Redirecting Output ); the >| redirection operator will override this noclobber option for an individual file.
- -e
- When this option is on, if a simple command fails for any of the reasons listed in
Consequences of Shell Errors or returns an exit status value >0, and is not part of the compound list following a while, until or if keyword, and is not a part of an AND or OR list, and is not a pipeline preceded by the "!" reserved word, then the shell will immediately exit.
- -f
- The shell will disable pathname expansion.
- -h
- Locate and remember utilities invoked by functions as those functions are defined (the utilities are normally located when the function is executed).
- -m
- All jobs are run in their own process groups. Immediately before the shell issues a prompt after completion of the background job, a message reporting the exit status of the background job will be written to standard error. If a foreground job stops, the shell will write a message to standard error to that effect, formatted as described by the jobs utility. In addition, if a job changes status other than exiting (for example, if it stops for input or output or is stopped by a SIGSTOP signal), the shell will write a similar message immediately prior to writing the next prompt. This option is enabled by default for interactive shells.
- -n
- The shell will read commands but not execute them; this can be used to check for shell script syntax errors. An interactive shell may ignore this option.
- -o option
- Set various options, many of which are equivalent to the single option letters. The following values of option are supported:
- allexport
- Equivalent to -a.
- errexit
- Equivalent to -e.
- ignoreeof
- Prevent an interactive shell from exiting on end-of-file. This setting prevents accidental logouts when control-D is entered. A user must explicitly exit to leave the interactive shell.
- monitor
- Equivalent to -m.
- noclobber
- Equivalent to -C (upper-case C).
- noglob
- Equivalent to -f.
- noexec
- Equivalent to -n.
- nolog
- Prevent the entry of function definitions into the command history. See
Command History List .- notify
- Equivalent to -b.
- nounset
- Equivalent to -u.
- verbose
- Equivalent to -v.
- vi
- Allow shell command-line editing using the built-in vi editor. Enabling vi mode disables any other command-line editing mode provided as an implementation extension. It need not be possible to set vi mode on for certain block-mode terminals.
- xtrace
- Equivalent to -x.
- -u
- The shell will write a message to standard error when it tries to expand a variable that is not set and immediately exit. An interactive shell will not exit.
- -v
- The shell will write its input to standard error as it is read.
- -x
- The shell will write to standard error a trace for each command after it expands the command and before it executes it.
The default for all these options is off (unset) unless the shell was invoked with them on (see sh). All the positional parameters will be unset before any new values are assigned.
The remaining arguments will be assigned in order to the positional parameters. The special parameter "#" will be set to reflect the number of positional parameters.
The special argument -- immediately following the set command name can be used to delimit the arguments if the first argument begins with "+" or "-", or to prevent inadvertent listing of all shell variables when there are no arguments. The command set -- without argument will unset all positional parameters and set the special parameter "#" to zero.
In the obsolescent version, the set command name followed by "-" with no other arguments will turn off the -v and -x options without changing the positional parameters. The set command name followed by "-" with other arguments will turn off the -v and -x options and assign the arguments to the positional parameters in order.
EXAMPLES
Write out all variables and their values:Set $1, $2 and $3 and set $# to 3:set
set c a b
Turn on the -x and -v options:
set -xv
Unset all positional parameters:
set - -
Set $1 to the value of x, even if x begins with - or +:
set - - "$x"
Set the positional parameters to the expansion of x, even if x expands with a leading - or +:
set - - $x
EXIT STATUS
Zero.
shift - Shift Positional Parameters
SYNOPSIS
shift [n]
DESCRIPTION
The positional parameters will be shifted. Positional parameter 1 will be assigned the value of parameter (1+n), parameter 2 will be assigned the value of parameter (2+n), and so on. The parameters represented by the numbers $# down to $#-n+1 will be unset, and the parameter "#" will be updated to reflect the new number of positional parameters.The value n will be an unsigned decimal integer less than or equal to the value of the special parameter "#". If n is not given, it will be assumed to be 1. If n is 0, the positional and special parameters will not be changed.
EXAMPLES
$ set a b c d e $ shift 2 $ echo $* c d e
EXIT STATUS
The exit status will be >0 if n>$#; otherwise, it will be zero.
times - Write Process Times
SYNOPSIS
times
DESCRIPTION
Write the accumulated user and system times for the shell and for all of its child processes, in the following POSIX locale format:
"%dm%fs %dm%fs\n%dm%fs %dm%fs\n", <shell user minutes>, <shell user seconds>, <shell system minutes>, <shell system seconds>, <children user minutes>, <children user seconds>, <children system minutes> <children system seconds>
The four pairs of times correspond to the members of the XSH specification <sys/times.h> tms structure as returned by times(): tms_utime, tms_stime, tms_cutime and tms_cstime, respectively.
EXAMPLES
$ times 0m0.43s 0m1.11s 8m44.18s 1m43.23s
EXIT STATUS
Zero.
trap - Trap Signals
SYNOPSIS
trap [action condition ...]
DESCRIPTION
If action is "-", the shell will reset each condition to the default value. If action is null ("), the shell will ignore each specified condition if it arises. Otherwise, the argument action will be read and executed by the shell when one of the corresponding conditions arises. The action of the trap will override a previous action (either default action or one explicitly set). The value of $? after the trap action completes will be the value it had before the trap was invoked.The condition can be EXIT, 0 (equivalent to EXIT) or a signal specified using a symbolic name, without the SIG prefix, as listed in the tables of signal names in the XSH specification under <signal.h>; for example, HUP, INT, QUIT, TERM. Implementations may permit lower-case signal names or names with the SIG prefix as an extension. Setting a trap for SIGKILL or SIGSTOP produces undefined results.
The environment in which the shell executes a trap on EXIT will be identical to the environment immediately after the last command executed before the trap on EXIT was taken.
Each time the trap is invoked, the action argument will be processed in a manner equivalent to:
eval "$action"
Signals that were ignored on entry to a non-interactive shell cannot be trapped or reset, although no error need be reported when attempting to do so. An interactive shell may reset or catch signals ignored on entry. Traps will remain in place for a given shell until explicitly changed with another trap command.
When a subshell is entered, traps that are not being ignored are set to the default actions. This does not imply that the trap command cannot be used within the subshell to set new traps.
The trap command with no arguments will write to standard output a list of commands associated with each condition. The format is:
"trap - - %s %s ...\n", <action>, <condition>
The shell will format the output, including the proper use of quoting, so that it is suitable for reinput to the shell as commands that achieve the same trapping results. For example:
save_traps=$(trap) ... eval "$save_traps"
XSI-conformant systems also allow numeric signal numbers for the conditions, corresponding to the following signal names:
Signal Number Signal Name 1 SIGHUP 2 SIGINT 3 SIGQUIT 6 SIGABRT 9 SIGKILL 14 SIGALRM 15 SIGTERM The trap special built-in supports the XBD specification, Utility Syntax Guidelines .
EXAMPLES
Write out a list of all traps and actions:trap
Set a trap so the logout utility in the directory will execute when the shell terminates:
trap '$HOME/logout' EXIT
or:
trap '$HOME/logout' 0
Unset traps on INT, QUIT, TERM and EXIT:
trap - INT QUIT TERM EXIT
EXIT STATUS
If the trap name or number is invalid, a non-zero exit status will be returned; otherwise, zero will be returned. For both interactive and non-interactive shells, invalid signal names or numbers will not be considered a syntax error and will not cause the shell to abort.
unset - Unset Values and Attributes of Variables and Functions
SYNOPSIS
unset [-fv] name ...
DESCRIPTION
Each variable or function specified by name will be unset.If -v is specified, name refers to a variable name and the shell will unset it and remove it from the environment. Read-only variables cannot be unset.
If -f is specified, name refers to a function and the shell will unset the function definition.
If neither -f nor -v is specified, name refers to a variable; if a variable by that name does not exist, it is unspecified whether a function by that name, if any, will be unset.
Unsetting a variable or function that was not previously set is not considered an error and will not cause the shell to abort.
The unset special built-in supports the XBD specification, Utility Syntax Guidelines .
Note that:
is not equivalent to an unset of VARIABLE; in the example, VARIABLE is set to "". Also, the variables that can be unset should not be misinterpreted to include the special parameters (seeVARIABLE=
Special Parameters ).EXAMPLES
Unset variable:unset -v VISUAL
Unset the functions foo and bar:
unset -f foo bar
EXIT STATUS
- 0
- All cases of name were successfully unset.
- >0
- At least one name could not be unset.