Previous section   Next section

Practical Programming in Tcl & Tk, Third Edition
By Brent B. Welch

Table of Contents
Chapter 11.  Regular Expressions

The regexp Command

The regexp command provides direct access to the regular expression matcher. Not only does it tell you whether a string matches a pattern, it can also extract one or more matching substrings. The return value is 1 if some part of the string matches the pattern; it is 0 otherwise. Its syntax is:

regexp ?flags? pattern string ?match sub1 sub2...?

The flags are described in Table 11-6:

Table 11-6. Options to the regexp command.
-nocaseLowercase characters in pattern can match either lowercase or uppercase letters in string.
-indicesThe match variables each contain a pair of numbers that are in indices delimiting the match within string. Otherwise, the matching string itself is copied into the match variables.
-expandedThe pattern uses the expanded syntax discussed on page 144.
-lineThe same as specifying both -lineanchor and -linestop.
-lineanchorChange the behavior of ^ and $ so they are line-oriented as discussed on page 143.
-linestopChange matching so that . and character classes do not match newlines as discussed on page 143.
-aboutUseful for debugging. It returns information about the pattern instead of trying to match it against the input.
--Signals the end of the options. You must use this if your pattern begins with -.

The pattern argument is a regular expression as described earlier. If string matches pattern, then the results of the match are stored in the variables named in the command. These match variable arguments are optional. If present, match is set to be the part of the string that matched the pattern. The remaining variables are set to be the substrings of string that matched the corresponding subpatterns in pattern. The correspondence is based on the order of left parentheses in the pattern to avoid ambiguities that can arise from nested subpatterns.

Example 11-2 uses regexp to pick the hostname out of the DISPLAY environment variable, which has the form:

Example 11-2 Using regular expressions to parse a string.
set env(DISPLAY) sage:0.1
regexp {([^:]*):}$env(DISPLAY) match host
=> 1
set match
=> sage:
set host
=> sage

The pattern involves a complementary set, [^:], to match anything except a colon. It uses repetition, *, to repeat that zero or more times. It groups that part into a subexpression with parentheses. The literal colon ensures that the DISPLAY value matches the format we expect. The part of the string that matches the complete pattern is stored into the match variable. The part that matches the subpattern is stored into host. The whole pattern has been grouped with braces to quote the square brackets. Without braces it would be:

regexp (\[^:\]*): $env(DISPLAY) match host

With advanced regular expressions the nongreedy quantifier *? can replace the complementary set:

regexp (.*?): $env(DISPLAY) match host

This is quite a powerful statement, and it is efficient. If we had only had the string command to work with, we would have needed to resort to the following, which takes roughly twice as long to interpret:

set i [string first : $env(DISPLAY)]
if {$i >= 0} {
    set host [string range $env(DISPLAY) 0 [expr $i-1]]

A Pattern to Match URLs

Example 11-3 demonstrates a pattern with several subpatterns that extract the different parts of a URL. There are lots of subpatterns, and you can determine which match variable is associated with which subpattern by counting the left parenthesis. The pattern will be discussed in more detail after the example:

Example 11-3 A pattern to match URLs.
set url http://www.beedub.com:80/index.html
regexp {([^:]+)://([^:/]+)(:([0-9]+))?(/.*)}$url \
   match protocol x serverport path
=> 1
set match
=> http://www.beedub.com:80/index.html
set protocol
=> http
set server
=> www.beedub.com
set x
=> :80
set port
=> 80
set path
=> /index.html

Let's look at the pattern one piece at a time. The first part looks for the protocol, which is separated by a colon from the rest of the URL. The first part of the pattern is one or more characters that are not a colon, followed by a colon. This matches the http: part of the URL:


Using nongreedy +? quantifier, you could also write that as:


The next part of the pattern looks for the server name, which comes after two slashes. The server name is followed either by a colon and a port number, or by a slash. The pattern uses a complementary set that specifies one or more characters that are not a colon or a slash. This matches the //www.beedub.com part of the URL:


The port number is optional, so a subpattern is delimited with parentheses and followed by a question mark. An additional set of parentheses are added to capture the port number without the leading colon. This matches the :80 part of the URL:


The last part of the pattern is everything else, starting with a slash. This matches the /index.html part of the URL:



Use subpatterns to parse strings.

To make this pattern really useful, we delimit several subpatterns with parentheses:


These parentheses do not change the way the pattern matches. Only the optional port number really needs the parentheses in this example. However, the regexp command gives us access to the strings that match these subpatterns. In one step regexp can test for a valid URL and divide it into the protocol part, the server, the port, and the trailing path.

The parentheses around the port number include the : before the digits. We've used a dummy variable that gets the : and the port number, and another match variable that just gets the port number. By using noncapturing parentheses in advanced regular expressions, we can eliminate the unused match variable. We can also replace both complementary character sets with a nongreedy .+? match. Example 11-4 shows this variation:

Example 11-4 An advanced regular expression to match URLs.
set url http://www.beedub.com:80/book/
regexp {(.+?)://(.+?)(?::([0-9]+))?(/.*)}$url \
   match protocol server port path
=> 1
set match
=> http://www.beedub.com:80/book/
set protocol
=> http
set server
=> www.beedub.com
set port
=> 80
set path
=> /book/

Sample Regular Expressions

The table in this section lists regular expressions as you would use them in Tcl commands. Most are quoted with curly braces to turn off the special meaning of square brackets and dollar signs. Other patterns are grouped with double quotes and use backslash quoting because the patterns include backslash sequences like \n and \t. In Tcl 8.0 and earlier, these must be substituted by Tcl before the regexp command is called. In these cases, the equivalent advanced regular expression is also shown.

Table 11-7. Sample regular expressions.
{^[yY]}Begins with y or Y, as in a Yes answer.
{^(yes|YES|Yes)$}Exactly "yes", "Yes", or "YES".
"^\[^ \t:\]+:"Begins with colon-delimited field that has no spaces or tabs.
{^\S+:}Same as above, using \S for "not space".
"^\[ \t]*$"A string of all spaces or tabs.
{(?n)^\s*$}A blank line using newline sensitive mode.
"(\n|^)\[^\n\]*(\n|$)"A blank line, the hard way.
{^[A-Za-z]+$}Only letters.
{^[[:alpha:]]+$}Only letters, the Unicode way.
{[A-Za-z0-9_]+}Letters, digits, and the underscore.
{\w+}Letters, digits, and the underscore using \w.
{[][${}\\]}The set of Tcl special characters: ] [ $ { } \
"\[^\n\]*\n"Everything up to a newline.
{.*?\n}Everything up to a newline using nongreedy *?
{\.}A period.
{[][$^?+*()|\\]}The set of regular expression special characters: ] [ $ ^ ? + * ( ) | \
<H1>(.*?)</H1>An H1 HTML tag. The subpattern matches the string between the tags.
<!--.*?-->HTML comments.
{[0-9a-hA-H][0-9a-hA-H]}2 hex digits.
{[[:xdigit:]]{2}}2 hex digits, using advanced regular expressions.
{\d{1,3}}1 to 3 digits, using advanced regular expressions.

      Previous section   Next section