The http Package

The standard Tcl library includes an http package that is based on the code I wrote for this chapter. This section documents the package, which has a slightly different interface. The library version uses namespaces and combines the Http_Get, Http_Head, and Http_Post procedures into a single http::geturl procedure. The examples in this chapter are still interesting, but you should look at http.tcl in the Tcl library, which I also wrote. Definitely use the standard http package for your production code.

http::config

The http::config command is used to set the proxy information, time-outs, and the User-Agent and Accept headers that are generated in the HTTP request. You can specify the proxy host and port, or you can specify a Tcl command that is run to determine the proxy. With no arguments, http::config returns the current settings:

http::config
=> -accept */* -proxyfilter httpProxyRequired -proxyhost
{}-proxyport {}-timeout unlimited
-useragent {Tcl http client package 2.0}

If you specify just one option, its value is returned:

http::config -proxyfilter
=> httpProxyRequired

You can set one or more options:

http::config -proxyhost webcache.eng -proxyport 8080

The default proxy filter just returns the -proxyhost and -proxyport values if they are set. You can supply a smarter filter that picks a proxy based on the host in the URL. The proxy filter is called with the hostname and should return a list of two elements, the proxy host and port. If no proxy is required, return an empty list.

The -timeout value limits the time the transaction can take. Its value is unlimited for no timeout, or a milliseconds value. You can specify 500, for example, to have a half-second timeout.

http::geturl

The http::geturl procedure does a GET, POST, or HEAD transaction depending on its arguments. By default, http::geturl blocks until the request completes and it returns a token that represents the transaction. As described below, you use the token to get the results of the transaction. If you supply a -command callback option, then http::geturl returns immediately and invokes callback when the transaction completes. The callback is passed the token that represents the transaction. Table 17-1 lists the options to http::geturl:

Table 17-1. Options to the http::geturl command.

-blocksize num	Block size when copying to a channel.
-channel fileID	The fileID is an open file or socket. The URL data is copied to this channel instead of saving it in memory.
-command callback	Calls callback when the transaction completes. The token from http::geturl is passed to callback.
-handler command	Called from the event handler to read data from the URL.
-headers list	The list specifies a set of headers that are included in the HTTP request. The list alternates between header keys and values.
-progress command	Calls command after each block is copied to a channel. It gets called with three parameters: command token totalsize currentsize
-query codedstring	Issues a POST request with the codedstring form data.
-timeout msec	Aborts the request after msec milliseconds have elapsed.
-validate bool	If bool is true, a HEAD request is made.

For simple applications you can simply block on the transaction:

set token [http::geturl www.beedub.com/index.html]
=> http::1

The leading http:// in the URL is optional. The return value is a token that is also the name of a global array that contains state about the transaction. Names like http::1 are used instead of using the URL as the array name. You can use upvar to convert the return value from http::geturl to an array variable:

upvar #0 $token state

By default, the URL data is saved in state(body). The elements of the state array are described in Table 17-2:

Table 17-2. Elements of the http::geturl state array.

body	The contents of the URL.
currentsize	The current number of bytes transferred.
error	An explanation of why the transaction was aborted.
http	The HTTP reply status.
meta	A list of the keys and values in the reply header.
status	The current status: pending, ok, eof, or reset.
totalsize	The expected size of the returned data.
type	The content type of the returned data.
url	The URL of the request.

A handful of access functions are provided so that you can avoid using the state array directly. These are listed in Table 17-3:

Table 17-3. The http support procedures.

http::data $token	Returns state(body).
http::status $token	Returns state(status).
http::error $token	Returns state(error).
http::code $token	Returns state(http).
http::wait $token	Blocks until the transaction completes.
http::cleanup $token	Unsets the state array named by $token.

You can take advantage of the asynchronous interface by specifying a command that is called when the transaction completes. The callback is passed the token returned from http::geturl so that it can access the transaction state:

http::geturl $url -command [list Url_Display $text $url]
proc Url_Display {text url token} {
   upvar #0 $token state
   # Display the url in text
}

You can have http::geturl copy the URL to a file or socket with the -channel option. This is useful for downloading large files or images. In this case, you can get a progress callback so that you can provide user feedback during the transaction. Example 17-12 shows a simple downloading script:

Example 17-12 Downloading files with http::geturl.

#!/usr/local/tclsh8.0
if {$argc < 2} {
   puts stderr "Usage: $argv0 url file"
   exit 1
}
set url [lindex $argv 0]
set file [lindex $argv 1]
set out [open $file w]
proc progress {token total current} {
   puts -nonewline "."
}
http::config -proxyhost webcache.eng -proxyport 8080
set token [http::geturl $url -progress progress \
   -headers {Pragma no-cache}-channel $out]
close $out
# Print out the return header information
puts ""
upvar #0 $token state
puts $state(http)
foreach {key value}$state(meta) {
   puts "$key: $value"
}
exit 0

http::formatQuery

If you specify form data with the -query option, then http::geturl does a POST transaction. You need to encode the form data for safe transmission. The http::formatQuery procedure takes a list of keys and values and encodes them in x-www-url-encoded format. Pass this result as the query data:

http::formatQuery name "Brent Welch" title "Tcl Programmer"
=> name=Brent+Welch&title=Tcl+Programmer

http::reset

You can cancel an outstanding transaction with http::reset:

http::reset $token

This is done automatically when you setup a -timeout with http::config.

http::cleanup

When you are done with the data returned from http::geturl, use the http::cleanup procedure to unset the state variable used to store the data.