NAMEperlopentut - tutorial on opening things in Perl
DESCRIPTIONPerl has two simple, built-in ways to open files: the shell way for convenience, and the C way for precision. The choice is yours.
Open à la shellPerl's $ myprogram file1 file2 file3 $ myprogram < inputfile $ myprogram > outputfile $ myprogram >> outputfile $ myprogram | otherprogram $ otherprogram | myprogram And here are some more advanced examples: $ otherprogram | myprogram f1 - f2 $ otherprogram 2>&1 | myprogram - $ myprogram <&3 $ myprogram >&4 Programmers accustomed to constructs like those above can take comfort in learning that Perl directly supports these familiar constructs using virtually the same syntax as the shell.
Simple OpensThe For example: open(INFO, "datafile") || die("can't open datafile: $!"); open(INFO, "< datafile") || die("can't open datafile: $!"); open(RESULTS,"> runstats") || die("can't open runstats: $!"); open(LOG, ">> logfile ") || die("can't open logfile: $!"); If you prefer the low-punctuation version, you could write that this way: open INFO, "< datafile" or die "can't open datafile: $!"; open RESULTS,"> runstats" or die "can't open runstats: $!"; open LOG, ">> logfile " or die "can't open logfile: $!"; A few things to notice. First, the leading less-than is optional. If omitted, Perl assumes that you want to open the file for reading. The other important thing to notice is that, just as in the shell, any white space before or after the filename is ignored. This is good, because you wouldn't want these to do different things: open INFO, "<datafile" open INFO, "< datafile" open INFO, "< datafile" Ignoring surround whitespace also helps for when you read a filename in from a different file, and forget to trim it before opening: $filename = <INFO>; # oops, \n still there open(EXTRA, "< $filename") || die "can't open $filename: $!"; This is not a bug, but a feature. Because
Pipe OpensIn C, when you want to open a file using the standard I/O library,
you use the If the leading character is a pipe symbol, open(PRINTER, "| lpr -Plp1") || die "cannot fork: $!"; print PRINTER "stuff\n"; close(PRINTER) || die "can't close lpr: $!"; If the trailing character is a pipe, you start up a new command and open a read-only filehandle leading out of that command. This lets whatever that command writes to its standard output show up on your handle for reading. For example: open(NET, "netstat -i -n |") || die "cannot fork: $!"; while (<NET>) { } # do something with input close(NET) || die "can't close netstat: $!"; What happens if you try to open a pipe to or from a non-existent command?
In most systems, such an If you would like to open a bidirectional pipe, the IPC::Open2 library will handle this for you. Check out Bidirectional Communication with Another Process in the perlipc manpage
The Minus FileAgain following the lead of the standard shell utilities, Perl's
If minus can be used as the default input or default output, what happens
if you open a pipe into or out of minus? What's the default command it
would run? The same script as you're currently running! This is actually
a stealth
Mixing Reads and WritesIt is possible to specify both read and write access. All you do is add a ``+'' symbol in front of the redirection. But as in the shell, using a less-than on a file never creates a new file; it only opens an existing one. On the other hand, using a greater-than always clobbers (truncates to zero length) an existing file, or creates a brand-new one if there isn't an old one. Adding a ``+'' for read-write doesn't affect whether it only works on existing files or always clobbers existing ones. open(WTMP, "+< /usr/adm/wtmp") || die "can't open /usr/adm/wtmp: $!"; open(SCREEN, "+> /tmp/lkscreen") || die "can't open /tmp/lkscreen: $!"; open(LOGFILE, "+>> /tmp/applog" || die "can't open /tmp/applog: $!"; The first one won't create a new file, and the second one will always
clobber an old one. The third one will create a new file if necessary
and not clobber an old one, and it will allow you to read at any point
in the file, but all writes will always go to the end. In short,
the first case is substantially more common than the second and third
cases, which are almost always wrong. (If you know C, the plus in
Perl's In fact, when it comes to updating a file, unless you're working on a binary file as in the WTMP case above, you probably don't want to use this approach for updating. Instead, Perl's -i flag comes to the rescue. The following command takes all the C, C++, or yacc source or header files and changes all their foo's to bar's, leaving the old version in the original file name with a ``.orig'' tacked on the end: $ perl -i.orig -pe 's/\bfoo\b/bar/g' *.[Cchy] This is a short cut for some renaming games that are really the best way to update textfiles. See the second question in the perlfaq5 manpage for more details.
FiltersOne of the most common uses for $ myprogram file1 file2 file3 Can have all its files opened and processed one at a time using a construct no more complex than: while (<>) { # do something with $_ } If @ARGV is empty when the loop first begins, Perl pretends you've opened
up minus, that is, the standard input. In fact, $ARGV, the currently
open file during You are welcome to pre-process your @ARGV before starting the loop to make sure it's to your liking. One reason to do this might be to remove command options beginning with a minus. While you can always roll the simple ones by hand, the Getopts modules are good for this. use Getopt::Std; # -v, -D, -o ARG, sets $opt_v, $opt_D, $opt_o getopts("vDo:"); # -v, -D, -o ARG, sets $args{v}, $args{D}, $args{o} getopts("vDo:", \%args); Or the standard Getopt::Long module to permit named arguments: use Getopt::Long; GetOptions( "verbose" => \$verbose, # --verbose "Debug" => \$debug, # --Debug "output=s" => \$output ); # --output=somestring or --output somestring Another reason for preprocessing arguments is to make an empty argument list default to all files: @ARGV = glob("*") unless @ARGV; You could even filter out all but plain, text files. This is a bit silent, of course, and you might prefer to mention them on the way. @ARGV = grep { -f && -T } @ARGV; If you're using the -n or -p command-line options, you
should put changes to @ARGV in a Remember that a normal $pwdinfo = `domainname` =~ /^(\(none\))?$/ ? '< /etc/passwd' : 'ypcat passwd |'; open(PWD, $pwdinfo) or die "can't open $pwdinfo: $!"; This sort of thing also comes into play in filter processing. Because
$ myprogram f1 "cmd1|" - f2 "cmd2|" f3 < tmpfile That program will read from the file f1, the process cmd1, standard input (tmpfile in this case), the f2 file, the cmd2 command, and finally the f3 file. Yes, this also means that if you have a file named ``-'' (and so on) in
your directory, that they won't be processed as literal files by One of the more interesting applications is to change files of a certain name into pipes. For example, to autoprocess gzipped or compressed files by decompressing them with gzip: @ARGV = map { /^\.(gz|Z)$/ ? "gzip -dc $_ |" : $_ } @ARGV; Or, if you have the GET program installed from LWP, you can fetch URLs before processing them: @ARGV = map { m#^\w+://# ? "GET $_ |" : $_ } @ARGV; It's not for nothing that this is called magic
Open à la CIf you want the convenience of the shell, then Perl's
sysopen HANDLE, PATH, FLAGS, [MASK] The HANDLE argument is a filehandle just as with Although the traditional values of read-only, write-only, and read-write are 0, 1, and 2 respectively, this is known not to hold true on some systems. Instead, it's best to load in the appropriate constants first from the Fcntl module, which supplies the following standard flags: O_RDONLY Read only O_WRONLY Write only O_RDWR Read and write O_CREAT Create the file if it doesn't exist O_EXCL Fail if the file already exists O_APPEND Append to the file O_TRUNC Truncate the file O_NONBLOCK Non-blocking access Less common flags that are sometimes available on some operating
systems include Here's how to use To open a file for reading: open(FH, "< $path"); sysopen(FH, $path, O_RDONLY); To open a file for writing, creating a new file if needed or else truncating an old file: open(FH, "> $path"); sysopen(FH, $path, O_WRONLY | O_TRUNC | O_CREAT); To open a file for appending, creating one if necessary: open(FH, ">> $path"); sysopen(FH, $path, O_WRONLY | O_APPEND | O_CREAT); To open a file for update, where the file must already exist: open(FH, "+< $path"); sysopen(FH, $path, O_RDWR); And here are things you can do with To open a file for writing, creating a new file which must not previously exist: sysopen(FH, $path, O_WRONLY | O_EXCL | O_CREAT); To open a file for appending, where that file must already exist: sysopen(FH, $path, O_WRONLY | O_APPEND); To open a file for update, creating a new file if necessary: sysopen(FH, $path, O_RDWR | O_CREAT); To open a file for update, where that file must not already exist: sysopen(FH, $path, O_RDWR | O_EXCL | O_CREAT); To open a file without blocking, creating one if necessary: sysopen(FH, $path, O_WRONLY | O_NONBLOCK | O_CREAT);
Permissions à la modeIf you omit the MASK argument to Why so permissive? Well, it isn't really. The MASK will be modified
by your process's current For example, if your You should seldom use the MASK argument to
Obscure Open Tricks
Re-Opening Files (dups)Sometimes you already have a filehandle open, and want to make another
handle that's a duplicate of the first one. In the shell, we place an
ampersand in front of a file descriptor number when doing redirections.
For example, open(SAVEOUT, ">&SAVEERR") || die "couldn't dup SAVEERR: $!"; open(MHCONTEXT, "<&4") || die "couldn't dup fd4: $!"; That means that if a function is expecting a filename, but you don't want to give it a filename because you already have the file open, you can just pass the filehandle with a leading ampersand. It's best to use a fully qualified handle though, just in case the function happens to be in a different package: somefunction("&main::LOGFILE"); This way if If you have one of those tricky, newfangled I/O objects that the C++
folks are raving about, then this doesn't work because those aren't a
proper filehandle in the native Perl sense. You'll have to use use IO::Socket; $handle = IO::Socket::INET->new("www.perl.com:80"); $fd = $handle->fileno; somefunction("&$fd"); # not an indirect function call It can be easier (and certainly will be faster) just to use real filehandles though: use IO::Socket; local *REMOTE = IO::Socket::INET->new("www.perl.com:80"); die "can't connect" unless defined(fileno(REMOTE)); somefunction("&main::REMOTE"); If the filehandle or descriptor number is preceded not just with a simple
``&'' but rather with a ``&='' combination, then Perl will not create a
completely new descriptor opened to the same place using the $fd = $ENV{"MHCONTEXTFD"}; open(MHCONTEXT, "<&=$fd") or die "couldn't fdopen $fd: $!"; If you're using magic
Dispelling the DweomerPerl is more of a DWIMmer language than something like Java--where DWIM is an acronym for ``do what I mean''. But this principle sometimes leads to more hidden magic than one knows what to do with. In this way, Perl is also filled with dweomer, an obscure word meaning an enchantment. Sometimes, Perl's DWIMmer is just too much like dweomer for comfort. If magic $file =~ s#^(\s)#./$1#; open(FH, "< $file\0") || die "can't open $file: $!"; This assumes, of course, that your system considers dot the current
working directory, slash the directory separator, and disallows ASCII
NULs within a valid filename. Most systems follow these conventions,
including all POSIX systems as well as proprietary Microsoft systems.
The only vaguely popular system that doesn't work this way is the
proprietary Macintosh system, which uses a colon where the rest of us
use a slash. Maybe If you want to use # "Sam sat on the ground and put his head in his hands. # 'I wish I had never come here, and I don't want to see # no more magic,' he said, and fell silent." for (@ARGV) { s#^([^./])#./$1#; $_ .= "\0"; } while (<>) { # now process $_ } But be warned that users will not appreciate being unable to use ``-'' to mean standard input, per the standard convention.
Paths as OpensYou've probably noticed how Perl's Some warning at scriptname line 29, <FH> line 7. That's because you opened a filehandle FH, and had read in seven records from it. But what was the name of the file, not the handle? If you aren't running with open($path, "< $path") || die "can't open $path: $!"; while (<$path>) { # whatever } Since you're using the pathname of the file as its handle, you'll get warnings more like Some warning at scriptname line 29, </etc/motd> line 7.
Single Argument OpenRemember how we said that Perl's open took two arguments? That was a
passive prevarication. You see, it can also take just one argument.
If and only if the variable is a global variable, not a lexical, you
can pass $FILE = "/etc/motd"; open FILE or die "can't open $FILE: $!"; while (<FILE>) { # whatever } Why is this here? Someone has to cater to the hysterical porpoises. It's something that's been in Perl since the very beginning, if not before.
Playing with STDIN and STDOUTOne clever move with STDOUT is to explicitly close it when you're done with the program. END { close(STDOUT) || die "can't close stdout: $!" } If you don't do this, and your program fills up the disk partition due to a command line redirection, it won't report the error exit with a failure status. You don't have to accept the STDIN and STDOUT you were given. You are welcome to reopen them if you'd like. open(STDIN, "< datafile") || die "can't open datafile: $!"; open(STDOUT, "> output") || die "can't open output: $!"; And then these can be read directly or passed on to subprocesses. This makes it look as though the program were initially invoked with those redirections from the command line. It's probably more interesting to connect these to pipes. For example: $pager = $ENV{PAGER} || "(less || more)"; open(STDOUT, "| $pager") || die "can't fork a pager: $!"; This makes it appear as though your program were called with its stdout already piped into your pager. You can also use this kind of thing in conjunction with an implicit fork to yourself. You might do this if you would rather handle the post processing in your own program, just in a different process: head(100); while (<>) { print; } sub head { my $lines = shift || 20; return unless $pid = open(STDOUT, "|-"); die "cannot fork: $!" unless defined $pid; while (<STDIN>) { print; last if --$lines < 0; } exit; } This technique can be applied to repeatedly push as many filters on your output stream as you wish.
Other I/O IssuesThese topics aren't really arguments related to
Opening Non-File FilesWhen is a file not a file? Well, you could say when it exists but isn't a plain file. We'll check whether it's a symbolic link first, just in case. if (-l $file || ! -f _) { print "$file is not a plain file\n"; } What other kinds of files are there than, well, files? Directories,
symbolic links, named pipes, Unix-domain sockets, and block and character
devices. Those are all files, too--just not plain files. This isn't
the same issue as being a text file. Not all text files are plain files.
Not all plain files are textfiles. That's why there are separate To open a directory, you should use the opendir(DIR, $dirname) or die "can't opendir $dirname: $!"; while (defined($file = readdir(DIR))) { # do something with "$dirname/$file" } closedir(DIR); If you want to process directories recursively, it's better to use the File::Find module. For example, this prints out all files recursively, add adds a slash to their names if the file is a directory. @ARGV = qw(.) unless @ARGV; use File::Find; find sub { print $File::Find::name, -d && '/', "\n" }, @ARGV; This finds all bogus symbolic links beneath a particular directory: find sub { print "$File::Find::name\n" if -l && !-e }, $dir; As you see, with symbolic links, you can just pretend that it is
what it points to. Or, if you want to know what it points to, then
if (-l $file) { if (defined($whither = readlink($file))) { print "$file points to $whither\n"; } else { print "$file points nowhere: $!\n"; } } Named pipes are a different matter. You pretend they're regular files, but their opens will normally block until there is both a reader and a writer. You can read more about them in Named Pipes in the perlipc manpage. Unix-domain sockets are rather different beasts as well; they're described in Unix-Domain TCP Clients and Servers in the perlipc manpage. When it comes to opening devices, it can be easy and it can tricky. We'll assume that if you're opening up a block device, you know what you're doing. The character devices are more interesting. These are typically used for modems, mice, and some kinds of printers. This is described in How do I read and write the serial port? in the perlfaq8 manpage It's often enough to open them carefully: sysopen(TTYIN, "/dev/ttyS1", O_RDWR | O_NDELAY | O_NOCTTY) # (O_NOCTTY no longer needed on POSIX systems) or die "can't open /dev/ttyS1: $!"; open(TTYOUT, "+>&TTYIN") or die "can't dup TTYIN: $!"; $ofh = select(TTYOUT); $| = 1; select($ofh); print TTYOUT "+++at\015"; $answer = <TTYIN>; With descriptors that you haven't opened using use Fcntl; fcntl(Connection, F_SETFL, O_NONBLOCK) or die "can't set non blocking: $!"; Rather than losing yourself in a morass of twisting, turning What else can you open? To open a connection using sockets, you won't use one of Perl's two open functions. See Sockets: Client/Server Communication in the perlipc manpage for that. Here's an example. Once you have it, you can use FH as a bidirectional filehandle. use IO::Socket; local *FH = IO::Socket::INET->new("www.perl.com:80"); For opening up a URL, the LWP modules from CPAN are just what the doctor ordered. There's no filehandle interface, but it's still easy to get the contents of a document: use LWP::Simple; $doc = get('http://www.linpro.no/lwp/');
Binary FilesOn certain legacy systems with what could charitably be called terminally convoluted (some would say broken) I/O models, a file isn't a file--at least, not with respect to the C standard I/O library. On these old systems whose libraries (but not kernels) distinguish between text and binary streams, to get files to behave properly you'll have to bend over backwards to avoid nasty problems. On such infelicitous systems, sockets and pipes are already opened in binary mode, and there is currently no way to turn that off. With files, you have more options. Another option is to use the binmode(STDIN); binmode(STDOUT); while (<STDIN>) { print } Passing sysopen(BINDAT, "records.data", O_RDWR | O_BINARY) || die "can't open records.data: $!"; Now you can use On systems with exotic I/O systems, it turns out that, astonishingly
enough, even unbuffered I/O using while (sysread(WHENCE, $buf, 1024)) { syswrite(WHITHER, $buf, length($buf)); } Depending on the vicissitudes of your runtime system, even these calls
may need
File LockingIn a multitasking environment, you may need to be careful not to collide with other processes who want to do I/O on the same files as others are working on. You'll often need shared or exclusive locks on files for reading and writing respectively. You might just pretend that only exclusive locks exist. Never use the existence of a file Perl's most portable locking interface is via the File locking does not lock out another process that would like to do I/O. A file lock only locks out others trying to get a lock, not processes trying to do I/O. Because locks are advisory, if one process uses locking and another doesn't, all bets are off. By default, the Here's how to get a blocking shared lock on a file, typically used for reading: use 5.004; use Fcntl qw(:DEFAULT :flock); open(FH, "< filename") or die "can't open filename: $!"; flock(FH, LOCK_SH) or die "can't lock filename: $!"; # now read from FH You can get a non-blocking lock by using flock(FH, LOCK_SH | LOCK_NB) or die "can't lock filename: $!"; This can be useful for producing more user-friendly behaviour by warning if you're going to be blocking: use 5.004; use Fcntl qw(:DEFAULT :flock); open(FH, "< filename") or die "can't open filename: $!"; unless (flock(FH, LOCK_SH | LOCK_NB)) { $| = 1; print "Waiting for lock..."; flock(FH, LOCK_SH) or die "can't lock filename: $!"; print "got it.\n" } # now read from FH To get an exclusive lock, typically used for writing, you have to be
careful. We use 5.004; use Fcntl qw(:DEFAULT :flock); sysopen(FH, "filename", O_WRONLY | O_CREAT) or die "can't open filename: $!"; flock(FH, LOCK_EX) or die "can't lock filename: $!"; truncate(FH, 0) or die "can't truncate filename: $!"; # now write to FH Finally, due to the uncounted millions who cannot be dissuaded from wasting cycles on useless vanity devices called hit counters, here's how to increment a number in a file safely: use Fcntl qw(:DEFAULT :flock); sysopen(FH, "numfile", O_RDWR | O_CREAT) or die "can't open numfile: $!"; # autoflush FH $ofh = select(FH); $| = 1; select ($ofh); flock(FH, LOCK_EX) or die "can't write-lock numfile: $!"; $num = <FH> || 0; seek(FH, 0, 0) or die "can't rewind numfile : $!"; print FH $num+1, "\n" or die "can't write numfile: $!"; truncate(FH, tell(FH)) or die "can't truncate numfile: $!"; close(FH) or die "can't close numfile: $!";
SEE ALSOThe
AUTHOR and COPYRIGHTCopyright 1998 Tom Christiansen. When included as part of the Standard Version of Perl, or as part of its complete documentation whether printed or otherwise, this work may be distributed only under the terms of Perl's Artistic License. Any distribution of this file or derivatives thereof outside of that package require that special arrangements be made with copyright holder. Irrespective of its distribution, all code examples in these files are hereby placed into the public domain. You are permitted and encouraged to use this code in your own programs for fun or for profit as you see fit. A simple comment in the code giving credit would be courteous but is not required.
HISTORYFirst release: Sat Jan 9 08:09:11 MST 1999
|