Unix FAQ (for OS X)
This document is intended as a way of answering some of the questions that are frequently asked (especially by beginners) about Unix on OS X. The answers have been updated for Leopard (OS X version 10.5). Feel free to suggest additions or corrections via the email address given at the bottom of the page.This FAQ was originally hosted on the MacOSXHints forums site and was sparked by questions asked there and suggestions by forum members.
- What are the most basic things I need to know?
- Why do I get "command not found" when I try to run a command?
- How do I set up my shell execution PATH?
- How can I tell what shell I am running?
- What is the difference between a "login" shell and a "non-login" shell, and why do I care?
- What startup files are read by the shell? (shell configuration)
- What's the deal with permissions? How do I change them?
- How do I gain "root" privileges? Do I need to enable the "root" user?
- How do I navigate to different folders?
- What integration is there between Finder and Terminal?
- How do I edit text files (e.g. configuration files)?
- What is an end-of-line character and why should I care?
- How do I create new files and folders?
- How do I delete files and folders?
- What are the most commonly used commands?
- How does wildcarding work? (or why do I see stars?)
- How do I make a command iterate over a bunch of files?
- How can I redirect the output from a command into a file?
- How do I run a script?
- How can I make a script double-clickable? (How do I run a script from Finder?)
- How can I make my script interoperate with AppleScript?
- How can I compile open source software?
- How can I remember all these complicated commands? (Answer: shell aliases)
- What is a "symbolic link"? How is it different from a "Finder alias"?
- What is a "resource fork"? Are resource forks supported by Unix commands?
- What is an "extended attribute"? Are xattrs supported by Unix commands?
- How do I set an "environment variable"?
- Terminal is behaving strangely - how can I fix it?
- How do I do XYZ?
- Where can I learn more?
What are the most basic things I need to know?
The Unix command-line provides an alternative to the GUI provided by OS X.Most Macintosh users will not need to use the command-line but it provides a more efficient way of accomplishing some tasks (especially repetitive ones). The command-line is also an essential troubleshooting tool in some situations.
Terminal
You get access to the command-line by launching the application called
"Terminal" which is in the "/Applications/Utilities" folder.
When you open a new Terminal window, it will show some text like
"Welcome to Darwin!" and then it will show a prompt on the next line.
Command-line Prompt
The "prompt" is some text that is displayed in the Terminal window whenever
it is ready for you to type a command. The usual prompt shows the name of your
Mac, the name of the current folder, and your username, and finishes with a
dollar sign. For example if your Mac is named "red" and your username is "joe",
and you are in your home folder (which is designated as "~"),
the usual prompt would look like:
red: ~ joe$
Executing commands
To execute a Unix command, you first need to know what command you want
and the syntax of this command. The following sections of this FAQ explain
how to use the most commonly used commands.
You type in the command after the prompt in a Terminal window,
then press Return to execute the command.
Any textual output from the command will be displayed (in the Terminal window)
immediately following the line you typed. If something went wrong with the
command, it will display a terse error message indicating the nature of the
problem.
After the command has finished executing, the prompt will be displayed on
a new line, indicating that it is ready for the next command.
Most commands finish quickly, but some are designed to keep running until you stop them manually with Control-C (i.e. by holding down the Control ("ctrl") key and pressing the C key).
Success or failure?
Not all commands have output - many of them just do what they are supposed
to do without giving any feedback at all. So if you execute a command
and it doesn't produce any output (i.e. you are just returned to the prompt),
it means that the command succeeded in doing whatever it was supposed
to do.
But if you get an error message, that means that the command didn't work
and you will need to try to figure out why.
Be careful
It is important to note that when you execute Unix-level commands,
there is no "undo" or other safety net. You need to understand what
each command does and be sure that that is what you want to do before
you press Return.
When entering commands given by others, it is very important to check that you have got it exactly right - each character and space is important. If someone supplies a command for you to run, it is often best to copy and paste the command rather than typing it in.
Filenames, Folders, Directories, and Filepaths
When you are working in the Terminal, there is the concept of being "in"
a certain location in the filesystem. People speak of the "current folder"
or "current directory". (The word "directory" in Unix is a synonym for
"folder" but usually I will use the word "folder" in this FAQ.)
If you supply the name of a file as part of a command, it is assumed to be in the current folder. If you want to operate on a file that is some other folder, you can either move to that folder (see "How do I navigate to different folders?") or specify the path to that file by prefacing the name of the file with the names of the folders and sub-folders separated by slashes.
The folder at the very top of the drive that contains everything on that drive is called "/". Your home folder is a sub-folder of the "/Users" folder (i.e. the folder "Users" which is in the folder "/"). If your username is "joe", then the full path to your home folder is "/Users/joe". The symbol "~" (above the back-quote on the key near the top left of standard US keyboards) can be used as an alternative way of specifying your home folder.
Thus a file "foo" that is in the "Documents" sub-folder of your home folder can be referred to as "/Users/joe/Documents/foo" or as "~/Documents/foo". If you are "in" your home folder, then you can refer to this file as "Documents/foo" - the lack of an initial slash indicates that this is a "relative path" (i.e. one relative to the current folder).
Spaces and command-line arguments
The almost universal convention for Unix commands is that command-line
arguments are separated by spaces. For example, in commands that deal
with files, you specify the files to be operated on by listing their
names with spaces between them.
This obviously causes difficulties if the name of a file contains
a space (e.g. if the file is named "diary 2005").
If the name of a file (or folder) contains a space, you need to
enclose the name in quotes when you use it with a command.
For example, to read the above file using the command 'more',
you would do this:
Example commands in this document
In this FAQ, example commands will usually be shown in "code" boxes
to set them off from the rest of the text (as shown just above).
Usually I will only show the command that you would type but
in examples where I show the commands and the output from
those commands, I preface the commands with "%" so it is more clear
which lines are the commands and which lines are the output.
The "%" represents the prompt - it isn't part of the command that you type.
Why do I get "command not found" when I try to run a command?
When you type a command at the shell prompt (in a Terminal window), the shell looks in certain predefined folders/directories for that command. The places it looks are those that are defined by your shell execution PATH variable.You can see the current contents of your PATH variable by running the following command:
As an alternative, you can always execute commands by specifying
the full path to the executable. For example, the executable
for 'ls' is in the folder "/bin", so you can run the "ls" command
by typing:
But suppose that you put a program named "foo" in a folder "Tools" under your home folder (i.e. in the folder ~/Tools) and you haven't yet got around to adding the Tools folder to your PATH. You could execute the "foo" program by typing its full path:
Other execution contexts
Note that when you run commands in other execution contexts
(outside of the Terminal - e.g. from AppleScript's 'do shell script'
or from 'cron' or 'launchd' jobs), the PATH variable may be different from
what you have set up for the Terminal. In general you should use full paths
for all commands and files referred to in scripts - then the PATH variable
will not affect them.
The Perl script
'showLaunchdEnviron' will show you the environment
(variables, etc) that will be present for 'launchd' scripts.
How do I set up my shell execution PATH?
The answer depends on which shell you are using.The usual shells used in OS X are "bash" and "tcsh".
See "How can I tell what shell I am running?" if you don't know which shell you are using.
(And if you don't know why you should care about the PATH variable, see the previous section of this FAQ.)
It is possible to change the PATH for the current session via interactive commands, but what you almost always want is to set it up for all future sessions. To do this, you need to add commands to one of the shell startup files Note that these startup files do not exist by default - see the section on startup files.
Setting PATH in the bash shell
To set up your PATH in bash, you edit the file ~/.profile
(using any text editor that can save in plain text - e.g. 'pico')
and add a line like the following:
/bin
/sbin
/usr/bin
/usr/sbin
Note: a "directory" is the usual Unix term for what is called a "folder" in OS X. From here onwards, I will usually just use the term "folder".
This is the default PATH that is already set up for you in the file /etc/profile (which supplies system-wide defaults and which you should not usually be editing). If you want other folders to be in your PATH, you would add them to the colon (:) separated list in the above line.
Note: If you have Fink
(http://fink.sourceforge.net/)
installed, then your ~/.profile will already have a line in it
like the following:
Setting PATH in the tcsh shell
To set up your PATH in tcsh, you edit the file ~/.login
(using any text editor that can save in plain text - e.g. 'pico')
and add a line like the following:
/bin
/sbin
/usr/bin
/usr/sbin
This is the default PATH that is already set up for you in the file /etc/csh.login (which supplies system-wide defaults and which you should not usually be editing). If you want other folders to be in your PATH, you would add them to the colon (:) separated list in the above line.
Putting the current directory/folder in your PATH
Note that the current directory/folder (usually designated as ".")
is not in your PATH by default. That means that an executable that
is in the current directory will not be found.
(That is why you often see people executing commands like './foo'
as discussed above.)
To add the current directory to your PATH you would add a dot (.)
to the colon-separated list in the above line that sets the
PATH variable.
Note, however, that it is usually considered a small security risk
to do this since a malicious person could put an executable
in some directory and name it the same as some standard
executable (e.g. 'ls') and then you would end up executing
their version of 'ls' instead of the standard one.
How can I tell what shell I am running?
In Panther (OS X 10.3) and later, the default shell for new accounts is bash (/bin/bash), while in Jaguar (OS X 10.2) and earlier, it was tcsh (/bin/tcsh).If you carried a user account forward from an earlier version of OS X, your default shell is likely the same as it was when the account was created.
The default shell referred to above is the shell that will run
when you start a new Terminal window.
You can check what shell has been assigned to your user account
by running the following command:
or by running the following command:
And, of course, you can always start sub-shells by typing the name
of the shell you want to start. E.g. if you type the command:
So how can you determine what type of shell you are currently running?
One way is with the command:
Note that the tcsh shell sets the variable $shell to contain "tcsh" but $shell isn't set to anything by the bash shell. And the variable $SHELL (uppercase) is set to the default shell's path (e.g. "/bin/bash") so these aren't reliable methods of determining what shell you are running.
What is the difference between a "login" shell and a "non-login" shell?
When you start a new Terminal window, the shell that is started is a "login shell". (It is called this because historically, this the shell that is started when a user logged in to a text-only Unix machine.)When you start a sub-shell (by typing a shell's name at the command-prompt), you get a "non-login shell".
When you open a new 'xterm' window (under Apple's X11), the shell that you get is a "non-login shell". This is a key difference between Terminal and xterm and (as explained below) affects what startup files are read.
Another difference between shells is "interactive" versus "non-interactive". An "interactive" shell is the type that you interact with in a Terminal window. But when you run a shell script, a new shell is started for the purpose of that script, and that new shell is a "non-interactive" shell (since the user is not interacting with it via the command-prompt). A distinction is made between these two types of shells since some configuration settings are undesirable for non-interactive shells.
See below for the differences in startup sequences between "login" and "non-login" shells.
For more details on these terms, read the 'man' page for the shell you are using. (e.g. 'man bash' or 'man tcsh')
What startup files are read by the shell? (shell configuration)
You can customize the behaviour of the shells you use by editing the "startup files" that are read when a shell starts up. These files are often called "dot files" since their names usually start with a dot (.)Note that the per-user "dot files" are looked for in your home folder (designated by ~) and that they don't usually exist by default - you have to create them by using a text editor (e.g. 'pico'). Note also that since their names start with a dot, these files are not shown when you do 'ls' - you need to use the "-a" option: 'ls -a' to see all files.
(To list only those files whose names start with a dot, use the command 'ls -ld .*')
The following settings are often put in shell startup files:
- shell execution PATH (determines where the shell will look
for executables)
- MANPATH (determines where the 'man' program will look for man pages)
- shell aliases and functions (used to save you typing)
- shell prompts
- other environment variables
Which startup files are read depends on which shell you are using (e.g. bash or tcsh) and whether it is a "login shell" or a "non-login shell". Note however, that sub-shells inherit the environment variables of the parent shell. The following is a summary of the usual startup files read by "interactive" shells. See the relevant 'man' page for full details (e.g. 'man bash' or 'man tcsh').
Note: for debugging purposes, I recommend inserting an 'echo' command
in each of the below named startup files so you can see when they get
executed and in what order.
E.g. "echo now executing ~/.profile"
Note also that it isn't recommended to edit the system supplied startup files
(the ones in the "/etc" folder) since these will affect all users
and thus you won't be able to use the usual troubleshooting technique
of logging in as a different user if you make a mistake in editing one
of those files.
Bash Startup Files
When a "login shell" starts up, it reads the file "/etc/profile"
and then "~/.bash_profile" or "~/.bash_login" or "~/.profile"
(whichever one exists - it only reads one of these, checking for them
in the order mentioned).
When a "non-login shell" starts up, it reads the file "/etc/bashrc" and then the file "~/.bashrc".
Note that when bash is invoked with the name "sh", it tries to mimic the startup sequence of the Bourne shell ("sh"). In particular, a non-login shell invoked as "sh" does not read any dot files by default. See the bash man page for details.
Tcsh Startup Files
When a "login shell" starts up, it reads the files "/etc/csh.cshrc"
and "/etc/csh.login" (in that order) and then "~/.tcshrc" or "~/.cshrc"
(whichever one exists - it only reads one of these, checking for them
in the order mentioned) and then ~/.login.
When a "non-login shell" starts up, it reads the file "/etc/csh.cshrc" and then "~/.tcshrc" or "~/.cshrc" (whichever one exists - it only reads one of these).
An Illustration
The following sequence shows the startup of several shells
(non-login and login) as an illustration of what has been described above.
(The 'head' command shows the first few lines of a file
- see 'man head' for details.)
Startup files for shell scripts
There are slightly different rules about which startup files get executed
for non-interactive shells (as used in shell scripts)
- see the bash man page for details.
But if you are writing a shell script, you really shouldn't rely
on any startup files at all
- you should make the shell script set up whatever environment
it needs on its own.
In particular, you should use full paths to all executables and other
files referred to by the shell script, so the PATH is not relevant.
What's the deal with permissions? How do I change them?
Each file or folder is considered to be owned by one user and one group (usually the main group to which that user belongs). There are a set of permissions associated with each file or folder that determines what actions on that file or folder are allowed for a particular user.
You can see the ownership and permissions for a file or folder
by using the 'ls' command with the "-l" option
- i.e. with the command 'ls -l'.
(Since it is sometimes difficult to distinguish lowercase L's
from I's and 1's, I note that phonetically this command is
"ell ess minus ell".)
For example, if "foo" is a file in the current folder:
The permissions are shown by the sequence of ten characters at the start of that line: "-rw-r--r--"
The first character is "-" for a file and "d" for a folder (d = directory, which is what folders are usually called in Unix).
The next three characters ("rw-") indicate the permissions for the owner of the file ("fred").
The next three characters ("r--") indicate the permissions for a user who is not the owner of the file but who is in the group ("staff") that owns the file.
The last three characters ("r--") indicate the permissions for a user who is not the owner of the file and who is not in the group that owns the file - i.e. for anyone else (often called "other" or "world").
These three characters indicate the "read", "write" and "execute" permissions
that apply to the relevant user.
An "r" indicates that the file is
readable, while lack of read permission is indicated by a "-".
A "w" indicates that the file is writable, while lack of write permission is indicated by a "-". It is necessary to have write permission on a folder in order to be able to add files to that folder or to remove files from that folder. Note that you can remove a read-only file from a folder if you have write permission for that folder.
An "x" indicates that the file is executable, while lack of execute permission
is indicated by a "-". Only program files (e.g. applications and scripts)
are normally executable - document files are not. But for folders,
execute permission is required in order to be able to traverse the folder
or to be able to 'cd' to the folder - this is usually referred to as "search"
permission.
This means that you almost always want folders to have execute permission
if they have read permission.
Note in particular that if you want someone to have access to a file in a particular folder, that user must have "search" permission (the "x") on all of the folders along the path from the top of the drive down to the folder where the file is. And then the file itself must have "read" permission for that user. And if you want someone to be able to see what files are in a particular folder, that folder must have "read" permission for that user.
Note that to see the permissions of a folder, you need to use the "-d" option
to 'ls' as well as the "-l" option. For example:
Changing ownership
To change the ownership of a file or folder, you use the command 'chown'.
This command needs to be executed with 'root' privileges - so you need to
preface it with 'sudo' and be prepared to supply your password.
(Only admin users can use 'sudo'.) For example, to change the file "foo"
to be owned by the user "mary" and the group "dept42", you would use the
command:
Changing permissions
To change the permissions on a file or folder, you use the command 'chmod'.
There are many different ways of using this command (for details,
see 'man chmod') but the easiest way is to use the "symbolic modes"
where you specify which section of permissions (owner, group, other)
you want to change, then one of "+", "-" or "=" (for adding, removing,
or setting permissions), and then the type of permissions
you want to modify.
You use 'u' to indicate that the change applies to the owner section
of the permissions, 'g' to indicate that it applies to the group section,
and 'o' to indicate that it applies to the "other" section.
For example, to change the permissions on the file "foo" so that users in
the group that owns the file will have write permission:
Numerical values for permissions
It is also possible to set the permissions using numerical values.
This is sometimes more concise than using the above "symbolic modes".
Read permission is assigned a numerical value of 4
Write permission is assigned a numerical value of 2
Execute permission is assigned a numerical value of 1
The total of these values is the numerical permission value.
For example, a permission value of 4 indicates only read permission,
a permission value of 6 indicates both read and write permission,
and a permission value of 7 indicates read, write, and execute permission.
A permission value of 5 indicates read and execute permission
(but no write permission).
The above applies to each section of the permissions (owner, group,
and "other"). Thus a permissions value of 644 indicates that the owner
has both read and write permissions, while group and "other" have only
read permission. This is a typical value for files.
A permissions value of 755 indicates that the owner has read, write and
execute permissions, while group and "other" have read and execute permissions
but no write permission. This is a typical value for folders.
The following two commands are equivalent. Doing either of them would
change the permission of the file "foo" to be readable, writable,
and executable by the owner, readable and executable by the group,
but only readable by "other":
Default permissions determined by "umask"
A newly created file or folder has permissions determined by
the "umask" value of the user who created the file or folder.
You can see the current value of your "umask" by executing the
command of the same name:
This value means that nothing (0) will be removed from the permissions for the owner - thus newly created files will be readable and writable by the owner, but that write permission (2) will be removed from the permissions for the group and other - thus newly created files will be readable (but not writable) by group and other. You can think of this as subtraction of the umask values from the numerical starting permissions "666" for files and "777" for folders - hence with the default umask value "022", newly created files have permissions "644", while newly created folders have permissions "755".
If you want a different "umask" value, you can use the 'umask' command in your shell startup file. But note that GUI apps (e.g. Finder) will continue to use the default "umask" value - to change that, see this article on the MacOSXHints site.
Special cases
In addition to the standard permissions described above, there are several
special "flags" that can be set on a file or folder. You can see the state
of these special flags by using the command 'ls -lO' on the file
(or 'ls -lOd' on the folder)
In Tiger (OS X 10.4), these commands are:
'ls -lo' and 'ls -lod'.
You can change the flags with the 'chflags' command.
See
this article
on the MacOSXHints site for details.
Access control lists
Tiger (OS X 10.4) introduced "access control lists" (ACL's)
- these provide an additional mechanism that allows fine-grained control
over which users can do certain operations with files & folders.
They are used for some system and user folders in Leopard (OS X 10.5).
See
this ArsTechnica article and
this MacOSXHints article for more details on ACLs but I'll just note
that to see the ACL associated with a file, you use the command 'ls -le'.
Note also the 3rd-party utility
"Sandbox" that deals with ACLs.
For full details on ownership and permissions, read the man pages:
'man ls', 'man 'chown', 'man chmod', 'man chflags'
How do I gain "root" privileges? Do I need to enable the "root" user?
Sometimes you need to run a command with 'root' privileges. The 'root' user is a user account that is all powerful - it has complete control over everything on the machine and thus it is sometimes called the "superuser".
Logging in as the 'root' user is disabled by default in OS X
(and it is best to leave it that way - see below) but OS X supplies
the 'sudo' command that allows you to gain 'root' privileges temporarily
if your account is an "admin" account.
(The first account created on an OS X system is by default an "admin" account.
You can give administrator status to other user accounts via the Accounts
preference panel - doing this makes that user be a member of the
'admin' group.)
To run a command with 'root' privileges, you preface the command with 'sudo'. You will be asked to supply your password (the password you normally supply when you log into your account). Note that the characters of your password are not displayed on the screen as you type them - this is a security measure. If this is the first time you have used the 'sudo' command, you will also get a small warning lecture about being careful and using it responsibly.
For example, suppose you want to remove a file named "foo" from a folder
whose permissions would normally prevent you from deleting files.
If you run the following command:
Note that you need to be extra careful when using the 'sudo' command since when you run commands as 'root', the system will hardly ever say "No" even if what you are doing will destroy the system.
Subsequent uses of 'sudo' will not ask for your password if it has been less than 5 minutes since the last use of 'sudo'. This avoids the annoyance of being continually asked for your password when you run a sequence of commands that need root privileges.
If you are going to need root privileges for an extended period of time, you can start a new shell as 'root' via the command 'sudo -s'. If you do this, make sure you remember that this shell is running as root and exit from it when you are finished. It might be a good idea to change the Terminal colours so that you have a visual reminder of the special status of this shell.
There is no need to enable the 'root' user for login. Everything that you
might need 'root' for can be done via the 'sudo' command.
Some people enable 'root' in order to be able to login as 'root'
and thus run GUI apps with root privileges - but this is a bad idea
since you should minimize the amount of code that is running as root
for security reasons. If you really need to run a GUI app as root,
you can launch just that one app as root by using the 3rd-party utility
"
Pseudo".
Note also that an advantage to using 'sudo'
(instead of enabling the 'root' user) is that everything is logged
so you have a record of what you did.
See also: xkcd comic about 'sudo'
How do I navigate to different folders?
When you open a new Terminal window, you will be "in" your home folder.The "current directory" or "current folder" is the folder that you currently are "in". (The word "directory" is the Unix word for what is called a "folder" in OS X. I will usually use the word "folder" from this point on.)
You can find out the current folder by running the following command:
Note also that the usual command-prompt indicates the current folder.
To change to a different folder, you use the 'cd' command
(cd = change directory). E.g. if you have a sub-folder of your home folder
named "Tools", and you are currently in your home folder, you could change
to the "Tools" folder with the command:
The "." folder represents the current folder, while the ".." folder represents the parent folder (the folder that contains the current folder).
You will see these folders listed if you use the "-a" option to 'ls' but otherwise they don't show up in the output from 'ls' since their names start with a dot.
Thus if you want to change to the parent folder of wherever you are,
you can do that with the command:
The top most folder on the disk is named "/". It is sometimes referred
to as the "root" of the disk - but note that this has nothing to do
with the 'root' user. To change to the folder at the top of the disk,
you would do:
And if you have a Finder window open to the folder you want to go to in the Terminal, you can type the 'cd ' part of the command and then drag the folder from the Finder window into the Terminal and it will supply the path to that folder when you drop it. This is especially useful if you use it in combination with Exposé since then the Terminal window doesn't have to be visible when you are in Finder.
And if you find yourself going back to a certain folder repeatedly,
see this
MacOSXHints article
for a method that allows you to define "nicknames" for commonly used folders
and then to go to those folders by using the nicknames instead of the
folder paths. Note that the original version of that article supplies
aliases suitable for the tcsh shell, but I supply aliases and functions
for the bash shell in a comment to the article.
Accessing other disks
The folder "/" is the top level folder of the disk drive that you booted
from (i.e. where the copy of OS X that you are currently using resides)
- no matter what it is called in Finder.
If you have other disk drives (external disks or network-mounted disks
or mounted disk image (DMG) files), they will be accessible in Terminal
via the "/Volumes" folder.
E.g. if you have an external drive named "Fred", its files will be
under the folder "/Volumes/Fred".
Note that (as usual for all paths) if there is a space
(or other non-letter characters) in the name of your external drive,
you will need to put the path inside double quotes when using it with
Unix commands.
E.g. if your external drive is named "Red Disk", you can navigate to it
via the command:
cd "/Volumes/Red Disk"
For consistency, the disk you booted from is also accessible under "/Volumes".
Note that if you start seeing extra numbers appended to the names of your external disks (e.g. "Fred-1"), that is a sign that you hadn't properly unmounted the disk at some time in the past and so the old folder "/Volumes/Fred" is still there. If this happens, unmount and then disconnect all disks and then there should be nothing under /Volumes except your internal disk, so any other folders there are detritus (from improper unmounting) and can be deleted.
What integration is there between Finder and Terminal?
Opening a Finder window from TerminalTo open a Finder window from Terminal, you can use the 'open' command - for example:
Opening a Terminal window from Finder
To open a Terminal window from Finder and set the current folder in that new
Terminal window to the folder shown in Finder, use Marc Liyanage's
"OpenTerminalHere" AppleScript which you can install on your Finder toolbar.
How do I edit text files (e.g. configuration files)?
Unix text files (often called "plain text" files, or "ASCII" files) can be edited with any text editor that can save files in "plain text" format (i.e. without any special formatting or fonts, etc).Another requirement for a text editor for Unix text files is that it save files using the standard Unix convention for end-of-line characters. (See "What is an end-of-line character and why should I care?")
There are several text editors available to you on OS X. They fall into two categories: those with a GUI, and those that run within a Terminal window. The following are only partial lists of those available - there are many more, and each person tends to have his/her favourite.
Text Editors with GUI
TextEdit
- comes with OS X (under /Applications)
- can edit "rich text" documents as well (even MS Word documents
in the latest versions) - but this is to be considered a deficit
in the context of editing Unix text files since you always have to be careful
to make sure it saves as "plain text". I recommend avoiding TextEdit
when editing Unix text files for this reason.
TextWrangler
- the successor to "BBedit Lite" and the baby brother of "BBEdit"
- many high-end features
- can edit files with "root" privileges
- can edit "hidden" files and files on FTP servers
- completely free (as in beer)
SubEthaEdit
- supports collaborative editing (uses Apple's "Bonjour")
- many high-end features
- can edit files with "root" privileges
- older version is free for non-commercial use
Text Editors that run within a terminal
pico (or nano)
- comes with OS X (/usr/bin/pico or /usr/bin/nano)
- relatively easy to use since it provides command reminders at the bottom
- is available even in single-user mode
-
many tutorials available
vi
- comes with OS X (/usr/bin/vi)
- difficult to learn, but powerful once you have learned it
- is available even in single-user mode
-
much better than 'emacs'
-
many tutorials available
emacs
- comes with OS X (/usr/bin/emacs)
- difficult to learn, but powerful once you have learned it
- is available even in single-user mode ( ? I have to check this)
- much better
than 'vi'
-
many tutorials available
The text editors with a GUI are generally considered easier to use, but it is important to know at least the basics of one of the non-GUI editors since these are all that will be available when you are in a non-graphical environment (e.g. remotely logged in via ssh, or in single-user mode when troubleshooting).
Basics of editing with 'pico' (or 'nano')
I'll explain the basics of editing with 'pico' since that is the most
appropriate non-GUI editor for beginners.
To start editing a file "foo" you type the command:
Use the arrow keys to navigate within the existing text. Enter the text you want to add. Use the delete key to delete characters as needed. When you are ready to save, hold down the Control key and press the O key to save the modified file - it will show you the filename at the bottom - and then press Return. If you want to exit without saving, hold down the Control key and press the X key.
Note: Even if you used 'pico' as the command to start up this editor, you may notice that it says "nano" at the top. This is because 'pico' has been replaced by 'nano' in recent versions of OS X. (The file /usr/bin/pico is actually a symbolic link to /usr/bin/nano.) Don't worry about this - the differences between 'pico' and 'nano' are not important for basic editing.
Editing files that are owned by 'root'
It is often the case that you need to edit a configuration file that is
owned by 'root', or which you don't have permission to edit for some
other reason.
For example, the Apache configuration file "/etc/httpd/httpd.conf"
is owned by 'root' and has write permission only for owner.
To edit such files with a non-GUI editor (e.g. 'pico'), all you have to do
is preface the command with 'sudo', e.g.:
Several of the GUI editors can also provide this functionality. E.g. in TextWrangler, if you click on the little badge at top-left that indicates that the file is read-only, you will be asked to authenticate and then you will be able to edit the file and save.
Making a backup copy
It is usually a good idea to make a backup copy of any configuration file
that you are editing. That way, if you find that you have made a mistake,
you can revert to the backup copy and no harm done.
You can make a backup copy using the 'cp' command in the shell,
for example:
What is an end-of-line character and why should I care?
When you look at a text file, you see it as a series of lines.When you edit a text file, you enter a line break by pressing the Return key.
This inserts a special character into the text file at that point. This special character is known as an "end-of-line" character or a "line-break" character. This is sometimes abbreviated as "EOL". Programs that read text files (e.g. text editors) look for these characters and display the text appropriately.
(If you are curious, you can view the end-of-line characters in a text file by looking at it with a program that shows the raw contents of the file - e.g. with the command 'hexdump -C')
However, there are three different conventions that are commonly used for indicating ends of lines in text files. They are usually known as the "Unix", the "Macintosh", and the "Windows" end-of-line characters.
The "Unix" end-of-line character is often known as "line feed" or "newline" and indicated as "\n". It is the character whose ASCII code is 10 in decimal (0x0a in hexadecimal).
The "Macintosh" end-of-line character is often known as "carriage return"
and indicated as "\r". It is the character whose ASCII code is 13
in decimal (0x0d in hexadecimal).
Since Macintosh computers now use OS X, which is a version of Unix,
it is more precise to refer to this as the "traditional Macintosh"
end-of-line character.
The "Windows" end-of-line indicator is actually two characters, not one. It is often referred to as "carriage return, line feed" and indicated as "\r\n". It is thus the sequence of two characters whose ASCII codes are 13 and 10 in decimal (0x0d and 0x0a in hexadecimal). This is also known as the "DOS" end-of-line indicator.
The existence of these three different
conventions for indicating the ends of lines in text files causes
difficulties when a text file is created on a system using one convention
and then read on a system using a different convention.
If you look at a file using a line-ending convention that is different
than what is expected on the system you are on, you may see extra
characters (e.g. ^M) or you may see all the lines strung together
without line breaks. However, most good text editors are capable
of detecting and correcting for foreign line-endings, so if you
are using a good text editor, you might not notice this issue at all.
And if you transfer a file from one system to another using FTP
in "ascii" mode, the FTP software will automatically change the line-endings
to whatever is native on your system.
However, you need to be aware that almost all Unix configuration files and script files must use the standard "Unix" end-of-line characters. If, as mentioned above, your editor is nice to you and hides the fact that a file is using foreign line-endings, you may be surprised to find that the config file or script will simply not work on a Unix system (e.g. OS X).
This means that it is important to configure your editor to always save files using "Unix" line-endings. (This is usually one of the preference items for GUI editors. For the command-line editors, it is the default behaviour, so you don't need to worry about it.)
If you receive a file from some other system that has foreign line-endings,
you can convert it to use "Unix" line-endings by running this
Perl command:
The above command makes a backup copy of the original file with a "~" suffix before editing the original file to change it to use "Unix" line-endings.
The following defines a Bash function (that you could put in your ~/.profile) to do this fixing of the end-of-line characters:
Technical note: an explanation of the above Perl command:
The "-p" option to perl makes it loop over all input lines and print
each line
The "-i~" option makes it edit the file in place after making a
backup copy of the file with "~" appended to the file name
The "-e" option makes it execute the supplied Perl expression
on each line.
The "s/\r\n?/\n/g" expression is a substitution (s = substitution)
that makes it substitute anything that matches the regular expression "\r\n?"
with "\n" and does this globally (the 'g') so it happens even if
the regular expression matches more than once on the same line.
The regular expression "\r\n?" matches a "\r" followed by an optional "\n".
I.e. it matches a "\r" by itself or a "\r\n" pair.
The former is what is used in "traditional Macintosh" text files
while the latter is what is used in Windows text files.
So the above substitution changes both of these cases to use a single "\n"
which is the correct end-of-line character for Unix text files.
How do I create new files and folders?
Creating filesTo create a new empty file in the current folder, use the 'touch' command. For example:
But a more common way to create new files is to use the output redirection facilities to store the results from some other command.
Creating folders
To create a new folder in the current folder, use the 'mkdir' command.
For example:
Ownership and permissions on new files & folders
Newly created files and folders are owned by the user account that created them.
They have permissions assigned according to the current "umask" value
- see discussion in the section about permissions.
How do I delete files and folders?
To delete a file, you use the 'rm' command ("rm" = remove).For example, if there is a file named "foo" in the current folder that you want to delete, you would use the following command:
Deleting multiple files
You can remove several files with one command by listing the filenames
with spaces between them. (If any of the filenames contain spaces,
then you need to enclose those filenames in quotes.)
For example, if you want to delete the 3 files named "foo", "bar",
and "burning bush" from the current folder:
A common mistake is to accidentally type an extra space when doing something like the above. If you ran the command:
Prompting for confirmation
If you use the "-i" option with the 'rm' command,
it will prompt you for confirmation before deleting each file.
This can therefore be used as a sort of "safety" when using 'rm'
with wildcards. For example:
Dealing with permissions
If you use 'rm' with a file that is read-only, you will be prompted to
confirm the deletion. If you are deleting a bunch of files
(e.g. with wildcards), this may be undesirable. To force the deletion
of files without asking for confirmation, you can use the "-f" option.
Note that to delete a file from a folder, you only need write permission on the folder - the permissions on the file itself are not relevant. That is why you can override the warning about deleting a read-only file.
If the ownership or permissions are such as to prevent deletion
of a file, you can force the deletion by prefacing the 'rm' command
with 'sudo'
(See "How do I gain "root" privileges?")
Exceptional cases
If you have difficulty in deleting a file even with the use of 'sudo',
it is likely due to some special "flags" having been set on that file.
You can see the state of these special flags by using the command 'ls -lO'
on the file.
(In Tiger (OS X 10.4), you would use 'ls -lo')
In some cases it is necessary to change the flags before you
can delete the file - you do this with the 'chflags' command.
See
this article
on the MacOSXHints site for details and links to other articles
about deleting locked files.
Deleting a folder
To remove a folder (directory), you use the command 'rmdir'.
But you can only remove an empty folder using this command.
If there are any files (or sub-folders) in the folder, you need to remove
them first. If you are having difficulty with this, don't forget
that files/folders whose names start with a dot (.) are not shown
when you do 'ls' - you need to use 'ls -a' to see all files in a folder.
If you are sure that you want to delete a folder and all that it contains
(files and sub-folders), you can use the "-r" (for "recursive") option
to 'rm'. This is often combined with the "-f" option to prevent the
confirmation prompts about read-only files. For example:
You will want to be extra, extra careful when using this option - especially if you are also using wildcards!
Read 'man rm' and 'man rmdir' for full details.
What are the most commonly used commands?
Here are some short notes on the most commonly used commands. (I explain the origin of the command name in those cases where it isn't obvious.)Note that there are more details about several of these commands in some of the other sections of this FAQ.
For full details about any of these commands, read the corresponding 'man' page - for example to find out more about the 'ls' command, read 'man ls'.
(By the way, the man pages are split up into sections. You will mostly be interested in section 1 since that describes the commands you can run from the shell. Sections 2 & 3 describe C-functions that are useful when you are writing a program.)
And if you are looking for a command to do something that isn't mentioned here, note that 'man -k some_subject' will show a list of man pages where "some_subject" is mentioned. That might help you find a command to do what you want. Another method of finding useful commands is to look at the "See also" section of related commands.
The 'whatis' command will give you a quick summary of what a given
command is for.
See
this MacOSXHints article
for a way to get a list of all commands in your execution PATH,
with a summary of what each command does.
File and folder management
cd ("change directory")
- to navigate to a different location in the
filesystem
pwd ("print working directory")
- show the current location in the filesystem (the "current directory")
ls ("list")
- to show details about files and folders
touch
- to create an empty file or to change the date on an existing file
mkdir ("make directory")
- to create a new folder
chown ("change owner")
- to change the owner of a file or folder
chmod ("change mode")
- to change the permissions of a file or folder
mv ("move")
- to move a file or folder to a different location in the filesystem, or just to rename it
cp ("copy")
- to copy a file or folder
ln ("link")
- to make a link to a file or folder
rm ("remove")
- to delete a file
rmdir ("remove directory")
- to remove a folder
find
- to search the filesystem for files or folders matching given criteria
mdfind
- to search for files or folders using the "Spotlight" meta-data
locate
- to see where certain files are located (uses a database that is updated periodically)
du ("disk usage")
- to show the amount of disk space used by a file or folder
tar ("tape archive")
- to bundle up a bunch of files/folders into one file
zip, gzip
- to bundle up a bunch of files/folders into one file and compress it to take less disk space
unzip, gunzip
- to uncompress a .zip or .gz file
File content
more
- to display the contents of a file page by page (press Return to go down one line, press space to go down one page)
cat ("concatenate")
- to display the full contents of a file or to concatenate two or more files into one
head
- to display the first part of a file
tail
- to display the last part of a file
mdls
- display the meta-data for a file
xattr
- display the extended attributes for a file
diff
- to compare two text files (or, with the "-r" option, two folders)
cmp
- to compare two binary files
pico
- an editor for displaying and editing text files
grep ("global regular expression print")
- to search inside a file for lines matching a given pattern
sed ("stream editor")
- to modify the text that is streaming through a pipe
sort
- to sort the lines of text files
uniq
- to filter out repeated lines of text files
fold
- to wrap long lines of text files (useful when printing)
hexdump
- to show the contents of a file as hexadecimal numbers
textutil
- convert plain text to HTML or RTF (and vice versa)
Processes
ps ("process status")
- to see detailed info about the processes running
top
- to get a summary of the processes running and resource consumption
kill
- to terminate a process identified by process-id (or to send other signals)
killall
- to terminate a process identified by program name
lsof ("list open files")
- show which files (and sockets) are open by which program
fs_usage ("filesystem usage")
- show which programs are accessing (reading or writing) the filesystem
dtrace
- trace any system activity (processes or file access). Start by reading 'man -k dtrace'
Networking
ifconfig ("interface configure")
- to display and configure network interface parameters
ping
- to send a test packet to another computer (amusing article about the history of this command)
traceroute
- to see the route taken by packets across a network
host
- to find out the IP address corresponding to a hostname or viceversa
curl
- to download contents of a document via a URL
ftp
- command-line FTP client
ssh ("secure shell")
- remote login to another computer
System Info
hostname
- reports the name of your Mac
sw_vers
- reports the OS X version that you are using
system_profiler
- reports on the hardware and software that is on your Mac (i.e. a command-line version of the "System Profiler" utility)
sysctl -a
- reports values of the kernel parameters
Disk management
df
- shows info (including the amount of free space) about mounted disks
diskutil
- versatile disk management utility (info, formatting, mounting, repairing, etc)
Misc
echo
- to send something to "standard output" (useful in pipes, etc)
open
- to open a file (in the GUI application that is associated with that file type) or a folder (in the Finder)
date
- to display the current date and time (in various formats)
sleep
- to pause execution for a given number of seconds (useful in scripts)
wc ("word count")
- to display the number of characters, words, and lines in a text file
pbcopy ("pasteboard copy")
- copy to the clipboard from "standard input"
pbpaste ("pasteboard paste")
- paste from the clipboard to "standard output"
xargs
- pass arguments to another command (useful in pipes)
defaults
- read or write preference settings
id
- report info about your user account (numeric user-id, group ids, etc)
sudo ("superuser do")
- execute a command with 'root' privileges
su ("switch user")
- start a new shell using a different user account
man ("manual")
- to display detailed information about a command.
How does wildcarding work? (or why do I see stars?)
Suppose that you wanted to have a command apply to a bunch of files or folders. You could of course type out the names of the files/folders you want the command to apply to, separating each filename from the next with a space. But that is irksome if there are a large number of files/folders. The use of wildcards solves this problem.
For example, if you wanted to copy all of the files in the current folder
whose names started with "foo" to some other folder, you could do
that with the following command:
Wildcarding is often referred to as "globbing". The expansion of the wildcards to create a list of filenames is handled by the shell. That means that the details are dependent on which shell you are using. But the basics are the same for most shells:
* ("asterisk" or "star") is a wildcard that matches any string of zero or more characters
? is a wildcard that matches a single character
[xyz] matches any one of the characters within the square brackets
For example, if you wanted to copy the files "foo1", "foo2", "food", "foor",
etc - but not the files "foo" or "foobar" or "foo10", you could
do it with:
But there are two exceptions regarding the * and the ? wildcards - they don't match an initial dot (.) and they don't match a slash (/).
The reason for the initial dot exception is that files whose names start with a dot are intended to be hidden and thus it is considered undesirable to include them in wildcarding.
The reason for the slash exception is to allow more control over matching of files in other folders - recall that the slash is used to separate folders/directories in pathnames.
If you wanted to copy all files from the current folder except
those whose names start with a dot, you could use:
If you wanted to copy all files from the folder "~/MyStuff/Docs"
whose names started with "foo" and ended with either "a", "b", or "c"
before the suffix ".txt", you could use:
The above examples have all used the 'cp' command, but any command that can handle a list of filenames could have served as well.
Preventing wildcard expansion where not wanted
As mentioned above, the wildcard expansion is handled by the shell.
That means that the command (e.g. 'cp' in the above example) does not
see the wildcards - it receives a list of filenames.
(That's good because most commands don't understand wildcards.)
This means that in the relatively unusual case where a command does
handle wildcards, you need to take special steps to prevent the shell
from expanding the wildcards before they get to the command.
For example, if you are using the 'find' command to find all files/folders
under the current folder whose names start with "foo", you would
do it with:
Details
To get full details about wildcarding in the Bash shell, read 'man bash'
and look in the section titled "Pathname Expansion".
For the Tcsh shell, read 'man tcsh' and look in the section titled
"Filename substitution".
One final note: the meaning of the wildcard characters (e.g. * and ?) is different when used in regular expressions in 'grep' or Perl than it is in shell globbing. It's easy to get confused.
How do I make a command iterate over a bunch of files?
There are several ways of iterating over a bunch of files.Perhaps the most general way is to use one of the shell looping constructs. For example, in the Bash shell, you can use 'for' to loop over a given set of files. Suppose that you had a script named 'my_script' that took a single filename as a command-line argument.
If you wanted to run that script with all of the files in the current folder in succession, you could use:
Here's another example:
Other looping constructs are discussed in the Bash shell tutorials referred to in the last section of this FAQ.
If you wanted to run that script on each of the files in the current folder
and all those in sub-folders, and in sub-sub-folders, etc, you could
use the 'find' command's "-exec" option like this:
If you want to run a command on all of the files found by a 'find',
and the command takes a list of files as command-line arguments,
you can do that by using the 'xargs' command. For example:
It's slightly trickier to use 'xargs' when the command you want to run
takes other arguments and the list of files has to come before
those other arguments. For example, if you wanted to copy all those ".java"
files to a different folder, you'd want to use the 'cp' command.
But it has the form:
cp file1 file2 file3 ... destinationFolder
To get 'xargs' to insert the list of files at a specified place in
the command line, you use the "-J" option and supply your choice of placeholder
- for example:
How can I redirect the output from a command into a file?
If a command gives a large amount of output, it is often awkward to deal with in the Terminal window. In such cases, or if you want to preserve the output for later examination, you can "redirect" the output to a file.
To do this, you append the command with the ">" symbol
(think of it like a funnel) and a filename.
For example the following command:
If you want to append the output to an existing file
(instead of creating a new file), you can do this by using ">>"
instead of ">".
For example:
A similar redirection can be done with input, though this is less common. For input redirection (to tell a command to take its input from a file), you use the "<" symbol.
It is also possible to redirect the output from a command so that it
acts as the input to a second command - this is done with the "|" symbol
(the symbol above the backslash on standard US keyboards)
and is referred to as "piping". For example, the following command:
One final, more advanced note:
Some commands will send some of their output to "STDERR" instead of "STDOUT".
STDERR can be used (by the programmers of a program) to separate the
error messages from the usual output of the program. When you run a command
in the Terminal, both STDOUT and STDERR get displayed in the
Terminal window.
When you redirect the output of a command using ">" as above,
it is only the STDOUT that gets redirected. The idea is that you likely
want to see the error messages, not have them hidden away in the file
where you might not see them until later.
If you want to redirect both STDOUT and STDERR to a file,
you can do this in the Bash shell by using "&>" instead of ">".
Here's a test script that you can use to try things out:
How do I run a script?
WarningFirst off, let me caution you that if you need to ask this question, you are likely not experienced enough with Unix to be able to judge whether the script can be trusted not to do harm to your system or your files. You should only run scripts that you understand, or that came from a source that you trust implicitly. The same is true, of course, of any Unix command, but a script embodies a series of Unix commands and so is harder to understand and has the potential to do even more harm.
Save the script as a file
The first thing you need to do is save the script in a file somewhere
on your system - e.g. somewhere under your home folder.
Many people create a folder named something like "Scripts" for this purpose.
Since your home folder is designated as "~", the path to that folder would be
"~/Scripts".
If you downloaded the script as a file, all you need to do is move that file
to a convenient place as described above. But if you are copying the script
as text from a web page, etc, you will need to use a text editor to save
the script to a file.
See "How do I edit text files?".
Note especially that it is important that script files be saved with
"Unix" line-endings - see
"What is an end-of-line character and why should I care?".
(If you are getting an error message about a "bad interpreter" when you run
the script, that is likely an indication that the line-endings
are not correct.)
Make the script executable
Next you need to make the script be executable. You use the 'chmod' command
to do this. For example, if you have saved the script in the file
"~/Scripts/foo", you would execute the command:
Run the script
Now you can run the script by typing the full path of the script file
in a Terminal window - e.g.:
After you have done that, you can run the script just by typing its name in a Terminal window, e.g.:
Often scripts expect certain parameters to be supplied on the command-line. If so, you type the name of the script, then a space, then the first parameter, then a space, then the second parameter, etc. If there are spaces in the parameter value itself (e.g. if the parameter value is "John Doe") then you need to surround the parameter value with quotes. E.g.:
The very first line of a script should indicate the type of the script - i.e. it should indicate what type of "shell" or other "interpreter" is to be used to execute the script. This is done by using a special syntax starting with "#!", for example, a Bourne shell script would start with the line
How can I make a script double-clickable? (How do I run a script from Finder?)
The easiest way to make a script double-clickable is to save it in a file named with a ".command" suffix. Double-clicking on such files in Finder will open up a new Terminal window and start the script running in that window.Another way is to wrap the script in an AppleScript. You can execute a shell script (or any Unix command) from AppleScript with the "do shell script"s command. See the last part of the answer to the question "How can I make my script interoperate with AppleScript?" for more details and examples.
Other methods for making scripts double-clickable are provided by several 3rd-party utilities - e.g. "Platypus"
Note also that the 3rd-party utility "OnMyCommand" makes it easy to execute scripts from a contextual menu (via right-click or Control-click).
How can I make my script interoperate with AppleScript?
Invoking AppleScript from a shell scriptYou can execute AppleScript from within your shell script by using the command 'osascript'. It is usually best to use this with a "here document" in a shell script. For example, if your shell script had set the variable $fullpath to the full path to a file, the following would open Finder's "Get Info" window for that file:
If you want to get some information back from the AppleScript,
you can invoke it inside backquotes and assign the result to a shell variable.
Here's an example that illustrates that:
Invoking a shell script from AppleScript
You can execute a shell script (or any Unix command) from AppleScript
with the
"do shell script" command.
But note that it is important to get your shell script working from the Terminal command-line before attempting to wrap it in an AppleScript. Many beginners make the mistake of trying to use AppleScript as a development environment for shell scripts. Note also that in most cases it is a mistake to have several "do shell script" lines in your AppleScript - instead you should put all of the Unix commands together into one shell script, make sure it works when invoked from Terminal, and only then start testing it via your AppleScript. And if you are doing anything more than the most basic of scripts, it really is essential that you read Apple's Technote on the 'do shell script' command.
In particular, note that the current working directory (folder) and the environment variables are different when a shell script is run from 'do shell script' than when it is run from Terminal. (See the AppleScript in the section on environment variables.)
See this MacOSXHints forums thread for an example of an AppleScript droplet that invokes a Perl script to do the heavy lifting.
How can I compile open source software?
There is a lot of "open source" software available via the web. Much of it will work on OS X, although sometimes it requires some small changes. Often this software is available only in source code form and thus it often needs to be compiled before you can use it.Note that in order to compile or build or even to use much of this software requires at least a basic knowledge of Unix. Be sure to read the other sections of this Unix FAQ before you go further.
Fink and MacPorts
There are a few organizations
(Fink
and MacPorts)
that are trying to make most of this software available with a minimum of fuss.
They modify the software where needed to make it work on OS X and they also
provide pre-compiled (binary) versions of the software to make it
easier for you.
So the first thing you should do if you are interested in a particular
software package is check to see if it is available via Fink or MacPorts.
It will be much easier for you to install the software via one of these
than doing it "manually".
To check if a package is available via Fink:
http://pdb.finkproject.org/pdb/index.php?phpLang=en
To check if a package is available via MacPorts:
http://www.macports.org/ports.php
If the software you want is available via Fink or MacPorts, then you would follow their instructions to first install Fink or MacPorts and then to install the specific package you want. But note that unless they provide the software you want in pre-compiled (binary) form, you will need to have Apple's developer tools installed (see below). If you encounter any problems in using Fink or MacPorts, the first thing you should do is read (or re-read!) the documentation they provide - note in particular that many common problems are discussed in their FAQ documents.
Many open-source programs use X11 for their graphical user-interface. See the X11 FAQ that is in the "Unix - X11" section of the MacOSXHints forums.
Apple's Developer Tools
In order to be able to compile software from source - either "manually"
or via Fink or MacPorts - you will need to have Apple's developer tools
installed on your Mac. These tools (which include the GCC compiler and
necessary support files) are not installed by default in OS X.
They are on the Leopard and Tiger DVDs and were supplied on separate CDs
with Panther and Jaguar. But you can also download the latest version
of the developer tools from Apple's developer web site after registering
(for free) as a developer:
https://developer.apple.com/
(But be warned - the developer tools download is many hundreds
of megabytes!)
The version of the developer tools for use with Leopard, Tiger, and Panther
is called "XCode Tools". Note that if the version of the developer tools
that you have on CD or DVD is too old, it may not be suitable for use with
current software packages and so you may need to download the updated version
from the Apple web site.
Note that it is usually a good idea to install the "X11SDK" package when you are installing "XCode Tools". This is an optional part of the XCode install (i.e. it isn't installed by default) but you will need this SDK if you want to compile any program that uses X11 (see above).
Compiling "manually"
If the software you want is not available via Fink or MacPorts,
then you can try compiling it "manually". When you download open-source
software, it usually comes in compressed and bundled form. For example,
it may come as a .zip file or as a .tar.gz or .tgz file. The first thing you
need to do is uncompress and unzip or untar the software. Usually you can do
this most simply by just double-clicking the downloaded file in Finder.
To do it from the command-line (in Terminal), you would use commands
like 'unzip', 'gunzip', 'tar -xvf' etc. (Read the 'man' pages for these
commands for details) When you have done this, you should have a folder
with the source code and associated documentation.
Almost all open-source software includes some text files that explain how to compile and install the software. These instructions will be in files named "README" or "INSTALL" and you really do need to read them before going further.
The almost universal recipe for compiling and installing modern open-source software is to run the following commands after changing directory to the sub-folder that contains the source code files:
./configure
make
make install
But don't just blindly run these commands! First you need to read the "README" or "INSTALL" documents that came with the software. They will tell you if you need to set any configuration options etc.
The "./configure" command runs the "configure" script that is in
the current directory. This script creates a "Makefile" that is specific
for the version of OS X you have on your machine.
The "make" command uses the instructions that are in the "Makefile"
to compile the source code and create the binary executable(s).
The "make install" command moves the executable(s) to their usual location
(e.g. to /usr/local/bin) and installs support files (e.g. man pages)
You might not want to run the "make install" command if you want to keep the executable(s) in some other location. In any case, it is very important that you make yourself aware of where the executable(s) will get installed before you run this command (i.e. you need to read the documentation!) since otherwise the "make install" might overwrite files that came as part of OS X.
If the software doesn't come with a "configure" script, it probably comes with a "Makefile" already set up for the common configurations. In this case, you will likely need to edit that Makefile to set it up for your configuration.
If you get any errors when doing the above, first check that your version
of the developer tools is up to date. Otherwise, try googling for the exact
words of the error message you received (omitting any parts that refer to
specifics of your machine etc) and you are likely to find discussions about
how to fix the problem.
But be aware that much open-source software is targeted for
Linux
and makes assumptions that are not true for OS X. Making the necessary
changes in this case will usually require extensive programming experience
- hence the huge value added by the Fink and MacPorts projects.
A final word: the mistake made by many beginners to open-source software is not spending sufficient time in reading the documentation provided with the software. It often takes a lot of time in reading and understanding the documentation before you can install or use some of this software. Be sure you at least read the basic docs - e.g. the "README" and "INSTALL" files before asking for help when something goes wrong.
How can I remember all these complicated commands? (Answer: shell aliases)
In order to save typing, or just to make it easier to remember a complicated command, most experienced Unix users customize their shell environment by setting up "shell aliases". The word "alias" in general English usage means an alternative name for something.Unfortunately, the word "alias" is used for several quite different things in a computing context. Here I will be discussing "shell aliases" that provide a way to specify alternative names for commands. A second, unrelated use of the word "alias" is with "Finder aliases" (as created via the "Make Alias" menu item in Finder's "File" menu) - these are discussed in the next topic.
The ability to specify an alternative name for a command (or a command sequence) is a feature of the shell. The syntax for specifying an alias depends on which shell you are using. (see above for info on shells)
Aliases in Bash
To specify an alias in the Bash shell, you use the command 'alias'
which takes the following form:
alias desired_name='command sequence'
(with no spaces on either side of the '=' sign)
For example, the following would make 'll' (double-L) be an alias for
the command 'ls -l':
Functions in Bash
To specify an function in the Bash shell, you use a command of the
following form:
desired_name () { command sequence ; }
For example, the following would make 'll' be a function that can be used
to do 'ls -l' on specified files or folders and pipe the results
into 'more':
The $@ in the above represents all of the command-line arguments
- i.e. the names of the files or folders. For example, using this alias
we could do 'll /Users/fred' and it would expand to
'ls -l /Users/fred | more'
I enclosed the $@ in double quotes in order to avoid problems with file
or folder names that have spaces in them.
Aliases in Tcsh
To specify an alias in the Tcsh shell, you use the command 'alias'
which takes the following form:
alias desired_name 'command sequence'
In other words, it is the same as the 'alias' command in Bash except
you use a space instead of an '=' sign.
For example, the following command would make 'll' (double-L) be an alias
for the command 'ls -l':
There is nothing equivalent to Bash's functions in Tcsh. However the alias command in Tcsh is more powerful in that it understands positional parameters that can be used to substitute command-line parameters similar to the way used in Bash functions.
For example, the following makes 'll' be an alias that can be used to do 'ls -l' on specified files or folders and pipe the results into 'more':
Using command-line arguments in Tcsh aliases:
The first arg is \!:1
The second arg is \!:2
The last arg is \!$
To get a list of all args, use \!*
General considerations
When defining aliases or functions, you should avoid using the names of
existing commands. For example, even if you are sure that you always want
to use the "-l" option with the command 'ls', it would be a bad idea
to make an alias like:
alias ls='ls -l'
since that would effectively replace the standard 'ls' command which might
be used in scripts. Instead, you should use some other name that doesn't
collide with any of the existing commands.
As another example, some beginners get the idea of defining an alias for 'rm' that makes it do 'rm -i' - they think this will provide extra security against accidents with wildstars, etc. But if they start relying on the "-i" behaviour when they invoke 'rm', it will be catastrophic if this alias is ever not there for some reason (e.g. there is an error in a shell startup file that prevents their aliases from being defined, or they are logged into a different user account).
Note that if you want to make sure that you get the original command,
you can put a backslash in front of it when you invoke it.
For example, the following would execute the original 'ls' command
even if you had ignored my advice and used an alias like the one above
that redefined 'ls':
Where to put your aliases
In order for your aliases and functions to be persistent
(i.e. so they exist in all new Terminal windows) you need to put them
in some file that gets read by your shell startup files
(see "What startup files are read by the shell?").
You could put them directly in the startup file (e.g. in ~/.profile)
but it is more flexible if you keep them in a separate file that you
"source" from your shell's startup file.
For example, I keep my Bash aliases and functions in the file
"~/.aliases.bash" and I have the following line in my ~/.profile:
That way, if I add a new alias or function, I can "source" my "~/.aliases.bash" file in order to get the new definitions in my current Terminal session.
Note that aliases are not inherited by sub-shells, so if you want your aliases to be present in sub-shells then you should put them in one of the setup files that is read by "non-login" shells.
You can see lots more examples of bash aliases and functions in my "~/.aliases.bash" file.
What is a "symbolic link"? How is it different from a "Finder alias"?
Symbolic LinksA "symbolic link" (often abbreviated as "symlink") provides a way to make a file or folder appear to exist at a different location in the filesystem. I.e. it is a sort of pointer to a file or folder that exists elsewhere.
For example, if you use 'ls -ld' to look at the "/tmp" folder on OS X, you will see that "/tmp" is not actually a real folder - it is a symbolic link to "/private/tmp" which is the real folder.
This is called a "symbolic link" to distinguish it from a "hard link" which is a different type of filesystem entity that is used more on traditional Unix systems than on OS X. (Unlike traditional Unix filesystems, the HFS+ filesystem used by OS X does not have "hard links" as native entities - they are supported, but via extra infrastructure. For details on HFS+, see this Apple developer doc.)
You can create a symbolic link by using the "ln -s" command
(the "-s" option is necessary in order to create a "symbolic" rather
than a "hard" link). Read 'man ln' for details.
For example if you wanted to make it so that the folder
"/Library/WebServer/Documents" (the top of the hierarchy for HTML
documents served by Apache) appeared as if it was under your home folder
(for convenience), you could create a symbolic link like this:
Note the order of the two paths in the 'ln -s' command - the first path is that of the existing file or folder, the second path is where you want the symbolic link to be created. (This is the same order as the 'cp' or 'mv' command)
Finder Aliases
A similar purpose is served by "Finder aliases" (not to be confused
with "shell aliases" as described in the previous section of this FAQ).
You create a "Finder alias" by selecting an existing file or folder and
then using the "Make Alias" menu item from Finder's "File" menu.
This creates a "Finder alias" file right next to the original,
but with the word " alias" appended to the name. You then can move the
alias file to wherever you want it and it will serve as a pointer to the
original file or folder. You can rename the alias anything you want.
You can create Finder aliases from the command-line via this AppleScript embedded in a shell script.
Both Finder aliases and symbolic links appear in Finder with a little arrow
in their icons. There is no easy way (as far as I know) to distinguish
a Finder alias from a symbolic link in Finder. If you use "Get Info"
in Finder on a symbolic link, it will show its "Kind" as "Alias"
- which is a bit misleading at least. But a Finder alias will usually be
quite a bit larger in size (number of bytes taken up on disk)
- something like 35 KB for a Finder alias as opposed to 4 KB
(the minimum block size) for a symbolic link.
If you use 'ls -l' on a Finder alias, it will be listed as a zero-length file.
The contents of the alias are in the
"resource fork" of the file.
To delete either a Finder alias or a symbolic link, you can use the 'rm' command (in Terminal) or move it to the Trash using Finder. Deleting a Finder alias or a symbolic link does not affect the original - you are just deleting a pointer to the original.
One advantage of Finder aliases over symbolic links is that Finder aliases
keep pointing at the original file or folder even if the original file
or folder is moved to a different location on the disk.
Symbolic links are just a textual description of the Unix path to
the original file or folder, so if you move the original to some other
location, the symbolic link will need to be updated to make it point
to the new location. (To update a symbolic link, you just delete it
and then create a new one with the desired path.)
But most Unix-level commands and utilities do not understand Finder aliases, thus it is necessary to use symbolic links if the file or folder will be accessed from a Unix-level program (e.g. Apache).
What is a "resource fork"? Are resource forks supported by Unix commands?
A "resource fork" is a part of an OS X file that is sometimes used to store extra information, often meta-information. Resource forks started on the "classic" Mac OS but are still supported in OS X where they have been subsumed into the more general "extended attributes" feature. To distinguish it from the resource fork, the standard part of the file is sometimes referred to as the "data fork".You can read or write resource forks via the facilities provided for extended attributes. The name of the extended attribute for the resource fork is "com.apple.ResourceFork". But you can also read or write resource forks via a special (and rather strange) syntax. To see the size (number of bytes) of the resource fork of a file named "foo":
Since resource forks are implemented as extended attributes, any Unix command that supports extended attributes will support resource forks.
What is an "extended attribute"? Are xattrs supported by Unix commands?
An extended attribute is part of an OS X file that is sometimes used to store extra information, often meta-information. There can be an arbitrary number of extended attributes associated with a file. Each extended attribute has its own name. You can read and write extended attributes using the 'xattr' command. To list the extended attributes on a file named "foo":It is important to note that not all of the Unix commands support extended attributes - they often silently ignore them which means that you might lose information. For example, if you make a backup of some files with extended attributes using a Unix command that doesn't support extended attributes, your backup would not be a faithful copy of the original files. To find out if a particular command supports extended attributes, read the man pages carefully, or (better) do a test with sample files. Here's an example:
How do I set an "environment variable"?
An "environment variable" is a shell variable that persists in sub-processes. Normal shell variables do not get transferred into new processes that are started from that shell. Environment variables are often used to define configuration parameters for command-line programs. They are often set in the shell startup files. The PATH variable discussed above is an environment variable.
Environment variables in Bash
To set an environment variable in the Bash shell, use the 'export' command.
For example, the following command would set the environment variable FOO
to have the value 1234:
To see a list of all environment variables and their values,
use the 'printenv' command.
The Perl script
'showLaunchdEnviron' will show you the environment
(variables, etc) that will be present for 'launchd' scripts.
Environment variables in Tcsh
To set an environment variable in the Tcsh shell, use the 'setenv' command.
For example, the following command would set the environment variable FOO
to have the value 1234:
Environment variables in GUI programs
A GUI program will inherit the environment variables from the process that
started it. So if you launch a GUI program from the command-line, it will
inherit the environment variables of the shell where you launched it.
But if you launched the GUI program via Finder (by double-clicking)
or the Dock (or some other mechanism in OS X's GUI environment),
it will inherit the environment variables of the "loginwindow" process.
To set such environment variables, follow the procedure explained in
this Apple TechNote.
You need to create a folder named ".MacOSX" in your home folder
(note that since there is a dot at the beginning of the
folder name, it won't be visible to a simple 'ls'
- you would need to use 'ls -a')
and then create a file named "environment.plist" in the ~/.MacOSX folder.
Put the definitions of the environment variables that you want defined for all
GUI programs in that plist file using a format like the following example:
(You can use the "Property List Editor" that comes with
Apple's developer tools or a 3rd-party plist editor - or just an ordinary
text editor
if you are careful to get the format correct.)
Here's an AppleScript that you could use to see what environment variables
are set for GUI apps:
showEnvVars
(Run this from "Script Editor" or save it as an application
and then double-click it.)
I note also that you can define environment variables for one specific application in that app's "Info.plist" file (inside the app's bundle) via a dictionary entry named "LSEnvironment". (see this Apple doc on Info.plist files)
Finally, although it's not usually recommended, if you really really want to set an environment variable globally, you can specify it by using the 'setenv' command in the file "/etc/launchd.conf". Any 'launchctl' commands (see 'man launchctl') in that file will get executed at system startup.
Terminal is behaving strangely - how can I fix it?
Very small Terminal windowIf your Terminal window is showing up as very small, that is likely due to the Monaco font being disabled (e.g. using Font Book).
Underscores not showing up
If underscores (_) are not showing up, this is likely a problem with
your choice of font. E.g. the font "Courier New 10pt" has been reported
to cause this problem.
Strange characters in Terminal window
If you are seeing strange characters in your Terminal, that may also
be a problem with a font. Or it may just be due to bad control sequences
having been sent to Terminal - e.g. if you pasted in binary code
accidentally.
Try resetting Terminal's settings by running the following command
(you may have to type it "blind" if Terminal is screwed up):
If you are getting some strange error messages in your Terminal windows when they first open, that likely indicates a problem with your Terminal preferences. Try removing (or just renaming) the Terminal preferences file ("~/Library/Preferences/com.apple.Terminal.plist") and then restarting the Terminal application.
Miscellaneous problems
First try removing your Terminal preferences as explained above.
If you still have a problem, then look at what you have in your shell
"dot" files - see
"What startup files are read by the shell?".
If you can't figure out what the problem is, try removing (or just renaming)
those "dot" files.
One final place to look for problems is the "default.term" file under "~/Library/Application Support/Terminal". This is the file that stores the default settings for Terminal windows.
How do I do XYZ?
If you are faced with a particular task and are wondering how you can accomplish that task via Unix commands, the first thing you should do is try to find out if there is a command that does something related.You can look through lists of Unix commands to see if something strikes your eye. A list of commonly used commands is in an earlier section of this FAQ.
Another thing to try is running the command 'man -k' with various possible
keywords. E.g. if you want to find the lines in a file that match a certain
pattern, run the command:
man -k pattern
One of the results will point you to the 'grep' command (which was mentioned
in the list of commonly used commands).
Read the man pages for the commands that seem like they might be useful
and then proceed to read about the commands listed in the "See Also" section
of those man pages.
Often you will need to combine several commands in a pipeline or shell
script to do what you want. (See the resources in the
last section of this FAQ on learning shell scripting.)
Sometimes you can find a script written by someone else that does something
similar to what you want. Modifying someone else's script is usually easier
than starting from scratch. Google is of course a good starting point when
looking for a script.
If you know how to do some operation on one file but you need to apply it to many different files, or to a hierarchy of files & folders, see the above FAQ section about looping ( "How do I make a command iterate over a bunch of files?").
Browse through my shell aliases file to see if some of the things there might help you towards a solution. Note that I use that file as a place to keep miscellaneous reminders of how to do things as well as actual aliases & functions.
Perhaps one of the Bash scripts or Perl scripts I have written will be useful to you.
Where can I learn more?
There are many resources for learning about Unix available on the web. Although there are differences between the various flavours of Unix (OS X, Linux, Solaris, AIX, HPUX, etc), the commonalities greatly outweigh the differences. Googling for specific search terms will likely find answers if you have specific questions. If you want to restrict your search to things that are relevant to OS X, add in "OS X" (in quotes) at the end of your Google search terms.
For example, to find Unix tutorials, use this Google search:
http://www.google.com/search?q=Unix+tutorials
To restrict the search to Unix tutorials that at least mention "OS X",
use this Google search:
http://www.google.com/search?q=Unix+...als+%22os+x%22
A good Unix tutorial for OS X is provided at Adrian Mayo's Learning Center.
There are several other FAQs about Unix that contain a lot more advanced
questions. You can find them via Google:
http://www.google.com/search?q=Unix+FAQ
To learn more about shell scripting using 'bash':
http://developer.apple.com/documenta...hellScripting/
http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/abs/html/
http://www.gnu.org/software/bash/manual/bashref.html
Searching for answers to specific problems
In general, if you have a question about a specific command, or an error
message that you have received, it is quite likely that you will find
sufficient information via Google - you should always do a search before
asking a question. Most basic questions have been asked before.
Most error messages have been seen before.
If you google for the exact text of the error message (omitting anything
specific to your situation), putting it inside quotes so it will be treated
as a phrase, you likely will find discussions of how to fix the problem.
E.g. if you are trying to execute the command "foo" and you get the following
error message:
"-bash: foo: command not found"
then searching for the phrase "command not found" (including the quotes)
would be a good strategy:
http://www.google.com/search?q=%22command not found%22
And of course, a lot of questions have already been discussed on the
MacOSXHints forums (where I am active)
and in articles on the main MacOSXHints site. To limit your Google search
to the macosxhints.com site, you can use Google's "site:" directive.
E.g. to search for "AppleScript" on the macosxhints.com site, you would do:
http://www.google.com/search?q=AppleScript+site:macosxhints.com