| Task: | Commands: |
|---|---|
| Copy file to multiple directories (in same dir). | echo dir* | xargs -n 1 cp file |
| Reset password of a user to a temporary password and force reset of temporary password upon user's next login. | echo -e "pass\npass" | passwd user; chage -d 0 user |
| Parse the name 'Francis' from the string echo. | echo "drwxrwx--- 3 plf plf-grading 4096 Mar 4 07:31 COSC206_201601_0_p05_E00064478_Francis" | awk -F "[ \t_]+" '{ print "["$14"]"}' |
| An ls produces file names of following format -
COSC206_201601_0_p05_E00014478_Francis COSC206_201601_0_p05_E00032298_Cameron and task is to extract the names from the end of the strings and sort them. The following will yield: [] [Cameron] [Francis] | ls | awk -F "[ \t_]+" '{ print "["$6"]"}' | sort | uniq |
| Accomplish similar task as above but using sed: Note: awk would have (in this case) produced en empty [], this sed command might not (depends on how you use it): | ls | sed 's/.*_//' | sort |
| Similar task as above, but instead choosing to extract the E-numbers and names and print them in reverse order. Francis E00014478 Cameron E00032298 |
ls | sed 's/.*_\(.*\)_\(.*\)/\2 \1/' | sort |
| awk -F '[:]' '{ print $3 }' /etc/passwd | sort -n | parse /etc/passwd on colons and sort uids (col 3) numerically. |
| Common Terminology & Slang: Note: These terms are NOT canonical in any way. These terms are misused frequently from their canonical meaning when bandied about, and thus this is to aid in comprehending a light interpretation of the speakers meaning. | ||
|---|---|---|
| Term: | Meaning: | |
| Note: Some of these terms are technical; some are simply slang. | ||
| CAT a file | View file contents (from command: cat) | |
| CD | Change directory. | |
| CLI | Command Line Interpreter (shell, console, etc.). Compare to: GUI Note: One can open a CLI from a GUI; however, sometimes these terms are casually used to contrast graphical & non-graphical sessions. | |
| CMD | 1.) Enter a command at the shell. 2.) Generic foo-like term for command. | |
| CWD | Current Working Directory (Windows; '.' is the CWD). Compare to: PWD | |
| DIR | Meaning 1: short for folder or directory. Meaning 2: Windows command for ls | |
| EOF | End of file marker (hidden, non-printable character at end of file). | |
| GUI | Graphical User Interface (Desktops, point-and-click, etc.). Compare to: CLI Note: One can open a CLI from a GUI; however, sometimes these terms are casually used to contrast graphical & non-graphical sessions. | |
| KBD (or kbd) | keyboard | |
| LHS | Left-hand side of a binary operator (In '2+3', 2 is the LHS of the binary operator +). | |
| PID | Process ID. Each process on Unix-like systems gets a PID number when it starts. If there are multiple instances of a program running, they could each have separate PIDs. See kill for killing PIDs. | |
| PATH | 1.) Location of a file in a file system - can be described as: A.) full - e.g., /usr/bin/perl (Unix-like OS) or C:\dirname\file (Windows), or B.) relative - e.g., ../dirname/file (Unix-like OS) or ..\dirname\file (Windows). 2.) An environment variable where executable specifying a set of directories to check for executable programs. | |
| PWD (or pwd) | Present working directory (Unix-like OS - logical PWD; '.' is the PWD). Compare to: CWD The period is often the notation for PWD; double-period (i.e., '..') is the notation for parent dir of PWD. | |
| RHS | Right-hand side of a binary operator (In '2+3', 3 is the RHS of the binary operator +). | |
| STDERR | Standard error - typically terminal screen (Compare to: STDIN & STDOUT). STDERR has file desciptor 2. See > (write to) for examples of how to redirect error messages to a file (good for cron jobs or screen sessions). | |
| STDIN | Standard input - typically KBD (Compare to: STDERR & STDOUT). STDIN has file desciptor 0. | |
| STDOUT | Standard output - typically terminal screen (Compare to: STDERR & STDIN). STDOUT has file desciptor 1. | |
| Common Linux Commands & Utilities: | |
|---|---|
| Command (depending on system, can be case-senstive): | Meaning: |
| Note: Not all options/switches/arguments exist for every version (e.g., GNU, BSD, POSIX, etc.) | |
| . | PWD (the current directory, which is always quite relative). |
| find . | grep -i cosc | Locate all files in the PWD and subdirs & grep those that contain 'cosc' in filename |
| .. | The parent directory of PWD (parent of the current directory, which is always quite relative). |
| find .. | grep -i group | Start in parent dir of PWD and all subdirs & list all files & then grep those that contain 'cosc' in filename |
| # | Comment in some shells (and a few languages including Perl). |
| # | Comment character - everything after to end of line is comment. |
| | | Send the output of the LHS of | as the input to the RHS of |. |
| find . | xargs grep hello | Find the name of all files in PWD and subdirs, then search through each file for the string (In essence, grepping all files for string 'hello'). |
| ls -l /home/ | grep cosc | wc -l | list all home folders in home directory and then search that list for folders with 'cosc' in the name (these are group accounts for the department classes) and then of that list of group accounts, count how many there are (wc -l). |
| > | Write LHS to RHS. Warning: overwrites existing content of RHS. |
| echo hello > msg.txt | Echo the string 'hello' and write it to the file 'msg.txt'. |
| ls -l /home/ | grep cosc > list_group_accounts.txt | list all home folders in home directory and then search that list for folders with 'cosc' in the name (these are group accounts for the department classes) & write that list to list_group_accts.txt. |
| /usr/bin/env perl myprog.pl 2> ~/errors.log | Run myprog.pl and redirect error messages to the file in the home dir 'errors.log' |
| >> | Append LHS to RHS. |
| echo hello >> msg.txt | Echo the string 'hello' and append it to the end of the file 'msg.txt'. |
| <Ctrl> + C | Kill a process with signal SIGINT. |
| <Ctrl> + C | Kills a process that is running in the shell (e.g., a script with an infinite loop). |
| <Ctrl> + D | Ends text input for many Unix-like utilities by sending the EOF. |
| <Ctrl> + D | In programs like sendmail, you must end it and send it by sending the EOF on it's own line (and pressing <Enter>). |
| <Ctrl> + H | Delete last character typed in the terminal. |
| <Ctrl> + H | Delete last character typed in the terminal. |
| <Ctrl> + L | Clear terminal screen. |
| <Ctrl> + L | Clear the terminal screen. This is a wise habit to develop before running code particularly when debugging with 2 people. This makes it easier for the 2nd person to see only what's most-recently important. |
| <Ctrl> + R | reverse-i-search (find a previous shell command). |
| <Ctrl> + R | After pressing this, shell can respond with 'reverse-i-search & awaits input. As you type a command, it attempts to resolve it to previous commands. Very useful for avoiding retyping long strings at the shell all the time. |
| <Ctrl> + U | Delete last line typed in the terminal. |
| <Ctrl> + U | Delete last line typed in the terminal. |
| <Ctrl> + W | Delete last word typed in the terminal. |
| <Ctrl> + W | Delete last word typed in the terminal. |
| <Ctrl> + Z | Suspend a process with signal SIGSTOP. |
| <Ctrl> + Z | Suspends a process Note: You will get you're shell back, but process is gone. It is good to learn how to find & kill it later. |
| adduser | Add user to the system (See also:useradd) |
| adduser username | Add user to system. |
| ash | Almquist shell (Debian's dash is based on ash). |
| ash | Open a new ash shell. |
| awk | AWK is a tool for text processing, data extraction, reporting tool. |
| See compound commands above. | GNU AWK ref site (here) |
| bash | Bourne-again shell (replacement for the sh Bourne shell). |
| bash | Open a new bash shell. |
| cal | Calendar that prints given month or year. |
| cal | print current month |
| cal -m 2 | prints a calendar for February of current year. |
| cal -y 2016 | prints a 12-month calendar for 2016. |
| cat | view file contents - concatentate (or catenate) |
| cat filename | print file contents on STDOUT (from concatentate file contents to STDOUT). |
| cat -n filename | display line numbers before each line. |
| cat -b filename | display line numbers before each non-blank line. |
| cd | change directory Note: The root directory in Unix-like systems (i.e., '/') is equivalent to C:\ in Windows. |
| cd | change directory to home dir ( '~/', which is relative to user). |
| cd /dir | change directory to 'dir' which is a subfolder of '/' |
| cd ~ | change directory to home dir ( '~/', which is relative to user). |
| cd ~/ | change directory to home dir ( '~/', which is relative to user). |
| cd ~/dirname | change directory dirname folder that is subdir of home directory ( '~/', which is relative to user). |
| cd .. | change directory to parent of PWD (up one dir). |
| cd ../.. | change directory to parent of parent of PWD (up two dirs). |
| cd dir1 | change directory to dir1 (a subfolder of PWD). |
| cd ./dir1 | change directory to dir1 (a subfolder of PWD). |
| cd dir1/dir2 | change directory to dir2 (subfolder of dir1 - in PWD). |
| cd .. | change directory to parent of PWD. |
| cd .. | change directory to parent of PWD. |
| cd - | change last directory that user was in. Note: repetitious use of this command toggles back-and-forth between 2 dirs. |
| cp | Copy files. Warning: When copying files, one should assume that the destination will be overwritten. |
| cp file1 file2 | Make a copy of file1 and call it file2. Warning: If file2 exists already, it's contents will be overwritten. See next example for a safer way. |
| cp -i file1 file2 | Make a copy of file1 and call it file2, but ask for confirmation if a file2 exists. Warning: Does not work in conjunction with -n |
| cp -n file1 file2 | Make a copy of file1 and call it file2, but only if file2 doesn't exist. Warning: Does not work in conjunction with -i |
| TODO cp | TODO: create more example for cp |
| chage | Change user password expiry information. |
| chage -d 0 {user-name} | Force user to change password upon first login. (Set number of days since 1/1/1970 when password was last changed. |
| chmod | Change modifiers (access permissions/privileges) on a file. Note: there are many more examples beyond these - look them up. |
| chmod u+r filename | add read permission to filename's group privilege. |
| chmod g-x filename | remove executable privileges for filename's group. |
| chmod go-rwx filename | remove all privileges for filename's other and primary group. |
| chmod 777 filename | turn on all permissions (for everyone). |
| chmod 544 filename | Set filename's permissions to read+write for primary owner and read for primary group & other. |
| clear | Clear terminal screen. |
| crontab | Maintain crontab files for individual users. Warning: crontab -e is considered safer. |
| crontab -e | remove the current crontab. |
| crontab -l | Display current crontab on STDOUT. |
| crontab -r | remove current crontab. |
| crontab -u USER | Display crontab for current user. |
| crontab -l > file #Edit file crontab file | list crontab edit file load crontab with contents of file. |
| csh | The C shell. TODO |
| csh | Open a new C shell. |
| dash | Debian's Almquist shell (based on ash). |
| dash | Open a new dash shell. |
| date | Print system date on STDOUT or set it. |
| date | Print system date. |
| date -s (or --set) | Set date or time. |
| date -u (or --utc) | Display in UTC. |
| date +%M/%Y | Display 2-digit month (01-12) and 4-digit year. |
| date +"%Y-%m-%d %H:%M:%S" | Display current (locale) date & time per the specified format. |
| date -u +"%Y-%m-%d %H:%M:%S" | Display aforementioned converted to UTC. |
| dig | DNS lookup utility |
| dig tmrc.mit.edu | Passing a domain name returns A record by default. |
| dig tmrc.mit.edu a +short | View short output - only IP. |
| dig tmrc.mit.edu +short | View IP addr for given domain (e.g, returns 18.150.1.81). |
| dig -x 18.150.1.81 [+short] | View domain for given IP (e.g., returns tmrc.mit.edu). |
| diff | Compare two or more files and display their differences on the terminal. |
| diff file1 file2 | Compare the differences between file1 and file2. Note: In the result, the differences are displayed by using > and < to show the two files... Since there are 2 files in comparison, could view diff as a boolean operator with an LHS & an RHS. In this sense, < is the LHS (i.e., file1) and > is the RHS (i.e., file2). diff is not a boolean op, but you might hear this type of talk in conversation (purely improper slang). |
| emacs | TODO |
| echo | print/display specified string to terminal (e.g., STDOUT). |
| echo Hello World | prints 'Hello World' to STDOUT. |
| echo -n foo | print 'foo' without trailing newline character. |
| echo -e | Enable backslash escape sequence interpretation (e.g, \n, \t, \r, \\). |
| egrep | Extended Global Regular Expression search and
Print Note: egrep is very powerful and useful. Note: Can be used with find. TODO: Point out grep and alternatives. |
| egrep hello hello.pl | Search for the lines in hello.pl containing the regular expression 'hello' and print to terminal. |
| egrep 'print|exit' hello.pl | Match the pattern 'print' or 'exit' in file hello.pl (compare to next option) |
| egrep -w 'print|exit' hello.pl | Match the word 'print' or 'exit' in file hello.pl (compare to previous option) |
| env | Set and print environment. |
| env | Display environment info. |
| env -i /bin/sh | Clear environment for a new shell (without existing environment vars). |
| exit | Terminate a shell or session, sometimes a utility (though some utils use 'quit'). |
| find | Search for files in a directory hierarchy. |
| find . | Search the PWD and subdirs (and return all file names). |
| find . -name "*.pdf" -type f | Find pdf files in pwd (and recursively down). |
| find -maxdepth 2 | grep cosc | Search only 2 subdirs deep for all files and print those with 'cosc' in file name. |
| find TODO | TODO: create more find examples. |
| grep | Global Regular Expression search and Print Note: Grep is very powerful and useful. Note: Can be used with find. TODO: Point out egrep and alternatives. |
| grep hello hello.pl | Search for the lines in hello.pl containing the regular expression 'hello' and print to terminal. |
| grep 'print\|exit' hello.pl | Match the pattern 'print' or 'exit' in file hello.pl (compare to next option) |
| grep -w 'print\|exit' hello.pl | Match the word 'print' or 'exit' in file hello.pl (compare to previous option) |
| head | Display the beginning lines of text file or piped data on STDOUT (Compare to tail). |
| head filename | Display first 10 lines of file or piped data on STDOUT. |
| head -n20 filename | Display first 20 lines of file or piped data on STDOUT |
| history | command line history |
| history -c | Clear the history list. |
| history -n | Append the history lines not already from the history file to the current list. These lines are appended to the history file since the beginning of the current session. |
| id | Print real & effective user & group IDs. |
| id USER | print real & effective user & group IDs for USER. |
| kill | Kill a process (program, utility, etc.) |
| kill -9 187 | Kill process with PID 187. |
| TODO kill | TODO: create more kill examples. |
| less | View file. Compare to cat & more. |
| less filename | TODO |
| ls | List directory contents (of PWD, if path specified).. |
| ls -l | Long listing (details about each file). |
| TODO | TODO: need more ls examples. |
| lsof | Lists on STDOUT file info about files opened by processes. |
| lsof | List all open files belonging to all active processes. |
| lsof -p s | This option excludes or selects the listing of files for the processes whose optional process IDentification (PID) numbers are in the comma-separated set s - e.g., ''123'' or ''123,^456''. (There should be no spaces in the set.) |
| lsof -u s | This option selects the listing of files for the user whose login names or user ID numbers are in the comma-separated set s - e.g., ''abe'', or ''548,root''. (There should be no spaces in the set.) |
| man | (operation) manual system facilities (utilities, etc.). |
| man man | Read the manual pages on the man page. |
| man echo | Read man pages on how to use echo. |
| man awk | Read man pages on how to use extraction & reporting tool AWK. |
| man sed | Read man pages on how to use the stream editor sed. |
| man gcc | Read man pages on how to use the GNU C Compiler. |
| md5 | Calculate a message-digest fingerprint (checksum) for a file. Compare to: shasum (& others). |
| md5 TODO | TODO |
| mkdir | Make directories. |
| mkdir dir42 | Create a directory in the PWD with name 'dir42'. |
| mkdir ../dir42 | Create a directory in the parent dir of PWD with name 'dir42' (this new dir will then be in the parent dir as PWD). |
| mkdir ~/dir42 | Create a directory in user's home directory called 'dir42'. |
| mkdir /folder1/folder2/dir42 | Create a directory in /folder1/folder2/ called 'dir42'. Note: If the full path doesn't exist, see next example. |
| mkdir -p /folder1/folder2/dir42 | Create a directory in /folder1/folder2/ called 'dir42' AND if folder1 and folder2 don't exist, create them first too. |
| more | View file. Compare to cat & less. |
| more filename | TODO |
| mv | Move (or rename) files. Warning: When moving (or renaming) files, one should assume that the destination will be overwritten. |
| mv file1 file2 | In essence, rename the file file1 to file2. To be safer, use next examples (e.g., if file2 might already exist). |
| mv -i file1 file2 | In essence, rename the file file1 to file2, only after confirmation. |
| mv -n file1 file2 | In essence, rename the file file1 to file2, only if file2 doesn't already exist. |
| nslookup | Name Service Lookup for obtaining a domain name or IP addr mapping. |
| nslookup bls.gov | Get IP addr for bls.gov |
| nslookup 146.142.4.73 | Get domain name for 146.142.4.73 |
| passwd | Change a user's password. |
| passwd | change password for current user. Note: Does not echo typing. |
| passwd -e user | Expire the password of the named account (user). |
| passwd -l user | Lock the password of the named account (user). |
| passwd -u user | Unlock the password of the named account (user). |
| passwd -q | Perform action in quiet mode. |
| passwd -d user | Delete password for user (Quick way to disable a user). |
| passwd -S user | Display password status information from a user's account. |
| ps | Report a snapshot of the CURRENT processes. |
| ps | Report a snapshot of the CURRENT processes. |
| ps -u user42 | Report current processes for user42. |
| pwd | Present working directory (Unix-like OS) |
| pwd | |
| pwd -L | display logical pwd. |
| pwd -P | display physical pwd. |
| pwdx | Report current working directory of a process. |
| pwdx pids... | Report current working directory of a process. |
| perl | Run the Perl interpreter. |
| perl script | Invoke the Perl interpreter and run 'script' on it. |
| perl -e 'print "Hello World!\n"' | Run the one-liner that prints "Hello World!\n". |
| perl -p -i.bak -e 's/foo/bar/g' *.txt | Replace all occurances of 'foo' with 'bar' in all text files (of PWD) & keep backups. |
| ping | Send ICMP/ICMP6 ECHO_REQUEST packets to network hosts. |
| ping -c 4 164.76.250.120 | Send 4 packets to that address. |
| rm | Remove files or directories. Warning: When removing items, one should assume there will be no confirmation of removal. In other words, once you press |
| rm student_list.txt | Remove file 'student_list.txt'. |
| rm dir42 | Cannot do typically; will get response: rm: cannot remove dir42: Is a directory Instead try 'rm -R dir42' (next example). |
| rm -R dir42 | Recursively remove all contents of dir42 and then remove dir42 itself. |
| rm -i very_important.txt | Useful: Confirm removal of file before relegating it to the netherworld in perpetuity. |
| rmdir | Remove empty directories (if you need to remove non-empty directory, see rm -R |
| rmdir dir_containing_files | Result: failed to remove 'dir_containing_files': Directory is not empty. |
| rmdir empty_dir | Remove the empty directory 'empty_dir'. |
| rsync | Remote update protocol |
| rsync -r dir dir2 | recursive (necessary for directory syncing. |
| rsync -avn dir dir2 | Dry run and verbose. |
| rsync -a dir/ dir2 | Copy contents of dir into dir2 |
| rsync -a dir dir2 | Copy dir and its contents into dir2 |
| rsync -a ~/dir username@host:dest | Push from local to remote host (requires ssh). |
| rsync -a user@host:/path/to/src/dir /local/path/to/dest | Pull from remote host to local (requires ssh). |
| Note: Dry run with -n: rsync -anv dir dircopy Note: There is a difference between the commands 'rsync dir dir2' and 'rsync dir/ dir2' (the former copies the folder dir into dir2; the latter copies just the contents of dir to dir2). | |
| sed | sed - the stream editor |
| See compound commands above. | GNU sed ref site (here) |
| shasum | Print or check SHA Checksums. Compare to: md5 (& others). |
| shasum TODO | TODO |
| shutdown | shutdown system. |
| shutdown -h now | Shutdown system and turn power off immediately. |
| shutdown -h +5 | Shutdown system in 5 min. |
| shutdown -r now | Shutdown now and reboot. |
| shutdown -Fr now | Shutdown now and reboot, and force filesystem check on reboot. |
| shutdown -k | Kick all users off system (and disable logins for all users except root). |
| sort | TODO |
| sort filename | Sort in ascending order |
| sort -r filename | Sort in descending order |
| sort -t: -k 3n filename | less | Sort by third field and pipe to less |
| stat | Display file status |
| stat filename | Display file info - permissions, MAC times, & more. |
| tail | Display the tail end of a text file or piped data on STDOUT (Compare to head). |
| tail filename | Display last 10 lines of filename. |
| tail -n2 filename | Display last 2 lines of filename. |
| tail -n +2 filename | Display lines 2 through last line of filename (i.e., line 2 to the EOF). |
| tail -n1 filename* | Display last line of all files with name beginning with 'filename'. Note: For multiple files, this prints also the filename before respective content. If just content without the file names is desired, use the '--silent' option (below). |
| tail --silent -n1 filename* | Display last line of all files with name beginning with 'filename'. Compare to: tail -n1 filename* |
| tcsh | tc shell (essentially the C shell with cmd-line completion & other added features. |
| tcsh | open a new instance of tc shell. |
| top | display and update sorted information about processes |
| top | display and update sorted information about processes |
| top -u username | Show processes of username |
| touch | Change file access and mod times. Note: Can be a quick way to create an empty file. |
| touch filename | Change file access & mod times for filename. |
| uniq | Print to STDOUT data with duplicates removed (items, lines, etc.) |
| uniq filename | print only unique (non-repeating) input lines. |
| uniq -i filename | ignore case |
| uniq -d filename | Print only the repeating (non-unique) input lines. |
| uniq -c filename | Print only unique input lines with the duplicate frequency prepended (ignores -u & -d). |
| uniq -u filename | Print only the unique (non-repeating) input lines. |
| useradd | Add user to the system (See also:adduser) |
| useradd username | Add user to system. |
| useradd -s /sbin/nologin (or /usr/sbin/nologin) username | Add user to system with no shell to login. Note: Make sure you understand how to use nologin! |
| useradd -s /bin/false username | Add user to system with no shell to login. Note: Make sure you understand how to use false! |
| Note: For more info on 'false' and 'nologin', start here. | |
| xargs | Command used, among other ways, to aid in piping output data from one command as input data to another command. |
| find . | xargs grep cosc | Means to find all files in PWD and subdirs and then grep each file for lines containing 'cosc'. Compare to: 'find . | grep cosc', (which is making a list of files & returning those files containing 'cosc' in the file name). |
| w | Show who is logged in and what they are doing. |
| w | Show who is logged in and what they are doing. |
| wc | word count (and similar stats about a file). Note: Shell will display the result and filename, so how to get just the result? Answer: wc -l filename | awk '{ print $1 }' |
| wc -l filename | line count of input file printed on STDOUT. |
| wc -b filename | number of bytes of input file printed on STDOUT. |
| wc -m filename | character count of input file printed on STDOUT. |
| wc -w filename | word count of input file printed on STDOUT. |
| whereis | Identify path to an executable application. |
| whereis application | where is this executable app found? Compare to: which |
| which | Identify path to an executable application. |
| which application | where is this executable app found? Compare to: whereis |
| who | Show who is logged on. Be careful: read up - might not give you exactly what you want. |
| who | who is logged in. There are more options - read the man pages. |
Conversions useful for Unix-time calculations: 1 hour= 3600 seconds 1 day= 86400 seconds 1 week= 604800 seconds 1 month (30.44 days)=2629743 seconds 1 year (365.24 days)=31556926 seconds