linux

useful commands: the basics

• • cd

    • "change directory"

  • cd ~

    • change directory to home

  • cd ..

    • go up a directory

  • rm name*

    • delete all files begining with name

  • rm *

    • delete all files in this folder

  • mkdir

    • create a new directory

  • df -h

    • • "disk free" how much space is there, how much is in use

      • -h for "human" readable

  • ls -hl

    • • list structure: show files in current directory

      • -l for list permissions

      • -h for "human" readable

  • pwd

    • • print working directory: where am I?

  • less %filename%

    • • a command line text reader

  • tail %filename%

    • view the last portion of the file. Then you don't load the entire file.

  • the [UP] key

    • cycle through previous commands

useful commands: advanced

• • CONTROL-R

    • history search

  • mmv

  • screen

  • find | grep

  • shell scripts with loops and variables

  • vi

    • • there are books written on the one command. Text editor, not wysiwyg.

  • emacs

    • • more books on this command. Text editor, not wysiwyg

  • gedit

    • • GUI text editor. Does NOT work from command line

  • dmesg

    • • system messages. Common usage: dmesg | tail

  • locate %file%

  • gparted

    • • GUI disk manager (partioning, formatting)

  • su

    • • elevate to root

  • sudo

    • • elevate following command to root

        • • sudo apt-get install %packagename%

  • whoami

  • tac: reverse the order of a file

  • sort

  • du -xak .|sort -n|tail -50

  • rsync -ave ssh username@127.0.0.1:/home/folder/home/tuesday/primary_storage/folder/

  • rsync -ave 'ssh -p 1022' /home/username/primary_storage/folder/ username@127.0.0.1:/home/username/folder/

  • rsync -r -c --progress --stats -a -v -e ssh user@remote.computer.edu:/folder/name/qu* .

  • smbget

  • http://forums.opensuse.org/network-internet/396597-copying-files-smb-linux-terminal.html

see also http://slashdot.org/comments.pl?sid=1019609

download apple trailers using wget by changing user agent

wget -U "QuickTime/7.6.2" http://movies.apple.com/movies/fox/avatar/avatar2009aug0820a-tsr_h1080p.mov

find . ! -type d -print0 | xargs -0 md5sum

tree (from http://www.centerkey.com/tree/)

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'

search for a file somewhere in the .tgz and extract that file

tar xvf some.tgz --no-anchored some.txt

ssh chat

https://news.ycombinator.com/item?id=9716459

Just use tail -f and a bash function.

Put this in your .bashrc file:

function talk { echo "$USER: $@" >> talkfile; }

then run:

tail -f talkfile &

The & puts tail into the background so it continues running, and "talkfile" needs to be a file that both of you have write access to.

You can both communicate simply by using the talk function like any other bash command:

talk whatever you want and it'll be written to talkfile

vector graphics

GIMP

povray

xfig

inkscape

searchable pdf

Suppose you have thousands of PDF that you would like to grep.

ls *pdf >> files_to_convert

the find and replace each file name X.pdf with pdftotext 'X.pdf'

chmod 777 files_to_convert

./files_to_convert

mmv '*.txt' '#1.plaintxt'

grep -l 'inverse participation' *.plaintxt

file deletion

http://www.techthrob.com/2009/03/02/howto-delete-files-permanently-and-securely-in-linux/

shred -u -z -n 2 filename

secure-delete package by default uses 32 passes:

sfill -v /folder/path

quicker single pass:

sfill -v -l -l /folder/path

turn off shell beeps

$ vi ~/.inputrc

set bell-style visible

This replaces the audible bell with a visible bell that flashes the screen once. To have no bell at all, replace "visible" with "none".

$ vi ~/.bashrc

export INPUTRC=~/.inputrc

and restart existing shells

Emergency: kernel panic

First, Ctrl + Alt + Backspace to restart X doesn't work.

If you've enabled sysrq by doing

$ sudo su

# echo 1 > /proc/sys/kernel/sysrq

see

http://ubuntuforums.org/showthread.php?t=617349

this can also be done over the network, if you've installed the network version

http://www.ubuntugeek.com/manage-linux-sysrq-over-network-using-sysrqd.html

mass rename

mmv '100_0*.jpg' 'skydive_20090508_#1.jpg'

performance measurement

iotop -o

hardinfo

http://ubuntuforums.org/showthread.php?t=177234

GUI image resize for right-click

sudo apt-get install nautilus-image-converter

mass image resize using convert

for i in `ls *.jpg`;

do

convert $i -resize 25% $i

done

for i in `ls *.jpg`;

do

convert $i -resize 800×600 $i

done

remote terminal

$ ssh %username%@%ipAddress%

• • securely copy file to remote location

    • • %username%@%ipAddress% is same as ssh use

      • %filelocation% would be something like /home/user/file.txt

        • • usually one would ssh into the remote system first to determine where the file is located

      • %destination% on local box. Use "." without quotes to say "here"

$ scp %username%@%ipAddress%:%filelocation% %destination%

Example:

scp -r folder_* user@<ip>:/nethome/users/user/pha/alpha\=1_0_M00/Nc4/

change file permission

$ chmod 755 %filename%

• • • change permission to executable

• • sh %filename%

    • • run executable shell scripts

  • ./%filename%

    • • run an executable. For example, a compiled fortran file

  • convert -delay 30 *.png -loop 0 playme.gif

• • • Note: time is in 100th of a second

    • Note: all the source pictures must be the same dimensions and the same file type

    • http://www.linux.com/articles/46124

recursively zips contents

$ zip -r output.zip folder_*

empty the trash

$ rm -rf ~/.local/share/Trash/*

delete all files matching wildcard below a given path

$ rm `find /nethome/users/username/ -name '*.gz'`

delete all files matching name below a given path

$ rm `find /nethome/users/username/ -name 'status.dat'`

Argument list too long ? Use

find . -type f -name 'out_*.dat' -exec /bin/rm {} \;

to run a program in the background and redirect text to a file in linux: One can run multiple programs in the background at once. Use "top" to see what's running. the "nohup" means no hangup. thus, you don't have to keep the putty window open for it to run.

nice nohup ./a.out > outputfile.txt &

to backup a directory with the date embedded in the file name,

$ tar -cvf /autobackups/home_20$(date +%y%m%d).tar /home

Once you're comfortable, you may find yourself repeating certain commands. These can be written as shell scripts, the linux equivalent to windows batch files

matlab is a valid command on the nic cluster. See also matlab tips.

Remote desktop is through VNC

vinagre

permissions on linux

• to view permission in a terminal (like putty in Windows), type the command

ls -al

• if you see that you don't have execute permissions (x) you'll need to type

chmod 755 %filename%

chmod calculator

More security:

locking down your station:

Bastille

isolates applications in linux: AppArmor (from Novell)

http://www.linux.com/feature/58789

Another (more comprehensive) approach is SELinux (IBM, HP, NSA)

file transfer

scp

http://compsoc.dur.ac.uk/~djw/tarpipe.html

On the receiving end do:

# netcat -l -p 7000 | tar x

And on the sending end do:

# tar cf - * | netcat otherhost 7000

apache web server tips

add this to /etc/apache2/httpd.conf

<Directory /var/www/tech>

AuthUserFile /home/<user>/wherever/.htpasswd

AuthName "Please Log In"

AuthType Basic

require valid-user

</Directory>

and then create a password

$ sudo htpasswd -c /home/<user>/wherever/.htpasswd webusername

restart the web server if necessary

$ sudo /etc/init.d/apache2 restart

viewing remote GUI on Windows

Normally when one uses putty to SSH from Windows to a linux box, the applications run from the SSH shell can not be rendered on the Windows computer.

Following the instructions on http://www.math.umn.edu/systems_guide/putty_xwin32.html I am now able to display remote linux programs, like matlab, on the local windows session.

Get one 2Mb program from http://www.straightrunning.com/XmingNotes/ for your windows computer: http://sourceforge.net/project/downloading.php?group_id=156984&filename=Xming-6-9-0-31-setup.exe

and install it. This is the local Windows Xming server. You should end up with an "X" icon by your clock.

Now open a putty session to any linux box, except under Connections > SSH > X11 put a check mark in "Enable X11 forwarding" and fill in "X display location" with localhost:0

Once the putty session is open, try running wireshark or firestarter. Usually there is no GUI available in Windows, but now you will have a GUI.

backup

Create a compressed copy of everything on the hard drive, except for certain folders. Also, be sure to exclude the file you are creating

$ sudo tar cvpzf backup.tgz --exclude=/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/mnt --exclude=/sys /

Restoring: make sure you and the backup file are in the root of the filesystem, then

$ sudo tar xvpfz backup.tgz -C /

secure SSH tunnels

ssh user@remotehost -L123:example.com:456

Open a tunnel on your local machine, port 123, to example.com, port 456, via the remote host

ssh -R lets you go in the opposite direction (tunnel from remote end to local end), but if your application supports SOCKS, it's even easier:

ssh user@remotehost -D8080

Creates a secure tunnel supporting the SOCKS protocol.

misc

:(){ :|:& };:

:() defines a function named : with no arguments. { :|:& } is what the function does. :|: calls itself twice (with a pipe between the two), and the & at the end runs it in the background as a new process. The ; finishes off that command, then the last : runs the function, starting the fork bomb (as each run starts 2 new processes, each of which starts 2 new processes...).

!! # run again the last command that was run !9 # run again the command with history number 9 !v # run again the last command that started with a 'v' !vi # run the last command that started with "vi" !?foo? # run the last command that had the string "foo" anywhere in it diff oldfile newfile mv !$ !^ # same as "mv newfile oldfile" # !$ is last arg of previous command, !^ is first arg ls foo bar baz rm -f !!* # same as "rm -f foo bar baz" # !!* repeats all arguments from previous command

grep for a word in a specific file type extension

find -type f -name '*.m' -print0 | xargs -0 grep -o correlation

grep -lR keyword

How to delete a given file extension recursively

rm `find /home/users/path -name '*.gz'`