Friday, December 14, 2012

Really useful, Essential CentOS / UNIX Commands


I'll begin by saying that I am absolutely no UNIX expert. Having worked with Unix (specifically CentOS) systems for a few years - I've been Googling and learning lots of really essential and overall very useful commands. I've kept a spreadsheet for myself - but figured it may prove to be helpful and time-saving for someone else.

Finding a specific filename on the Unix filesystem in all Sub-Directories

  • Let's say you are currently inside /var/www/yoursite directory
  • You want to find a file called "barney.jpg" inside "yoursite" directory and all of its sub-directories
  • find . -name barney.jpg -type f

Executing a command on several Unix servers at the same time

  • In this example, imagine you have a cluster of 7 web-servers named webserver1 through 7
  • You want to execute the same command on all 7 machines without having to ssh individually to each machine and executing the same command line.
  • The command below will massh into 7 webservers, check the apc.ini file and output a grep search result of "apc.shm_size=256". The commands after -c can be anything, for example "w".
  • massh -r webserver[1-7] -c "cat /etc/php.d/apc.ini |grep apc.shm_size=256" -o

How to check the actual/real directory location inside Sym-linked Directory

  • Let's say that your /var/www/yoursite directory is a symbolic link to a release branch
  • Example: /var/www/yoursite/ => /var/www/releases/2012_release1/
  • To view the physical location inside the symbolic link directory you have to modify the "pwd" command slightly:
  • pwd -P

How to grep everything inside a directory including all of its sub-directories?

  • In this example you are inside a document root, such as /var/www/yoursite/
  • There are hundreds of .php files located in different directories
  • You want to find the exact file that references a specific keyword inside it
  • For example "whSysopRangeBans"
  • grep -r -i 'wgSysopRangeBans' ./

How to remove lines inside a file using VI (VIM Editor) that begin with a specific character

  • In this example you are editing a sample file called example.php
  • Imagine that every line in example.php starts with "X" without quotes
  • XBuffer do xyz
  • XRender zyg
  • For some reason you want to find and remove the lines that begin with the X character
  • The VIM command for this regular expression delete is:
  • :g/^x.*$/d
How to output top 10 requests in your Apache ERROR or ACCESS log

  • In this example you are checking your apache request log(s)
  • Want to find out what the top 10 requesting IP's are?
  • cat /var/log/httpd/yoursite.com-access-20120209.log | awk '{print $1}'| sort -n | uniq -c | sort -rn | head -n10 
  • Very helpful in quickly identifying malicious scrapers and other issues

How to migrate / move an entire site server to server or sync full sites / directories

  • In this example we have two different servers
  • You want to sync or move the files from one server to another; from source to destination
  • This command will rsync files that are updated or do not exist on the destination directory
  • So, if you want to synchronize both directories on two different servers, this command will help.
  • Below command assumes you are logged into the source server and you will be connecting/sending files to destination server; you will be syncing "uploads" directories:
  • rsync -auv -e ssh --progress /var/www/html/yoursitesource.org/upload/ username@destinationserverip:/var/www/html/destination/upload/

How to Tar Gzip an Entire Directory (create a tarball for entire directory and all sub-dirs)

  • In this example you are creating a backup.tgz file inside migration directory using all of the files and sub-directories from /yoursite.com/
  • tar -czvf /var/www/html/migration/backup.tgz /var/www/html/yoursite.com/

You have a large GZIP tar ball, how to download/move a large tar file server to server:

  • In this example you created a huge backup.tgz file on a server called serversource1
  • Now you want to move it to another server, serverdestination1
  • While logged in on serverdestination1 (new server), issue this command:
  • scp someusername@serversource1:/var/www/html/migration/backup.tgz  /var/www/html/destinationdirectory/

OK, How to "unzip" / un-tar my tar-gzip archive file?

  • You've successfully zipped and moved your large gzip file
  • The command below will unzip and re-create all of the files and sub-directories from the archive file
  • tar -xzf backup.tgz -C /var/www/html/destinationdirectory/


How to dump / create backup for your MySQL Database using mysqldump
  • For example, you would like to create a complete database backup file of your vBulletin forum or your Wordpress blog
  • mysqldump -h'yourmysqlserverhostname' -u'mysqlusername' -p'mysqlpassword' sourcedbname > backup.sql
  • Please note that -h parameter may be optional for some machines and you can remove -h'hostname' entirely from the above command

How to restore MySQL database from dump file

  • Earlier, you've created a backup.sql mysql dump file
  • Now you want to "restore" this backup file to a new clean database
  • Important note: You can't really restore a mysql dump file on top of existing database if the tables already exist, you will have to create a new clean database or "drop" the database entirely. Dropping the database will completely delete all of the tables and content!
  • mysql -h'mysqlhostname' -u'mysqlusername' -p'mysqlpassword' destination_databasename < backup.sql

How to merge / synchronize two file folders

  • Imagine that you have two slightly different images directories, images_new and images_old
  • You want to create just one directory that has all of the up-to-date files
  • You want to copy all of the content from images_old into images_new without overwriting anything that has not been updated or modified in any way according to timestamp, not filename.
  • rsync -va /var/www/html/images_old/ /var/www/html/images_new


How to find and delete a specific

  • In this example you are going to find a filename match for "backup.zip" and then delete it
  • $find . -type f -name "backup.zip" -exec rm -f {} \;

How to copy "cp" files from one directory to another, but only new files that do not exist already
  • In this example you are trying to copy files from images_old to images_new directories, but do not overwrite any files that exist already.
  • In essence you are merging two directories together without overwriting old files
  • sudo yes n | cp -i -R /mnt/images_old/* /var/www/sites/images_new/

How to create a symlink, How to create a fake directory alias for easy access
  • In this example you are creating a symbolic link between directories
  • Really usefull for SVN control also
  • In this example your "new_build" directory will replace the "live" directory
  • ln -s  /home/content//public_html/new_build /home/content/public_html/live