Georgi’s Blog

Rsync + ssh

I reserve the right to change the howto and I’ll not be held responsible if ANY
damage is caused by this how-to. If you don’t agree please do not use it! Thanks

The idea of this how-to is to help you deal with this situation:

Imagine you have server with let’s say 10 users. Your server is getting too old
and you want to upgrade it. A good system administrator will not just shut down
the old server work on the new one for few days and then bring it up. A better
solution would be to leave the old server runnin, build the new one, configure
the new one, copy user files from old one to new one and for few seconds just
switch the IPs of both servers.

Now the problem comes when users change their files after you have transfered
them to the new server but before you have switched the IPs.

In this howto I’ll give you set of commands to run and set a cron job that will
update your /home for example every hour without causing high trafic on the
network. Only the first time!

What you need is:
ssh(d) on both servers
rsync on both servers
OpenSSL on both servers

Let’s give some names to our servers:
New server = thunder(192.168.0.101)
Old server = light(102.168.0.100)

we have a bunch of directories in /home/ on “light” that we want to transfer to
/home/ on “thunder”

here is what we do:

SSH no password part:
login to thunder as root
execute:
thunder#mkdir -p /root/.ssh
thunder#chmod 0700 /root/.ssh
thunder#ssh-keygen -t dsa -f /root/.ssh/id_dsa -P ” (two single quotes)

now you will have two new files:
/root/.ssh/id_dsa(your private key)
/root/.ssh/id_dsa.pub(your public key)

login to light as root
execute:
light#scp 192.168.0.101:.ssh/id_dsa.pub .
light#cat id_dsa.pub >> /root/.ssh/authorized_keys2
light#chmod 0600 /root/.ssh/authorized_keys2
NOTE: you might have to do the same for /root/.ssh/authorized_keys

now try to login from thunder to light with the key:
from thunder execute:
thunder#ssh -i /root/.ssh/id_dsa 192.168.0.100
you should login to light without entering password

!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!
if someone gains root access on thunder he will be able to ssh to light without
entering any password if he figures out that you have a key!
!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!WARNING!

if everything is OK let’s test it trough rsync:

create a directory on light and touch some files:
light#mkdir /home/testdir
light#cd /home/testdir
light#touch a
light#touch b

from thunder execute:
thunder#rsync -azx –delete –numeric-ids -e “ssh -i /root/.ssh/id_dsa” \
192.168.0.100:/home/testdir/ /home/testdir/

you should now see /home/testdir on thunder with a and b in it

now move a to c on light:
light#mv /home/testdir/a /home/testdir/c

execute the same rsync command on thunder:
thunder#rsync -azx –delete –numeric-ids -e “ssh -i /root/.ssh/id_dsa” \
192.168.0.100:/home/testdir/ /home/testdir/

now on thunder you should have /home/testdir/b and /home/testdir/c

the ownership will be preserved while the uid and gid on both thunder and light
match(and they should, but that other how-to

now go to /etc/cron.hourly on thunder and add the rsync line:
thunder# cd /etc/cron.hourly
thunder# echo “rsync -azx –delete –numeric-ids -e “ssh -i /root/.ssh/id_dsa” \
192.168.0.100:/home/ /home/” > rsync_home_update

And that’s it! Every hour thunder will sync its /home/ with /home/ on light
remember that both directories will be identical, that means if you have new
users on thunder that are not on light, their home directoreis will be deleted!
to avoid that, create /home/oldusers/ and sync /home/ on light to /home/oldusers/
on thunder. Don’t forget to move it later when you switch the servers or change
/etc/passwd on thunder to point to the correct home directories!

Good Luck

Software RAID + LILO

This how-to is made to help you configure your software raid and boot your root from it.

I’m doing this on slackware 10 with updated udev and mdadm, kernel 2.6.11-rc4

first you need to make sure that your raid support is IN your kernel, not as modules.
here is my config:
/usr/src/linux# grep -i raid .config
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# Multi-device support (RAID and LVM)
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
# CONFIG_MD_RAID10 is not set
# CONFIG_MD_RAID5 is not set
# CONFIG_MD_RAID6 is not set

I have a small /dev/hda1 partition that I used to install the system and configure the latest kernel.

here is how my two drives are partitioned:
~# fdisk -l /dev/hda

Disk /dev/hda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 255 2048256 83 Linux
/dev/hda2 256 498 1951897+ 82 Linux swap
/dev/hda3 499 3537 24410767+ fd Linux raid autodetect
/dev/hda4 3538 19457 127877400 fd Linux raid autodetect

/dev/hdc is the same.

so, I have /dev/hda1 as a boot partition
/dev/hdc1 as a temp root partition
/dev/hd[ac]2 for swap
/dev/hd[ac]3 for /
/dev/hd[ac]4 for /home

when you create your partitions use FD not 83 for partition type if you want your raid to be detected at boot later…

now just execute the following:
mdadm –create /dev/md0 –chunk=32 –level=1 –raid-devices 2 /dev/hd[ac]3

this will create a raid1 device for / on /dev/md0

mdadm –create /dev/md1 –chunk=32 –level=1 –raid-devices 2 /dev/hd[ac]4

and finally a /dev/md1 for the /home partition under raid1 again

Now you can observe /proc/mdstat. You’ll have something like:

~# cat /proc/mdstat

md1 : active raid1 hdc4[1] hda4[0]
127877312 blocks [2/2] [UU]
resync=DELAYED
md0 : active raid1 hdc3[1] hda3[0]
24410688 blocks [2/2] [UU]
[================>....] resync = 82.5% (20147072/24410688) finish=1.5min speed=46944K/sec

…

if you have problems with /dev/md* just use –auto after the –create instead of /dev/mdX
and mdadm will create md1 and md2 for your two raids (it will number them by the order of
execution of the mdadm –create command)

next:
mkreiserfs (or whatever fs you prefer) /dev/md0 (or md1 if you have used –auto)

our next task is to move the current installation on the RAID:

mkdir /mnt/newroot
mount /dev/md0(1) /mnt/newroot
cd /

now I use tar to move all my files, that way I’m sure it will preserve my links and perms.

in bash:
(cd /;tar cpf - ./bin ./dev ./proc ./sbin ./tmp ./var ./etc ./lib ./opt ./root ./usr ./sys)|(cd /mnt/newroot;tar xvpf -)

and watch

you might see some errors in /sys/ but that’s OK
now create home and boot:
mkdir /mnt/newroot/home
mkdir /mnt/newroot/boot

if you have more directories just put them in the list above. I had to use a list because I don’t want to move /boot
(separate partition) and /mnt

create /mnt/newroot/mnt/oldroot
now edit your /mnt/newroot/etc/fstab. Here is mine:
/dev/hda2 swap swap defaults 0 0
/dev/hdc2 swap swap defaults 0 0
/dev/md0 / reiserfs defaults 1 1
/dev/md1 /home reiserfs defaults 1 2
/dev/hda1 /boot ext2 defaults 1 2
/dev/hdc1 /mnt/oldroot reiserfs defaults 1 2
/dev/cdrom /mnt/cdrom iso9660 noauto,owner,ro 0 0
/dev/fd0 /mnt/floppy auto noauto,owner 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
proc /proc proc defaults 0 0

this should do the work.

and the last one:
vim /etc/lilo.conf

if you have something like:
image = /boot/bzImage-2.6.11
root = /dev/hdc1
label = Linux2.6
read-only

add another block:
image = /boot/bzImage-2.6.11
root = /dev/md0
label = Linux2.6RAID
read-only

run “lilo”
in this case your kernel will be the same. I assume you are doing the above work under your new kernel.
That means you have raid support

now reboot and choose Linux2.6RAID

Good Luck!

RTFM for Slackware (BG)

< <<<< Tezi komandi sa za Slackware moje da ima razminavane v drugite OS>>>>>
apropos xxx - pokazva spisuk ot komandite v koito ima xxx
w - koi e lognat v momenta v sistemata
last - slisuk na poslednite logvali se v sistemata useri
lastb - slisuk na poslednite neuspeshno logvali se v sistemata useri
uptime - kolko vreme e rabotila sistemata sled posledniq reboot i oshte
nqkolko polezni informaciiki
ps - spisuk sus raboteshtite procesi
ps -aux - po-razbiraem spisuk s procesite
killall xxx - ubivane na procesite imashti vruzka s xxx
kill -9 xxx - ubivane na edin proces kudeto xxx e negoviq pid nomer
cat /etc/issue - pokazva distribuciqta
cat /etc/version - pokazva versiqta
cat file - pokazva sudurjanieto na “file”
lsmod - pokazva spisuk sus zaredenite moduli
ls - spisuk s direktoriite i failovete v tekushtata direktoriq
ls -all - podroben spisuk ——————- || —————
cp source destination - kopira “source” v “destination”
mv ———- || ——– - premestva ————- || ———-
ln -s ———— || —— - pravi simvolichna vruzka
mkdir xxx - pravi papka na ime xxx
rmdir xxx - iztriva papka na ime xxx
rm xxx - trie file na ime xxx
rm -rf xxx - trie file na ime xxx s razlika ot gore che tuk ne te pita i
go trie vednaga… ako e direktoriq trie vsichko v neq primer rm -fr / -
tova bi iztrilo celiq hard disk…
addusr - dobavq potrebitel (script s vuprosi)
tar - xzvf filename.tar.gz - razarhivirane na filename.tar.gz
tar - xvf filename.tar- razarhivirane na filename.tar
bunzip filename.bz2 - razarhivirane na filename.bz2
gunzip filename.gz - razarhivirane na filename gz
unzip filename.zip - razarhivirane na filename.zip
jobs - spisuk sus sprqnite i “background” procesite
fg xxx - produljavane na proces kudeto xxx e “job number”
lsof - pokazva otvorenite failove
lspci - pokazva informaciq za PCI kartite na mashinata
watch -n xx - povtarq komanda na vseki xx sekundi
modprove -l |more - pokazva modulite koito sa dostupni
modprobe modulename - zarejda avtomatichno modul s ime modulname
lsdev - informaciq za DMA, IRQ I\O
insmod module - zarejda modul s ime module
rmmod module - premahva modul s ime module
cd - change dir (sushtoto kato pod DOS)
passwd - smqna na parolata na tekushtiq user
ifconfig - nastroika i informaciq za mrejovite karti
ifconfig eth0 192.168.0.1 bcast 192.168.0.255 netmask 255.255.255.0 -
nastroiva interface eth0 sus ip 192.168.0.1, maska 255.255.255.0 i
broadcast adres 192.168.0.255
prekompilirane na qdro - komandi:
1. cd /usr/src/linux
2. make config
2. make menuconfig
2. make xconfig (pod X)
#izbira se edno ot gornite 3 za nastroika
3. make clean; make dep; make bzImage; make modules; make modules_install
4. cp arch/i386/boot/bzImage /boot/[ime na image-a]
5. mcedit /etc/lilo.cof - nastroika na bootmanager-a
6. lilo
7. reboot
chmod - smenq pravata
chown - smenq sobstvenik
chgrp - smenq grupata
chmod -u +w /temp/example - dava w (write) prava na sobstvennika na
/temp/example
chmod -u -w /temp/example - premahva w (write) prava na sobstvennika na
/temp/example
za gornite komandi:
-u - sobstvennik
-g - grupa
-o - ostanalite
-a - na vsichki
——————-
w - write prava
r - read prava
x - execute prava
gornite 3 opcii se slagat s + ili - v zavisimost ot todva dali sa davat ili otnemat suotvetnite prava za file (direktoriq)

RTFM for Slackware (EN)

< <<<< These commands are for Slackware. They might differ on different distributions>>>>>
pwd - full path of the current directory (where are you at in the system)
apropos xxx - list of commands that have xxx
w - who is logged in.
last - list of last logins and reboots
lastb - list of last bad logins
uptime - the uptime of the system after last reboot, load average and other info
ps - list of running processes
ps -aux - full list of processe - nice when kill is the next command
killall xxx - kills all processes xxx and all child processes, where xxx is name
kill -9 xxx - kills process xxx with -9 signal (almost always kills the process), where xxx is pid
cat /etc/issue - name of distribution(should be)
cat file - shows the content of “file”
lsmod - shows list of loaded modules
ls - like dir in windows - a list of directories and files in the current directory
ls -all - ——————- || ————— with more information
cp source destination - copies “source” to “destination”
mv ———- || ——– - moves ————- || ———-
ln -s ———— || —— - creates soft link destination that points to source
touch xxx - creates an empty file xxx
mkdir xxx - creates directory xxx
rmdir -r xxx - removes(delete) directory xxx
rm xxx - removes(delete) file xxx
rm -rf xxx - removes(delete) file or directory xxx and all subdirectories and files of xxx without asking
addusr - script for adding users to the system
tar - xzvf filename.tar.gz - extracts filename.tar.gz
tar - xvf filename.tar- extracts filename.tar
bunzip filename.bz2 - extracts filename.bz2
gunzip filename.gz - extracts filename gz
unzip filename.zip - extracts filename.zip
rar x filename.rar - extracts filename.rar
jobs - list of stopped and background processes
fg xxx - continues a process where xxx is a job number
lsof - list of open files
lspci - list of pci devices in the machine (video cards, sound cards, IDE and USB conotrollers etc.)
watch -n xx - executes a command every xx seconds and prints the output
modprove -l |more - list of available modules for loading
modprobe modulename - tries to load modulname auto.
lsdev - information about DMA, IRQ I\O
insmod module - loads module “module”
rmmod module - removes module “module”
cd - change dir (like in DOS)
passwd - use this to change your password
ifconfig - setup of network interfaces (ethernet, ppp, etc)
ifconfig eth0 192.168.0.1 bcast 192.168.0.255 netmask 255.255.255.0 -
will cetup interface eth0 with ip 192.168.0.1, mask 255.255.255.0 and
broadcast address 192.168.0.255

Recompilation of a 2.4 kernel:
1. cd /usr/src/linux
2. make config
2. make menuconfig
2. make xconfig (pod X)
#use one of the top tree commands
3. make clean; make dep; make bzImage; make modules; make modules_install
4. cp arch/i386/boot/bzImage /boot/[name of kernel image]
5. mcedit(or vim or pico or whatever editer you prefer) /etc/lilo.cof:
add these lines for every new kernel:

image = /boot/[name of kernel image]
root = /dev/hda6
label = somelabel
read-only

6. lilo
7. reboot

Recompilation of a 2.6 kenel:
1. cd /usr/src/linux
2. make config
2. make menuconfig
2. make xconfig (pod X)
#use one of the top tree commands
3. make clean; make; make modules_install
4. cp arch/i386/boot/bzImage /boot/[name of kernel image]
5. mcedit(or vim or pico or whatever editer you prefer) /etc/lilo.cof:
add these lines for every new kernel:

image = /boot/[name of kernel image]
root = /dev/hda6
label = somelabel
read-only

6. lilo
7. reboot

chmod - change of permissions of a file or directory
chown - change of owner of a file or directory
chgrp - change of group of a file or directory
chmod -u +w /temp/example - gives w (write) permission to the owner of /temp/example
chmod -u -w /temp/example - removes w (write) permission from the owner of /temp/example
here are the flags:
-u - owner
-g - group
-o - restoftheworld
-a - all of the above
——————-
w - write permission
r - read read permission
x - execute permission
the above 3 flags are used with + or - for adding or removing a particular permission