Backup module specification

Author: Ladislav Slezak < lslezak@suse.cz>

Changes:


Backup and system recovery solutions

Backup

Utilities like dump, kbackup and others are indended for full system backup. These utilities do not use rpm database to get changed files, but they support incremental backup or client/server architecture or other features.

YaST1

YaST1 backup module compares file flags stored in rpm database with actual file properties and backups only files which differ, the result is small backup archive.

How is backup done in YaST1:

  1. get list of installed packages
  2. get list of files and modification flags for each installed package
  3. select from list only modified files
  4. remove duplicate entries from file list
  5. find files which do not belong to any package (on demand): iterate through directory content (start at root directory), for each:
  6. display list of files to backup, user choose files from list
  7. display archiving options
  8. run tar to create backup archive

YaST1 can not restore files from backup archive. Restoration of system is done by new system installation and manually updating files from backup archive. There is no way how to check whether installed packages are same as in backup time or if they are installed in same place (not relocated to another directory).

System recovery utilities

Make CDROM recovery makes a bootable recovery ISO images, with full backup of system. System can be booted from CD and complete recovered from backup. Others utilities only make bootable floppy/CD with useful utilities, for example rip or zdisk. Gpart utility can guess lost partition table from disk content, this is useful if no partition backup is available.

Documentation

YaST1 source, tar, fdisk, e2image and rpm manual pages, Partition rescue mini HOWTO, Maximum RPM - Verification, Bugzilla entry

Features

List of features is available here.

Implementation

Backup

Backup archive contains files which differ from files in RPM packages and files which do not belong to any RPM package.

Changed files which belong to one package are stored to one subarchive with name

Currently num is 0 - this value is for future enhacement (backup archive created by package update have similar structure, so it is possible to include already created backup files to archive and this number can be used to differ file names). File extension ext is one of '.tar.gz', '.tar.bz2', '.tar' or '.star.gz', '.star.bz2', 'star' - depends on selected archiver (tar or star).

Some additional information will be stored in subdirectory info in the archive. This directory should contain files:

Files not owned by any package

Files not owned by any package are stored in NOPACKAGE subcharive. It is posible to exclude directory or file system type from search. This allows to exclude from backup /tmp directory or directories with mounted NFS file system.

Default excluded directories are: /media, /windows, /tmp, /var/lock, /var/run, /var/tmp and /var/cache. Default excluded file system is iso9660, ntfs and file systems with "nodev" flag in /proc/filesystems file (e.g. nfs, proc, tmpfs...).

Access Control Lists (ACLs)

Backup module can store ACLs using star archiver - in this case subarchives (with *.star extension) have exustar format and they are created with -acl flag to store ACLs.

System backup

System backup will be stored in system subdirectory in the archive. This subdirectory can contain files:

All package archives, files in info and system subdirectories are archived to one big archive by tar utility.

It is possible to store only list of changed files to file - script backup_archive.pl outputs file names from packages_info file. This list of files can be used to create another archive type.

Autoinstallation profile

Backup module also creates autoinstallation XML profile. Profile file name is same as backup archive except extension which is ".xml".

Autoinstallation profile contains installation settings which can be used in automatic installation withou user interaction.

Profiles

Yast2 Backup module contains profile manager. After backup archive is created it is possible to store entered settings to profile with specified name.

When backup module find any stored profile after start it opens profile dialog. Action which can be selected:

Problems and solutions

Multi volume archive

Size of created archive can be larger than size of backup media. In this case archive file can be manually splitted to more files, but for beginners is more acceptable automatic splitting.

Tar archiver creates multiple volume archives if -M or -L is used. Each volume is standalone archive - files can be extracted without need of other volumes (except files which were splitted to more volumes).

Each volume is named according to pattern <volume_number>_<archive_name>. If user enters archive name /tmp/archive.tar then volume names are: /tmp/01_archive.tar, /tmp/02_archive.tar, ...

There is small problem with tar and multi volume archives: Volume size is multiply of block size, which is default 20 * 512 bytes = 10kiB. So if requested volume size is 32kiB (with parametr -L 32) created volume will have size 40kiB. There is option to set block size, but with parameter -b 1 (block size 1*512B) or -b 2 tar exits on SIGSEGV. With -b 4 it seems that it works, so default block size is set to 4*512 bytes. This was tested with tar version 1.13.18.

Solution is to create smaller volumes - if requested volume size is 35kiB create volumes with size 34kiB (with 2kiB blocks).

Architecture

Backup module has two parts: backup scripts and YaST2 frontend.

Backup scripts are written in Perl. First script (backup_search.pl) collects files to backup, second script (backup_archive.pl)creates archive file. Two scripts are required so that user can exclude some files from backup after search. These scripts are independent on YaST2 and they can be used in batch mode backup. In batch mode all parametres are entered on command line and backup is done without any interaction with user. This feature can be used in shell scripts to do backup automatically.

YaST2 frontend collects information from user, then it runs backup scripts and displays progress of backup.

Restore

YaST2 retore module will use some information about system stored in backup archive and will allow simple restoring of files. Restoring partition table or restoring ext2 critical areas are dangerous operations, therefore these operations should be done only manually by expert user. Restoring partition table is described in Partition rescue mini HOWTO - chapter 8.1 Ext2 data are stored in image obtained by e2image utility, see e2image man page.

More information about restoring can be found in Yast2 restore module documentation.

Workflow

Overview

This is backup workflow overview. (source)

Dialog details

(Outdated Qt designer workflow dialogs are available here.)

Archive options - Archive file name, achive type and optional archive description are entered in this dialog.

Archive file options - Multi volume archive options

Backup options - Number of selected parts to backup affects number of dialogs in workflow.

Exclude directory - Directories can be excluded from search.

Exclude file system - Filesystems can be excluded from search.

System backup - Partition table and ext2 partitions can be selected here.

Searching modified files - Progress bar is displayed while searching modified files.

Searching files - Progress of searching files is displayed here.

File selection - All found files are displayed here.

Creating archive - Progess is displayed while creating archive.

Backup summary - Results of backup procedure are displayed here.

Summary details - Detailed results are displayed here.