Bacula Tapes Bacula is an open-source, enterprise-level computer backup system for heterogeneous networks. Bacula is by far the most popular Open Source backup program. It is designed to automate backup tasks that had often required intervention from a systems administrator or computer operator. Bacula supports Linux, UNIX, Windows, and macOS backup clients, and a range of professional backup devices including tape libraries.

Today I will look at configuring the Storage & Pools Resources. These are part of the bacula-dir.conf file. I will also look at the bacula-sd.conf file, which is responsible for defining the Storage Daemon.

Throughout the posts I will shamelessly quote the main Bacula Documentation as I see no sense in duplicating the good work that has already been done by the Bacula team. However, I will also go into detail where necessary to point out parts that I feel need extra explanation, relative to the configurations I have made.

Storage Configuration

Like the bacula-dir.conf file, Bacula also maintains the bacula-sd.conf file to define a storage daemon. The Director connects to the Storage Daemon to allocate storage devices to jobs, that the Director runs.

bacula-sd.conf

# Define the Storage Daemon
Storage { 
  Name =                     bacula.pbdigital.org-sd
  SDPort =                   9103
  WorkingDirectory =         "/var/db/bacula"
  Pid Directory =            "/var/run"
  Maximum Concurrent Jobs =  20
}

# Directors who are permitted to contact the Storage daemon
Director {
  Name =      bacula.pbdigital.org-dir
  Password =  "secret"
}

# Devices supported by this Storage daemon
Device {
  Name =                     usb-storage
  Archive Device =           /media/bacula
  Device Type =              File
  Media Type =               USB
  Random Access =            yes
  Maximum Concurrent Jobs =  5
}

Device {
  Name =                     file-storage
  Archive Device =           /tmp
  Device Type =              File
  Media Type =               File
  Random Access =            yes
  Maximum Concurrent Jobs =  5
}

# Send all messages to the Director,
# mount messages also are sent to the email address

Messages {
  Name =      Standard
  director =  bacula.pbdigital.org-dir = all
}

Of note I will outline the following bacula-sd.conf resources & directives:

  • Storage Resource - In the above bacula-sd.conf file, the Storage Resource directives and arguments are similar to what is required in the Includes/director.conf file. These should not need further explanation, however if you do want to read more on the Storage Resource directive, see section [21.1] in the Bacula manual.

  • Director Resource - The Director resource specifies the Name of the Director which is permitted to use the services of the Storage daemon. The Director Name and Password must match the corresponding values in the Director’s configuration file. For more information on the Director Resource directives, see section [21.2] in the Bacula manual.

  • Device Resource - The Device Resource specifies the details of each device (in my case, a USB drive) that can be used by the Storage Daemon. There may be multiple Device resources for a single Storage daemon, I have also included a file-storage device, however this will not be used. Below, will only list the directives for the usb-storage resource.

    • Name - Specifies the Name that the Director will use when asking to backup or restore to or from to this device. It is generally a good idea to make it correspond to the English name of the backup device. The name you specify here is also used in the Include/storage.conf file.

    • Archive Device - The provided argument gives the system file name of the storage device managed by this storage daemon. It may also be a directory name if you are archiving to disk storage. In this case, you must supply the full absolute path to the directory. Here we use the absolute path to the mounted directory, /media/bacula, as we are writing to a Device Type of File, which I explain immediatley in the next section.

    • Device Type - The Device Type specification allows you to explicitly tell Bacula what kind of device you are defining. Of the 3 defined types; File, Tape and Fifo, USB Devices use the File type.

    • Media Type - The provided argument gives the type of media supported by this device,for example, ”USB”. Media type names are arbitrary in that you set them to anything you want, but they must be known to the volume database to keep track of which storage daemons can read which volumes.

    • Random Access - This should be set to Yes for all file systems such as USB, and fixed files. It should be set to No for non-random access devices such as tapes and named pipes.

    NOTE!: I am using autofs(5) to automatically mount & unmount the USB media. If you are not using this, the directives for the Device Resource would be very different. I document how autofs in configured when I cover working with Removable Media.

    For full details see the Device Resource section [21.3] in the Bacula manual.

  • Message Resource - The Message Resource defines which messages will be handled in relation to the storage device.

    • Name - This is set to Standard which we have discussed above in the Includes/messages.conf section.

    • director - Send the message to the Director whose name is given in the address field. Note, in the current implementation, the Director Name is ignored, and the message is sent to the Director that started the Job.

    For full details see The Message Resource section [22] in the Bacula manual.

Storage Resource

Note: We are no longer working with the bacula-sd.conf file and the following resources now pertain to the bacula-dir.conf file.

The Storage resource defines which Storage daemons are available for use by the Director. I am backing up to USB so this resource reflects this.

Includes/storage.conf

# Definition of file storage device
Storage {
  Name =                     bacula.pbdigital.org-sd-usb
  Address =                  bacula.pbdigital.org
  SDPort =                   9103
  Password =                 "secret"
  Device =                   usb-storage
  Media Type =               USB
  Maximum Concurrent Jobs =  10
}

Storage {
  Name =                     bacula.pbdigital.org-sd-file
  Address =                  bacula.pbdigital.org
  SDPort =                   9103
  Password =                 "secret"
  Device =                   file-storage
  Media Type =               File
  Maximum Concurrent Jobs =  10
}

Of note I will outline the following directives:

  • Name - The name of the storage resource. This name appears on the Storage directive specified in the Job resource and is required. This is not the name of the Storage Daemon.

  • Address - A fully qualified domain name, or an IP address of the Storage Daemon. Please note that the address as specified here will be transmitted to the File daemon who will then use it to contact the Storage daemon. Hence, it is not a good idea to use localhost as the name but rather a fully qualified machine name or an IP address.

  • SDport - This is a registered IANA port of the Storage Daemon. Unless you have a very specific use case, you should probably not change this.

  • Password - This is the password to be used when establishing a connection with the Storage Daemon. This same password also must appear in Director Resource of the bacula-sd.conf file.

  • Device - This directive specifies the Storage daemon’s name of the device resource to be used for the storage.

  • Media Type - This directive specifies the Media Type to be used to store the data. This is an arbitrary string of characters up to 127 maximum that you define. The MediaType specified in the Director’s Storage resource, must correspond to the Media Type specified in the Device resource of the Storage daemon configuration file.

For more details see the Storage Resource [19.14].

Pools Resource

The Pool resource defines the set of storage Volumes (tapes or files) to be used by Bacula to write the data. By configuring different Pools, you can determine which set of Volumes (media) receives the backup data. This permits, for example, to store all full backup data on one set of Volumes and all incremental backups on another set of Volumes.

I will be defining the following Pools; Default, DAILY, WEEKLY & MONTHLY. In my situation, the Default Pool is a special case and is never used except in two cases.

  • The first as a place holder in the jobs directives of the Includes/jobs.conf file, that are later overridden to one of the DAILY, WEEKLY or MONTHLY pools when the Includes/schedules.conf file starts running the jobs.

  • The second case, also as a placeholder, is in respect to the Restore Job of the Jobs Resource file. In this case it will be required to be overridden manually when you choose a job to restore.

Objectives & Requirements

The following Includes/pools.conf resource will allow me to meet my objective of having full backups for a trailing 12 month period. Objectives & requirements follow:

  • Monthly Full backup, with a retention period of just under 12 months, This will require a minimum number of 12 backup volumes.
  • Weekly Differential backup against the monthly full backup, with a retention period of just under 1 month. This will require a minimum number of 4 backup volumes.
  • Daily Incremental backup against the previous backup (whether it be full, differential or incremental), with a retention period of 2 weeks. This will require a minimum number of 12 backup volumes.

Includes/pools.conf

# Default pool definition
Pool {
  Name =       Default
  Pool Type =  Backup
}

Pool {
  Name =                  MONTHLY
  Pool Type =             Backup
  Maximum Volume Bytes =  30G
  Volume Use Duration =   27 days
  AutoPrune =             yes
  Volume Retention =      360 days 
  Recycle =               yes
}
 
Pool {
  Name =                  WEEKLY
  Pool Type =             Backup
  Maximum Volume Bytes =  15G
  Volume Use Duration =   6 days
  AutoPrune =             yes
  Volume Retention =      34 days
  Recycle =               yes
}
 
Pool {
  Name =                  DAILY
  Pool Type =             Backup
  Maximum Volume Bytes =  7G
  Volume Use Duration =   22 hours
  AutoPrune =             yes
  Volume Retention =      13 days
  Recycle =               yes
}

Of note I will outline the following directives:

  • Name - The name of the pool.

  • Pool Type - This directive defines the pool type, which corresponds to the type of Job being run. Only Backup is current implemented.

  • Maximum Volume Bytes - This directive specifies the maximum number of bytes that can be written to the Volume. If you specify zero (the default), there is no limit except the physical size of the Volume. Otherwise, when the number of bytes written to the Volume equals size the Volume will be marked Full. When the Volume is marked Full it can no longer be used for appending Jobs, but it can be recycled if recycling is enabled, and thus the Volume can be re-used after recycling. The size specified is checked just before each block is written to the Volume and if the Volume size would exceed the specified Maximum Volume Bytes the Full status will be set and the Job will request the next available Volume to continue.

    This directive is particularly useful for restricting the size of disk volumes, and will work correctly even in the case of multiple simultaneous jobs writing to the volume.

    In my case for the DAILY, WEEKLY & MONTHLY volumes I will be using 8gb, 16gb & 32gb media, respectively.

  • Volume Use Duration - The Volume Use Duration directive defines the time period that the Volume can be written beginning from the time of first data write to the Volume. If the time-period specified is zero (the default), the Volume can be written indefinitely. Otherwise, the next time a job runs that wants to access this Volume, and the time period from the first write to the volume (the first Job written) exceeds the time-period-specification, the Volume will be marked Used, which means that no more Jobs can be appended to the Volume, but it may be recycled if recycling is enabled.

    Setting the DAILY to 22 Hours, the WEEKLY to 6 Days & the MONTHLY to 27 days, means that the volume is not able to be used for the next iteration of the DAILY, WEEKLY or MONTHLY pool and Bacula will request a new volume. This is intended, as we want to have one backup for each volume, until we have cycled through all our available volumes.

  • AutoPrune - If AutoPrune is set to yes (default), Bacula will automatically apply the Volume Retention period when a new Volume is needed and no appendable Volumes exist in the Pool. Volume pruning causes expired Jobs (older than the Volume Retention period) to be deleted from the Catalog and permits possible recycling of the Volume.

  • Volume Retention - The Volume Retention directive defines the longest amount of time that Bacula will keep records associated with the Volume in the Catalog database after the End time of each Job written to the Volume. When this time period expires, and if AutoPrune is set to yes Bacula may prune (remove) Job records that are older than the specified Volume Retention period if it is necessary to free up a Volume.

    Setting the DAILY to 13 Days, the WEEKLY to 34 Days & the MONTHLY to 360 days, means that the volume will be pruned and then recycled before we have cycled through all of the volumes in the Pool cycle, making the volume available for the new Pool cycle to start.

  • Recycle - This directive specifies whether or not Purged Volumes may be recycled. If it is set to yes (default) and Bacula needs a volume but finds none that are appendable, it will search for and recycle (reuse) Purged Volumes (i.e. volumes with all the Jobs and Files expired and thus deleted from the Catalog). If the Volume is recycled, all previous data written to that Volume will be overwritten.

For more details see the Pools Resource [19.16].

Wrapping Up

Yup, still a long way to go before we can fire up Bacula, but the good news is we are halfway through configuring Bacula now. Next I will discuss defining Clients and the Filesets that they will have backed up to the Bacula server. You can jump to it here.