Single Node Docker Backups

Audience: System Administrators

Content Summary: The Immuta Single Node Docker installation contains a backup script that creates a backup archive that can be easily used to restore an Immuta installation. The backup script is packaged inside of the database Docker container allowing it to be executed using the docker exec functionality. By default backups are stored in /var/lib/immuta/postgresql/backups.

Backup Volumes

Before enabling backups you will want to first add a volume to the database container in the docker-compose.yml file. This volume will be mounted into the container at /var/lib/immuta/postgresql/backups and used to store all backup archives. This volume may be backed by a host volume or any other docker supported volume.

Host Volume

If using a host volume, add a volume to the end of the volumes list.

  db:
   volumes:
    # ...Previously configured volumes...
    - <YOUR HOST PATH TO BACKUPS DIRECTORY>:/var/lib/immuta/postgresql/backups

mkdir -p ${IMMUTA_HOME}/backups
chown -R 1000:1000 ${IMMUTA_HOME}/backups

Restarting Database Container

After adding a backup volume, you must stop Immuta and remove the database container. Because the Postgres data is stored in an external volume, no data will be lost. Run the commands below to stop, remove, and restart the database container.

docker-compose stop db
docker-compose rm db
docker-compose up -d db

Executing Backups

After adding a backup volume, you can create a backup script that exec's into the database container and then clean up old backups. This script may then be used by Cron to perform the backups automatically or manually run periodically by an operator.

Example Backup Script

Below is an example backup script that may be used in a Cron job to backup the Immuta database.

#!/bin/bash

set -e

IMMUTA_HOME="/opt/immuta"
BACKUPS_DIR="${IMMUTA_HOME}/backups"

# Max number of backups to have. Any number of backups greater than or equal to this number will
# be removed. The number of backups that will exist will be BACKUP_THRESHOLD - 1.
#
# Example: if BACKUP_THRESHOLD=3, then only 2 backups will exist at a given time.
BACKUP_THRESHOLD=3

COMPOSE_INTERACTIVE_NO_CLI=1 /usr/local/bin/docker-compose --file "${IMMUTA_HOME}/docker-compose.yml" exec -T --user 1000 db backup-immuta.sh
find "${BACKUPS_DIR}" -type f -name '*.tar.gz' | sort -r | tail -n +${BACKUP_THRESHOLD} | xargs rm -f

Be sure to make the script executable:

chmod +x backup-immuta-task.sh

Automatic Backups using Cron

It is recommended to use Cron to run the backup job periodically. The Cron job must have access to the Docker engine, which typically means that it must run at root. You may add the entry in your preferred location, such as /etc/crontab or /etc/cron.d, or you may add it to the root user's configuration by running sudo crontab -e.

Below is an example crontab entry that runs the backup script every day at midnight.

0 0 * * * /opt/immuta/backup-immuta-task.sh > /opt/immuta/backup-immuta-task.log

Backups for Migration to an Immuta Kubernetes Helm Deployment

Migrating the Immuta Docker deployment to a Kubernetes Helm deployment can be done with the Docker backup file. The singular backup file consists of two databases: the Immuta Metadata database (bometadata) and the Immuta Query Engine database (immuta). The Immuta Metadata database and Immuta Query Engine database are separate components for the Immuta Kubernetes Helm deployment; however, the singular backup file can be used as the backup for both components. The file will need to be staged in an appropriate location for the Immuta Metadata and Immuta Query Engine databases.

Example:

cp volumes/immuta-db/backups/immuta-20210512103030.tar.gz <pvc or bucket>/database/immuta-20210512103030.tar.gz
cp volumes/immuta-db/backups/immuta-20210512103030.tar.gz <pvc or bucket>/query-engine/immuta-20210512103030.tar.gz

Once the backup is created, refer to the Import Backups Helm documentation for next steps.