备份与恢复

数据备份是生产环境运维的关键环节。本章介绍 Jenkins 的备份策略、工具和灾难恢复流程。

备份概述

需要备份的内容

┌─────────────────────────────────────────────────────────────────┐
│                    Jenkins 备份内容                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  $JENKINS_HOME                                                  │
│  ├── config.xml              ✅ 核心配置文件                      │
│  ├── *.xml (jobs/*)          ✅ 所有 Job 配置                     │
│  ├── secrets/                ✅ 凭据密钥 (加密)                   │
│  ├── identity.key            ✅ 实例标识                         │
│  ├── credentials.xml         ✅ 凭据定义                         │
│  ├── users/                  ✅ 用户数据                         │
│  ├── plugins/                ✅ 插件配置                         │
│  ├── workspace/              ⚠️  大多数情况可跳过                 │
│  ├── builds/                 ⚠️  可选（占用空间大）               │
│  └── logs/                   ❌ 通常不需要                        │
│                                                                  │
│  系统配置:                                                        │
│  ├── /etc/sysconfig/jenkins  ✅ Linux 配置                       │
│  ├── nginx 配置              ✅ 反向代理配置                      │
│  └── SSL 证书                ✅ TLS/SSL 证书                     │
└─────────────────────────────────────────────────────────────────┘

备份策略对比

策略	频率	保留	适用场景
全量备份	每天	30 天	小型实例
增量备份	每小时	7 天	中型实例
连续备份	实时	-	大型企业
配置即代码	每次提交	版本化	所有场景

手动备份

完整备份脚本

#!/bin/bash
# jenkins-backup.sh

# 配置
JENKINS_HOME="/var/lib/jenkins"
BACKUP_DIR="/backup/jenkins"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_NAME="jenkins_backup_${DATE}"
KEEP_DAYS=30

# 创建备份目录
mkdir -p ${BACKUP_DIR}/${BACKUP_NAME}

# 备份核心文件
echo "备份配置文件..."
tar czf ${BACKUP_DIR}/${BACKUP_NAME}/jenkins_home.tar.gz \
    -C ${JENKINS_HOME} \
    --exclude='workspace' \
    --exclude='builds' \
    --exclude='.*.tmp' \
    --exclude='*.log' \
    .

# 备份凭据密钥（重要！）
echo "备份凭据密钥..."
tar czf ${BACKUP_DIR}/${BACKUP_NAME}/secrets.tar.gz \
    -C ${JENKINS_HOME} \
    secrets/

# 备份 Job 配置
echo "备份 Job 配置..."
tar czf ${BACKUP_DIR}/${BACKUP_NAME}/jobs.tar.gz \
    -C ${JENKINS_HOME} \
    jobs/

# 创建备份清单
cat > ${BACKUP_DIR}/${BACKUP_NAME}/manifest.txt << EOF
Jenkins Backup Manifest
======================
Date: ${DATE}
Hostname: $(hostname)
Jenkins Version: $(cat ${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion 2>/dev/null || echo "Unknown")
Backup Contents:
- jenkins_home.tar.gz (configs, plugins, users)
- secrets.tar.gz (encrypted credentials)
- jobs.tar.gz (job configurations)
EOF

# 创建压缩包
cd ${BACKUP_DIR}
tar czf ${BACKUP_NAME}.tar.gz ${BACKUP_NAME}

# 计算校验和
sha256sum ${BACKUP_NAME}.tar.gz > ${BACKUP_NAME}.tar.gz.sha256

# 清理临时目录
rm -rf ${BACKUP_DIR}/${BACKUP_NAME}

# 清理旧备份
echo "清理超过 ${KEEP_DAYS} 天的备份..."
find ${BACKUP_DIR} -name "jenkins_backup_*.tar.gz" -mtime +${KEEP_DAYS} -delete
find ${BACKUP_DIR} -name "jenkins_backup_*.tar.gz.sha256" -mtime +${KEEP_DAYS} -delete

# 上传到远程存储（可选）
# aws s3 cp ${BACKUP_DIR}/${BACKUP_NAME}.tar.gz s3://my-bucket/jenkins/

echo "备份完成: ${BACKUP_DIR}/${BACKUP_NAME}.tar.gz"
ls -lh ${BACKUP_DIR}/*.tar.gz | tail -5

快速备份命令

# 快速备份核心配置
sudo tar czf /backup/jenkins_config_$(date +%Y%m%d).tar.gz \
    -C /var/lib jenkins/config.xml \
    jenkins/jobs/ \
    jenkins/secrets/ \
    jenkins/users/ \
    jenkins/credentials.xml

# 备份所有 Job 配置
sudo tar czf /backup/jenkins_jobs_$(date +%Y%m%d).tar.gz \
    -C /var/lib jenkins/jobs/

使用插件备份

ThinBackup 插件

安装插件：

系统管理 → 插件管理 → 搜索 "thin backup"

配置备份：

系统管理 → ThinBackup
    备份目录: /var/jenkins_backup
    备份策略:
        - 完整备份: H 2 * * * (每天凌晨2点)
        - 增量备份: H H/4 * * * (每6小时)
    
    备份内容:
        ✅ 配置文件
        ✅ Job 目录
        ✅ 用户内容
        ✅ 构建记录 (最近 N 个)
        ✅ 工作区 (最近 N 个)
    
    保留:
        - 最大备份数量: 30
        - 备份文件名: jenkins_${USER_NAME}_${DATE}

Backup 插件

系统管理 → Jenkins Backup Manager
    ✅ 启用备份
    备份目标: /backup/jenkins
    排除模式:
        - **/workspace/**
        - **/builds/**
        - **/*.log

备份 Job 配置

Job 配置导出

// Pipeline: 导出所有 Job 配置
import hudson.model.*
import hudson.tasks.*

def jobNames = Hudson.instance.projectNames
def exportDir = '/backup/jenkins_jobs'

new File(exportDir).mkdirs()

jobNames.each { name ->
    def job = Hudson.instance.getItem(name)
    def configFile = job.configFile
    
    if (configFile != null) {
        def xml = configFile.asString()
        def file = new File("${exportDir}/${name}.xml")
        file.text = xml
        println "Exported: ${name}"
    }
}

println "Total jobs exported: ${jobNames.size()}"

使用 Job DSL 备份

// 备份脚本
import javaposse.jobdsl.dsl.*

def workspace = new File('/tmp/jobs_backup')
workspace.mkdirs()

Jenkins.instance.projects.each { job ->
    def name = job.name
    def xml = Jenkins.instance.getJob(name).configFile.asString()
    
    new File(workspace, "${name}.xml") << xml
    println "Backup: ${name}"
}

定时备份任务

Crontab 配置

# /etc/cron.d/jenkins-backup

# 每天凌晨 2 点完整备份
0 2 * * * root /opt/scripts/jenkins-backup.sh full

# 每 6 小时增量备份
0 */6 * * * root /opt/scripts/jenkins-backup.sh incremental

# 每周日凌晨 3 点清理
0 3 * * 0 root /opt/scripts/jenkins-cleanup.sh

Jenkins Pipeline 备份

// backup.groovy
pipeline {
    agent any
    
    parameters {
        choice(name: 'BACKUP_TYPE', choices: ['full', 'configs', 'jobs'], description: '备份类型')
        string(name: 'RETENTION_DAYS', defaultValue: '30', description: '保留天数')
    }
    
    environment {
        BACKUP_DIR = '/backup/jenkins'
        S3_BUCKET = 's3://my-company-jenkins-backup'
        S3_PROFILE = 'jenkins-backup'
    }
    
    stages {
        stage('Prepare') {
            steps {
                script {
                    env.BACKUP_NAME = "jenkins_${params.BACKUP_TYPE}_${env.BUILD_NUMBER}_${new Date().format('yyyyMMdd_HHmmss')}"
                }
                sh "mkdir -p ${env.BACKUP_DIR}/${env.BACKUP_NAME}"
            }
        }
        
        stage('Backup Configs') {
            when {
                anyOf {
                    expression { params.BACKUP_TYPE == 'full' }
                    expression { params.BACKUP_TYPE == 'configs' }
                }
            }
            steps {
                sh '''
                    tar czf ${BACKUP_DIR}/${BACKUP_NAME}/jenkins_home.tar.gz \
                        -C ${JENKINS_HOME} \
                        --exclude='workspace' \
                        --exclude='builds' \
                        .
                '''
            }
        }
        
        stage('Backup Secrets') {
            when {
                expression { params.BACKUP_TYPE == 'full' }
            }
            steps {
                sh '''
                    tar czf ${BACKUP_DIR}/${BACKUP_NAME}/secrets.tar.gz \
                        -C ${JENKINS_HOME} secrets/
                '''
            }
        }
        
        stage('Upload to S3') {
            steps {
                withCredentials([string(credentialsId: 'aws-access-key', variable: 'AWS_ACCESS_KEY_ID')]) {
                    sh '''
                        export AWS_ACCESS_KEY_ID
                        export AWS_SECRET_ACCESS_KEY
                        
                        aws s3 sync ${BACKUP_DIR}/ s3://${S3_BUCKET}/ \
                            --storage-class STANDARD_IA \
                            --exclude "*.tmp"
                        
                        # 上传到 Glacier
                        aws s3 cp ${BACKUP_DIR}/${BACKUP_NAME}/jenkins_home.tar.gz \
                            s3://${S3_BUCKET}/archive/
                    '''
                }
            }
        }
        
        stage('Cleanup Old Backups') {
            steps {
                sh '''
                    # 清理本地备份
                    find ${BACKUP_DIR} -name "*.tar.gz" -mtime +${RETENTION_DAYS} -delete
                    
                    # 清理 S3 旧备份
                    aws s3 ls ${S3_BUCKET}/ | \
                        while read name date time; do
                            if [ $(stat -c %Y "$name") -lt $(date -d "${RETENTION_DAYS} days ago" +%s) ]; then
                                aws s3 rm "s3://${S3_BUCKET}/$name"
                            fi
                        done
                '''
            }
        }
    }
    
    post {
        always {
            echo "备份任务 ${env.BACKUP_NAME} 完成"
        }
        success {
            emailext(
                subject: "✅ Jenkins 备份成功: ${env.BACKUP_NAME}",
                body: "备份类型: ${params.BACKUP_TYPE}",
                to: 'ops-team@example.com'
            )
        }
        failure {
            emailext(
                subject: "❌ Jenkins 备份失败",
                body: "请检查备份任务日志",
                to: 'ops-team@example.com'
            )
        }
    }
}

恢复操作

恢复步骤

#!/bin/bash
# jenkins-restore.sh

set -e

BACKUP_FILE=$1
JENKINS_HOME="/var/lib/jenkins"

if [ -z "$BACKUP_FILE" ]; then
    echo "用法: $0 <backup_file.tar.gz>"
    exit 1
fi

echo "⚠️  警告: 此操作将覆盖现有 Jenkins 配置"
echo "备份文件: $BACKUP_FILE"
read -p "继续? (yes/no): " confirm

if [ "$confirm" != "yes" ]; then
    echo "操作取消"
    exit 0
fi

# 停止 Jenkins
echo "停止 Jenkins..."
sudo systemctl stop jenkins

# 备份当前配置
echo "备份当前配置..."
CURRENT_BACKUP="/tmp/jenkins_current_$(date +%Y%m%d_%H%M%S).tar.gz"
sudo tar czf "$CURRENT_BACKUP" -C ${JENKINS_HOME} .

# 解压备份
echo "解压备份文件..."
sudo tar xzf "$BACKUP_FILE" -C /tmp/

# 确定解压目录
BACKUP_DIR=$(tar tzf "$BACKUP_FILE" | head -1 | cut -f1 -d"/")

# 恢复配置
echo "恢复配置..."
sudo cp -r /tmp/${BACKUP_DIR}/* ${JENKINS_HOME}/

# 修复权限
echo "修复权限..."
sudo chown -R jenkins:jenkins ${JENKINS_HOME}
sudo chmod -R 755 ${JENKINS_HOME}
sudo chmod 700 ${JENKINS_HOME}/secrets

# 清理临时文件
rm -rf /tmp/${BACKUP_DIR}

# 启动 Jenkins
echo "启动 Jenkins..."
sudo systemctl start jenkins

echo "✅ 恢复完成！"
echo "旧配置已备份到: $CURRENT_BACKUP"

选择性恢复

# 仅恢复 Job 配置
sudo tar xzf backup.tar.gz -C /tmp/
sudo cp /tmp/jobs/* ${JENKINS_HOME}/jobs/ -r
sudo chown -R jenkins:jenkins ${JENKINS_HOME}/jobs
sudo systemctl restart jenkins

# 仅恢复单个 Job
JOB_NAME="my-project"
sudo tar xzf backup.tar.gz -C /tmp/
sudo cp /tmp/jobs/${JOB_NAME} ${JENKINS_HOME}/jobs/ -r
sudo chown -R jenkins:jenkins ${JENKINS_HOME}/jobs/${JOB_NAME}
sudo systemctl restart jenkins

灾难恢复演练

恢复检查清单

┌─────────────────────────────────────────────────────────────────┐
│                  Jenkins 灾难恢复检查清单                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  恢复前准备:                                                      │
│  □ 确认备份完整性 (文件大小、校验和)                               │
│  □ 确认 Jenkins 版本兼容                                          │
│  □ 准备目标服务器环境                                             │
│  □ 通知相关团队                                                    │
│                                                                  │
│  恢复执行:                                                        │
│  □ 停止 Jenkins 服务                                             │
│  □ 备份当前配置（以防万一）                                        │
│  □ 清理 JENKINS_HOME                                             │
│  □ 恢复备份文件                                                   │
│  □ 修复文件权限                                                   │
│  □ 启动 Jenkins                                                  │
│                                                                  │
│  恢复后验证:                                                      │
│  □ 验证 Web 界面访问                                              │
│  □ 验证登录功能                                                   │
│  □ 验证 Job 配置                                                  │
│  □ 验证凭据完整性                                                 │
│  □ 验证节点连接                                                   │
│  □ 测试构建功能                                                    │
│                                                                  │
│  后续操作:                                                        │
│  □ 通知团队恢复完成                                               │
│  □ 确认监控正常                                                   │
│  □ 更新文档                                                       │
└─────────────────────────────────────────────────────────────────┘

恢复验证脚本

#!/bin/bash
# verify-restore.sh

JENKINS_URL="http://localhost:8080"
EXPECTED_JOBS=("job1" "job2" "job3")

echo "验证 Jenkins 恢复..."

# 1. 检查服务状态
echo "1. 检查 Jenkins 服务..."
if curl -sf "${JENKINS_URL}/login" > /dev/null; then
    echo "   ✅ Jenkins Web 界面可访问"
else
    echo "   ❌ Jenkins Web 界面无法访问"
    exit 1
fi

# 2. 检查凭据
echo "2. 检查凭据..."
CREDENTIALS=$(curl -s -u admin:$(cat /var/lib/jenkins/secrets/initialAdminPassword) \
    "${JENKINS_URL}/credentials/api/json?tree=credentials[id,displayName]")
if echo "$CREDENTIALS" | grep -q "credentials"; then
    echo "   ✅ 凭据接口正常"
else
    echo "   ⚠️  凭据可能有问题"
fi

# 3. 检查 Job 配置
echo "3. 检查 Job 配置..."
for job in "${EXPECTED_JOBS[@]}"; do
    RESP=$(curl -sf -o /dev/null -w "%{http_code}" \
        "${JENKINS_URL}/job/${job}/config.xml")
    if [ "$RESP" == "200" ]; then
        echo "   ✅ Job '${job}' 存在"
    else
        echo "   ❌ Job '${job}' 不存在或无法访问"
    fi
done

# 4. 检查节点
echo "4. 检查节点连接..."
NODES=$(curl -s -u admin:$(cat /var/lib/jenkins/secrets/initialAdminPassword) \
    "${JENKINS_URL}/computer/api/json")
echo "   节点信息: $(echo $NODES | jq '.computer | length') 个节点"

echo ""
echo "恢复验证完成！"

配置即代码备份

备份到 Git

#!/bin/bash
# backup-to-git.sh

JENKINS_HOME="/var/lib/jenkins"
GIT_REPO="/opt/jenkins-config-backup"
JENKINS_USER="jenkins"

# 克隆或初始化仓库
if [ ! -d "$GIT_REPO/.git" ]; then
    git clone git@github.com:org/jenkins-config.git "$GIT_REPO"
fi

cd "$GIT_REPO"

# 导出配置
export-jenkins-configs() {
    # 导出全局配置
    curl -s -u admin:${JENKINS_API_TOKEN} \
        "${JENKINS_URL}/api/json?tree=mode,nodeDescription,nodes[displayName,executors],primaryView[name],views[name]" \
        > config/global.json
    
    # 导出所有 Job 配置
    for job in $(curl -s -u admin:${JENKINS_API_TOKEN} \
        "${JENKINS_URL}/api/json?tree=jobs[name]" | jq -r '.jobs[].name'); do
        curl -s -u admin:${JENKINS_API_TOKEN} \
            "${JENKINS_URL}/job/${job}/config.xml" \
            > "jobs/${job}.xml"
        echo "Exported: ${job}"
    done
    
    # 复制 secrets（加密）
    cp -r ${JENKINS_HOME}/secrets secrets/
}

# Git 操作
git add -A
git commit -m "Backup: $(date '+%Y-%m-%d %H:%M:%S')"
git push origin main

echo "配置已备份到 Git"

下一步

接下来让我们学习 Jenkins 最佳实践。

👉 最佳实践

⚠️ 重要提醒：务必定期测试备份恢复流程，确保备份可用！