upload-archive Git Command Guide
The git upload-archive command sends repository archives from a Git server to clients using the smart HTTP transport protocol. It creates and streams archive files (tar, zip) of repository content at specific commits, enabling efficient bulk transfers of repository snapshots without requiring full Git protocol support.
git upload-archive Syntax:
Section titled “git upload-archive Syntax:”git upload-archive <directory>Parameters:
Section titled “Parameters:”| Parameter | Description |
|---|---|
<directory> | Repository directory to serve |
Understanding Upload-Archive Operations:
Section titled “Understanding Upload-Archive Operations:”Smart HTTP Archive Transport:
Section titled “Smart HTTP Archive Transport:”Archive Transport Process:├── Client Request: HTTP POST to git-upload-archive├── Server Processing: git upload-archive execution├── Archive Generation: Create tar/zip from tree object├── Stream Response: Send compressed archive to client├── Client Extraction: Unpack archive locally└── Repository State: No local repository createdArchive Format Support:
Section titled “Archive Format Support:”Supported Archive Formats:├── tar: Uncompressed tar archive├── tgz: Gzip-compressed tar archive├── tar.gz: Same as tgz├── zip: Zip compressed archive└── Custom: User-defined formats via hooksServer-Side Architecture:
Section titled “Server-Side Architecture:”Git Upload-Archive Server Setup:├── HTTP Server: Web server handling Git requests├── CGI Script: git-http-backend or custom CGI├── Repository Access: Direct filesystem access to .git├── Archive Generation: On-demand archive creation└── Stream Response: Efficient HTTP streamingBasic Upload-Archive Operations:
Section titled “Basic Upload-Archive Operations:”Server Setup for Archive Serving:
Section titled “Server Setup for Archive Serving:”# Configure Apache for git-http-backendcat > /etc/apache2/sites-available/git-archive.conf << 'EOF'<VirtualHost *:80> ServerName git.example.com DocumentRoot /var/www/git
SetEnv GIT_PROJECT_ROOT /var/www/git SetEnv GIT_HTTP_EXPORT_ALL
<Directory /var/www/git> Options +ExecCGI AddHandler cgi-script .cgi DirectoryIndex gitweb.cgi
# Archive upload support RewriteEngine On RewriteCond %{REQUEST_URI} ^/git/(.*/git-upload-archive)$ RewriteRule ^/git/(.*)$ /usr/lib/git-core/git-http-backend [L] </Directory></VirtualHost>EOF
# Enable sitea2ensite git-archive.confsystemctl reload apache2Client Archive Download:
Section titled “Client Archive Download:”# Download archive using git archivegit archive --remote=http://git.example.com/repo.git HEAD | tar x
# Download specific formatgit archive --remote=http://git.example.com/repo.git --format=zip HEAD > archive.zip
# Download subdirectorygit archive --remote=http://git.example.com/repo.git HEAD:src/ | tar xRepository Configuration:
Section titled “Repository Configuration:”# Configure repository for HTTP archive accessgit config http.uploadpack truegit config http.receivepack false # Read-only access
# Set default archive formatgit config uploadarchive.allowUnreachable truegit config uploadarchive.allowIncompleteArchive trueAdvanced Upload-Archive Scenarios:
Section titled “Advanced Upload-Archive Scenarios:”Custom Archive Generation:
Section titled “Custom Archive Generation:”# Setup custom archive hookssetup_custom_archive() { echo "Setting up custom archive generation..."
# Create pre-upload-archive hook cat > .git/hooks/pre-upload-archive << 'EOF'#!/bin/bash# Custom archive preparation
echo "Preparing archive for upload..."
# Generate additional filesecho "$(date)" > VERSION.txtecho "$(git rev-parse HEAD)" > COMMIT.txt
# Add to index for archivinggit add VERSION.txt COMMIT.txt
echo "Archive preparation complete"EOF
chmod +x .git/hooks/pre-upload-archive
# Create post-upload-archive hook cat > .git/hooks/post-upload-archive << 'EOF'#!/bin/bash# Cleanup after archive upload
echo "Cleaning up after archive upload..."
# Remove temporary filesrm -f VERSION.txt COMMIT.txt
echo "Cleanup complete"EOF
chmod +x .git/hooks/post-upload-archive}
# Usagesetup_custom_archiveArchive Access Control:
Section titled “Archive Access Control:”# Implement access control for archivesarchive_access_control() { echo "Setting up archive access control..."
# Create access control script cat > /usr/local/bin/git-archive-access << 'EOF'#!/bin/bash# Control access to git upload-archive
REPO="$1"USER="$2"
# Allow read access to specific userscase "$USER" in alice|bob|charlie) exit 0 # Allow ;; *) echo "Access denied for user: $USER" exit 1 # Deny ;;esacEOF
chmod +x /usr/local/bin/git-archive-access
# Configure git-http-backend to use access control export GIT_ARCHIVE_ACCESS_CHECK=/usr/local/bin/git-archive-access}
# Usagearchive_access_controlLarge Repository Handling:
Section titled “Large Repository Handling:”# Handle large repositories with upload-archivelarge_repo_archive() { echo "Configuring large repository archive handling..."
# Configure for large archives git config uploadarchive.maxFileSize 100m git config uploadarchive.compression gzip git config uploadarchive.threads 4
# Setup resumable downloads cat > .git/hooks/pre-upload-archive << 'EOF'#!/bin/bash# Prepare large archive
# Check archive size estimateTREE_SIZE=$(git count-objects -v | grep size: | cut -d: -f2 | tr -d ' ')if [ "$TREE_SIZE" -gt 50000000 ]; then # > 50MB echo "Large archive detected, optimizing compression..." export GIT_ARCHIVE_COMPRESSION=gzip-9fiEOF
chmod +x .git/hooks/pre-upload-archive}
# Usagelarge_repo_archiveConfiguration and Best Practices:
Section titled “Configuration and Best Practices:”Server Configuration:
Section titled “Server Configuration:”# Apache configuration for upload-archivecat > /etc/apache2/mods-available/git-archive.conf << 'EOF'# Git archive upload configuration<IfModule mod_alias.c> Alias /git /usr/lib/git-core/git-http-backend</IfModule>
<Location /git> SetEnv GIT_PROJECT_ROOT /var/www/git SetEnv GIT_HTTP_EXPORT_ALL SetEnv REMOTE_USER=$REDIRECT_REMOTE_USER
# Security headers Header always set X-Frame-Options DENY Header always set X-Content-Type-Options nosniff
# Archive-specific settings SetEnv GIT_ARCHIVE_FORMAT tar.gz SetEnv GIT_ARCHIVE_PREFIX "repo-$(date +%Y%m%d)-"</Location>EOF
# Nginx configurationcat > /etc/nginx/sites-available/git-archive << 'EOF'server { listen 80; server_name git.example.com;
root /var/www/git; index index.html;
location ~ /git/(.*/git-upload-archive)$ { fastcgi_pass unix:/var/run/fcgiwrap.socket; fastcgi_param SCRIPT_FILENAME /usr/lib/git-core/git-http-backend; fastcgi_param GIT_PROJECT_ROOT /var/www/git; fastcgi_param GIT_HTTP_EXPORT_ALL 1; fastcgi_param REMOTE_USER $remote_user; include fastcgi_params; }}EOFClient Configuration:
Section titled “Client Configuration:”# Configure client for archive downloadsgit config http.postBuffer 524288000 # 500MB buffergit config http.lowSpeedLimit 1000 # 1KB/s minimum speedgit config http.lowSpeedTime 300 # 5 minute timeout
# Set preferred archive formatgit config archive.defaultFormat tar.gzgit config archive.defaultPrefix "project-"Security Best Practices:
Section titled “Security Best Practices:”# Secure upload-archive deploymentsecure_archive_setup() { echo "Setting up secure archive access..."
# Create dedicated git user useradd -r -s /bin/bash -m -d /home/git-archive git-archive
# Setup SSH key authentication mkdir -p /home/git-archive/.ssh cat > /home/git-archive/.ssh/authorized_keys << EOFcommand="/usr/local/bin/validate-git-access",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3... user@clientEOF
# Set proper permissions chown -R git-archive:git-archive /home/git-archive/.ssh chmod 700 /home/git-archive/.ssh chmod 600 /home/git-archive/.ssh/authorized_keys
# Setup repositories mkdir -p /srv/git-archive chown git-archive:git-archive /srv/git-archive
echo "Secure archive setup complete"}
# Access validation scriptcat > /usr/local/bin/validate-git-access << 'EOF'#!/bin/bash# Validate Git archive access
USER="$1"REPO="$2"
# Log accessecho "$(date): $USER accessed $REPO" >> /var/log/git-archive/access.log
# Check permissionsif [ "$USER" = "admin" ] || groups "$USER" | grep -q "developers"; then exec git-upload-archive "$REPO"else echo "Access denied" exit 1fiEOF
chmod +x /usr/local/bin/validate-git-accessIntegration with Development Workflows:
Section titled “Integration with Development Workflows:”CI/CD Pipeline Integration:
Section titled “CI/CD Pipeline Integration:”# Upload-archive in CI/CD pipelinesci_archive_deployment() { echo "=== CI/CD Archive Deployment ==="
# Create deployment archive create_deployment_archive() { echo "Creating deployment archive..."
# Archive current build git archive --format=tar.gz --prefix="app-$(date +%Y%m%d-%H%M%S)/" HEAD > deployment.tar.gz
# Upload to artifact repository curl -X POST \ -H "Authorization: Bearer $ARTIFACT_TOKEN" \ -F "file=@deployment.tar.gz" \ "$ARTIFACT_REPO_URL/deployments/"
echo "Deployment archive created and uploaded" }
# Deploy from archive deploy_from_archive() { local archive_url="$1" local deploy_path="$2"
echo "Deploying from archive: $archive_url"
# Download and extract curl -L "$archive_url" | tar xz -C "$deploy_path"
# Post-deployment setup cd "$deploy_path" npm install npm run build
echo "Deployment complete" }
# Automated deployment automated_deployment() { echo "Running automated deployment..."
# Build and test npm run build npm test
if [ $? -eq 0 ]; then create_deployment_archive deploy_from_archive "$ARTIFACT_REPO_URL/latest.tar.gz" "/var/www/app" else echo "Tests failed, deployment aborted" exit 1 fi }
# Run CI/CD steps automated_deployment}
# Usage in CIci_archive_deploymentRelease Management:
Section titled “Release Management:”# Release management with upload-archiverelease_archive_management() { echo "=== Release Archive Management ==="
# Create release archive create_release_archive() { local version="$1" local commit="$2"
echo "Creating release archive for v$version..."
# Create annotated tag if needed if ! git tag -l | grep -q "^v$version$"; then git tag -a "v$version" "$commit" -m "Release v$version" fi
# Create release archive git archive --format=tar.gz \ --prefix="project-$version/" \ --output="release-$version.tar.gz" \ "v$version"
# Generate checksum sha256sum "release-$version.tar.gz" > "release-$version.tar.gz.sha256"
# Upload to release server scp "release-$version.tar.gz*" "releases@example.com:/var/www/releases/"
echo "Release v$version archive created and uploaded" }
# Download release archive download_release_archive() { local version="$1" local destination="${2:-.}"
echo "Downloading release archive v$version..."
# Download archive curl -L "https://releases.example.com/release-$version.tar.gz" \ -o "$destination/release-$version.tar.gz"
# Verify checksum curl -L "https://releases.example.com/release-$version.tar.gz.sha256" \ -o "$destination/release-$version.tar.gz.sha256"
cd "$destination" if sha256sum -c "release-$version.tar.gz.sha256"; then echo "Archive integrity verified"
# Extract tar xzf "release-$version.tar.gz" echo "Release v$version extracted to $destination/$(basename "$destination")-$version/" else echo "Archive integrity check failed" exit 1 fi }
# Interactive release management echo "Release Archive Options:" echo "1. Create release archive" echo "2. Download release archive"
read -p "Select option (1-2): " option
case "$option" in 1) read -p "Version: " version read -p "Commit (default: HEAD): " commit create_release_archive "$version" "${commit:-HEAD}" ;; 2) read -p "Version: " version read -p "Destination (default: .): " dest download_release_archive "$version" "${dest:-.}" ;; esac}
release_archive_managementRepository Mirroring:
Section titled “Repository Mirroring:”# Repository mirroring with archivesarchive_based_mirroring() { echo "=== Archive-Based Repository Mirroring ==="
# Setup mirror repository setup_archive_mirror() { local source_repo="$1" local mirror_path="$2"
echo "Setting up archive-based mirror..."
# Create mirror directory mkdir -p "$mirror_path" cd "$mirror_path"
# Initial mirror from archive git archive --remote="$source_repo" --format=tar HEAD | tar x
# Initialize as git repository git init git add . git commit -m "Initial mirror from $source_repo"
# Setup remote git remote add origin "$source_repo"
echo "Archive mirror setup complete" }
# Update mirror from archive update_archive_mirror() { echo "Updating archive mirror..."
# Get latest archive temp_archive=$(mktemp) git archive --remote="$(git remote get-url origin)" --format=tar HEAD > "$temp_archive"
# Extract to temporary location temp_dir=$(mktemp -d) tar xf "$temp_archive" -C "$temp_dir"
# Update mirror cd "$temp_dir" git add . git commit -m "Mirror update $(date)"
# Replace mirror content rm -rf "$OLDPWD"/* mv "$temp_dir"/* "$OLDPWD"/
# Cleanup rm -rf "$temp_dir" "$temp_archive"
echo "Mirror updated from archive" }
# Automated mirroring setup_automated_mirroring() { echo "Setting up automated mirroring..."
cat > update-mirror.sh << 'EOF'#!/bin/bash# Automated mirror updates
cd "$(dirname "$0")"./update_archive_mirror.sh
# Log updateecho "Mirror updated at $(date)" >> mirror.logEOF
chmod +x update-mirror.sh
# Setup cron (crontab -l ; echo "0 */4 * * * $(pwd)/update-mirror.sh") | crontab -
echo "Automated mirroring configured" }
# Interactive mirror management echo "Archive Mirror Options:" echo "1. Setup archive mirror" echo "2. Update mirror" echo "3. Setup automated mirroring"
read -p "Select option (1-3): " option
case "$option" in 1) read -p "Source repository: " source read -p "Mirror path: " mirror setup_archive_mirror "$source" "$mirror" ;; 2) update_archive_mirror ;; 3) setup_automated_mirroring ;; esac}
archive_based_mirroringTroubleshooting Common Issues:
Section titled “Troubleshooting Common Issues:”Archive Generation Errors:
Section titled “Archive Generation Errors:”# Diagnose archive generation issuesdiagnose_archive_generation() { echo "Diagnosing archive generation issues..."
# Test basic archive creation if git archive --format=tar HEAD >/dev/null 2>&1; then echo "✓ Basic archive generation works" else echo "✗ Archive generation failed" return 1 fi
# Test remote archive access remote_url=$(git remote get-url origin 2>/dev/null) if [ -n "$remote_url" ]; then if git archive --remote="$remote_url" --format=tar HEAD >/dev/null 2>&1; then echo "✓ Remote archive access works" else echo "✗ Remote archive access failed" fi fi
# Check repository state if git fsck --full >/dev/null 2>&1; then echo "✓ Repository integrity OK" else echo "✗ Repository has integrity issues" fi}
# Fix common archive issuesfix_archive_issues() { echo "Fixing common archive issues..."
# Ensure HEAD exists if ! git rev-parse HEAD >/dev/null 2>&1; then echo "Setting HEAD to main/master branch..." if git rev-parse refs/heads/main >/dev/null 2>&1; then git update-ref HEAD refs/heads/main elif git rev-parse refs/heads/master >/dev/null 2>&1; then git update-ref HEAD refs/heads/master fi fi
# Clean up corrupted objects git gc --aggressive --prune=now
# Test archive generation if git archive --format=tar HEAD >/dev/null 2>&1; then echo "✓ Archive generation fixed" else echo "✗ Archive generation still failing" fi}HTTP Transport Issues:
Section titled “HTTP Transport Issues:”# Troubleshoot HTTP archive transporttroubleshoot_http_archive() { local repo_url="$1"
echo "Troubleshooting HTTP archive transport..."
# Test basic connectivity if curl -s "$repo_url" >/dev/null; then echo "✓ Repository URL accessible" else echo "✗ Repository URL not accessible" return 1 fi
# Test git-upload-archive endpoint if curl -s "$repo_url/git-upload-archive" >/dev/null; then echo "✓ Upload-archive endpoint accessible" else echo "✗ Upload-archive endpoint not accessible" fi
# Test with git archive if git ls-remote "$repo_url" >/dev/null 2>&1; then echo "✓ Git remote access works" else echo "✗ Git remote access failed" fi
# Test archive download if git archive --remote="$repo_url" --format=tar HEAD >/dev/null 2>&1; then echo "✓ Archive download works" else echo "✗ Archive download failed" fi}
# Usagetroubleshoot_http_archive "http://git.example.com/repo.git"Permission and Access Issues:
Section titled “Permission and Access Issues:”# Fix permission issuesfix_archive_permissions() { echo "Fixing archive access permissions..."
# Set proper permissions on repository find .git -type f -exec chmod 644 {} \; find .git -type d -exec chmod 755 {} \;
# Ensure web server can access web_user="www-data" if id "$web_user" >/dev/null 2>&1; then chown -R "$web_user:$web_user" .git fi
# Test access if [ -r .git/config ] && [ -x .git ]; then echo "✓ Repository permissions OK" else echo "✗ Repository permissions incorrect" fi}
# Configure access controlsetup_archive_access_control() { echo "Setting up archive access control..."
# Create access control file cat > .git/archive-access.conf << EOF# Archive access controlallow = alice, bob, charliedeny = malloryadmin = adminEOF
# Create access check script cat > /usr/local/bin/check-archive-access << 'EOF'#!/bin/bash# Check archive access permissions
USER="$1"REPO="$2"
if [ "$USER" = "admin" ]; then exit 0 # Admin has full accessfi
ACCESS_FILE="$REPO/.git/archive-access.conf"if [ -f "$ACCESS_FILE" ]; then if grep -q "^allow.*$USER" "$ACCESS_FILE"; then exit 0 elif grep -q "^deny.*$USER" "$ACCESS_FILE"; then exit 1 fifi
# Default denyexit 1EOF
chmod +x /usr/local/bin/check-archive-access
echo "Archive access control configured"}Performance Issues:
Section titled “Performance Issues:”# Optimize archive performanceoptimize_archive_performance() { echo "Optimizing archive performance..."
# Configure compression git config core.compression 9 git config pack.compression 9
# Enable parallel processing git config pack.threads 4
# Configure archive format preferences git config archive.defaultFormat tar.gz git config archive.useBuiltinTar true
# Setup caching mkdir -p /var/cache/git-archive git config uploadarchive.cacheDir /var/cache/git-archive
echo "Archive performance optimized"}
# Monitor archive performancemonitor_archive_performance() { echo "Monitoring archive performance..."
# Measure archive creation time start_time=$(date +%s.%3N) git archive --format=tar HEAD >/dev/null end_time=$(date +%s.%3N)
archive_time=$(echo "$end_time - $start_time" | bc) echo "Archive creation time: ${archive_time}s"
# Check archive size archive_size=$(git archive --format=tar HEAD | wc -c) echo "Archive size: $archive_size bytes"
# Performance recommendations if (( $(echo "$archive_time > 30" | bc -l) )); then echo "Consider optimizing repository or enabling caching" fi
if [ "$archive_size" -gt 1073741824 ]; then # > 1GB echo "Large archive detected, consider splitting or using git bundle" fi}Real-World Usage Examples:
Section titled “Real-World Usage Examples:”Enterprise Archive Distribution:
Section titled “Enterprise Archive Distribution:”#!/bin/bash# Enterprise archive distribution system
enterprise_archive_system() { echo "=== Enterprise Archive Distribution ==="
# Configuration ARCHIVE_ROOT="/srv/archives" WEB_ROOT="/var/www/archives" RETENTION_DAYS=90
# Setup archive repository setup_archive_repository() { echo "Setting up enterprise archive repository..."
# Create directories sudo mkdir -p "$ARCHIVE_ROOT" "$WEB_ROOT" sudo chown www-data:www-data "$ARCHIVE_ROOT" "$WEB_ROOT"
# Setup Apache configuration sudo tee "/etc/apache2/sites-available/archives.conf" << EOF<VirtualHost *:80> ServerName archives.company.com DocumentRoot $WEB_ROOT
<Directory $WEB_ROOT> Options Indexes FollowSymLinks AllowOverride None Require all granted
# Enable directory browsing IndexOptions FancyIndexing NameWidth=* DescriptionWidth=* HTMLTable
# Protect sensitive archives <Files *.tar.gz> Header set X-Robots-Tag "noindex, nofollow" </Files> </Directory>
# Archive generation endpoint ScriptAlias /generate-archive /usr/local/bin/generate-archive.cgi</VirtualHost>EOF
sudo a2ensite archives.conf sudo systemctl reload apache2
echo "Archive repository setup complete" }
# Generate project archive generate_project_archive() { local project="$1" local version="$2"
echo "Generating archive for $project v$version..."
# Create archive directory archive_dir="$ARCHIVE_ROOT/$project/$version" sudo mkdir -p "$archive_dir" sudo chown www-data:www-data "$archive_dir"
# Generate archive (as www-data) sudo -u www-data bash << EOFcd "$archive_dir"git clone --depth 1 --branch "v$version" "https://git.company.com/$project.git" sourcecd sourcegit archive --format=tar.gz --prefix="$project-$version/" HEAD > "../$project-$version.tar.gz"cd ..rm -rf source
# Generate metadatacat > "$project-$version.meta" << META_EOFProject: $projectVersion: $versionGenerated: $(date)Size: $(stat -f%z "$project-$version.tar.gz" 2>/dev/null || stat -c%s "$project-$version.tar.gz")SHA256: $(sha256sum "$project-$version.tar.gz" | cut -d' ' -f1)META_EOFEOF
# Create web symlink sudo ln -sf "$archive_dir" "$WEB_ROOT/$project-$version"
echo "Archive generated: $WEB_ROOT/$project-$version" }
# Cleanup old archives cleanup_old_archives() { echo "Cleaning up archives older than $RETENTION_DAYS days..."
find "$ARCHIVE_ROOT" -type f -name "*.tar.gz" -mtime +$RETENTION_DAYS | while read -r archive; do echo "Removing old archive: $archive" rm -f "$archive" # Remove metadata and web links meta_file="${archive%.tar.gz}.meta" web_link="$WEB_ROOT/$(basename "${archive%.tar.gz}")" rm -f "$meta_file" "$web_link" done
# Remove empty directories find "$ARCHIVE_ROOT" -type d -empty -delete find "$WEB_ROOT" -type l -exec test ! -e {} \; -delete
echo "Archive cleanup complete" }
# Interactive archive management echo "Enterprise Archive System Options:" echo "1. Setup archive repository" echo "2. Generate project archive" echo "3. Cleanup old archives"
read -p "Select option (1-3): " option
case "$option" in 1) setup_archive_repository ;; 2) read -p "Project name: " project read -p "Version: " version generate_project_archive "$project" "$version" ;; 3) cleanup_old_archives ;; esac}
enterprise_archive_systemCDN Integration for Archives:
Section titled “CDN Integration for Archives:”# CDN integration for archive distributioncdn_archive_integration() { echo "=== CDN Archive Integration ==="
# Configuration CDN_URL="https://cdn.company.com/archives" LOCAL_ARCHIVE_DIR="/var/www/archives" CDN_DISTRIBUTION_ID="E1234567890"
# Setup CDN distribution setup_cdn_distribution() { echo "Setting up CDN distribution for archives..."
# Configure AWS CloudFront (example) aws cloudfront create-distribution \ --distribution-config "{ \"CallerReference\": \"git-archives-$(date +%s)\", \"Origins\": { \"Quantity\": 1, \"Items\": [{ \"Id\": \"git-archives-origin\", \"DomainName\": \"archives.company.com\", \"CustomOriginConfig\": { \"HTTPPort\": 80, \"HTTPSPort\": 443, \"OriginProtocolPolicy\": \"https-only\" } }] }, \"DefaultCacheBehavior\": { \"TargetOriginId\": \"git-archives-origin\", \"ViewerProtocolPolicy\": \"redirect-to-https\", \"CachePolicyId\": \"4135ea2d-6df8-44a3-9df3-4b5a84be39ad\" }, \"Enabled\": true }"
# Setup invalidation automation cat > /usr/local/bin/invalidate-cdn.sh << EOF#!/bin/bash# Invalidate CDN cache for new archives
aws cloudfront create-invalidation \ --distribution-id $CDN_DISTRIBUTION_ID \ --paths "/*"EOF
chmod +x /usr/local/bin/invalidate-cdn.sh
echo "CDN distribution setup complete" }
# Upload archive to CDN upload_archive_to_cdn() { local archive_path="$1" local cdn_key="$2"
echo "Uploading archive to CDN..."
# Upload to S3 (backend for CloudFront) aws s3 cp "$archive_path" "s3://archives-bucket/$cdn_key" \ --metadata "git-commit=$(git rev-parse HEAD)"
# Invalidate CDN cache /usr/local/bin/invalidate-cdn.sh
echo "Archive uploaded to CDN: $CDN_URL/$cdn_key" }
# Download from CDN with fallback download_from_cdn() { local archive_name="$1" local local_path="$2"
echo "Downloading archive from CDN..."
# Try CDN first if curl -L "$CDN_URL/$archive_name" -o "$local_path" 2>/dev/null; then echo "✓ Downloaded from CDN" else # Fallback to direct download echo "CDN download failed, trying direct download..." curl -L "http://archives.company.com/$archive_name" -o "$local_path" || { echo "✗ All download methods failed" return 1 } fi
# Verify download if [ -f "$local_path" ]; then echo "Archive downloaded successfully: $local_path" else echo "Download verification failed" return 1 fi }
# Automated CDN sync setup_cdn_sync() { echo "Setting up automated CDN synchronization..."
cat > /usr/local/bin/sync-archives-cdn.sh << EOF#!/bin/bash# Sync local archives to CDN
ARCHIVE_DIR="$LOCAL_ARCHIVE_DIR"CDN_URL="$CDN_URL"
# Find new archivesfind "\$ARCHIVE_DIR" -name "*.tar.gz" -mtime -1 | while read -r archive; do cdn_key="archives/\$(basename "\$archive")"
# Check if already on CDN if ! curl -s "\$CDN_URL/\$cdn_key" >/dev/null; then echo "Uploading new archive: \$archive" upload_archive_to_cdn "\$archive" "\$cdn_key" fidone
echo "CDN sync complete"EOF
chmod +x /usr/local/bin/sync-archives-cdn.sh
# Setup cron for regular sync (crontab -l ; echo "0 */2 * * * /usr/local/bin/sync-archives-cdn.sh") | crontab -
echo "Automated CDN sync configured" }
# Interactive CDN management echo "CDN Archive Integration Options:" echo "1. Setup CDN distribution" echo "2. Upload archive to CDN" echo "3. Download from CDN" echo "4. Setup automated sync"
read -p "Select option (1-4): " option
case "$option" in 1) setup_cdn_distribution ;; 2) read -p "Archive path: " archive read -p "CDN key: " key upload_archive_to_cdn "$archive" "$key" ;; 3) read -p "Archive name: " name read -p "Local path: " path download_from_cdn "$name" "$path" ;; 4) setup_cdn_sync ;; esac}
cdn_archive_integrationAutomated Backup System:
Section titled “Automated Backup System:”# Automated backup system using upload-archiveautomated_backup_system() { echo "=== Automated Archive Backup System ==="
# Configuration BACKUP_ROOT="/srv/backups/git" RETENTION_COUNT=30 COMPRESSION_LEVEL=9
# Create backup archive create_backup_archive() { local repo_name="$1" local repo_url="$2"
echo "Creating backup archive for $repo_name..."
backup_dir="$BACKUP_ROOT/$repo_name" mkdir -p "$backup_dir"
# Generate backup filename timestamp=$(date +%Y%m%d-%H%M%S) backup_file="$backup_dir/backup-$timestamp.tar.gz"
# Create archive from repository git archive --remote="$repo_url" \ --format=tar.gz \ --prefix="$repo_name-$timestamp/" \ --output="$backup_file" \ HEAD
if [ $? -eq 0 ]; then echo "✓ Backup created: $backup_file"
# Create backup manifest cat > "${backup_file}.manifest" << EOFRepository: $repo_nameURL: $repo_urlTimestamp: $timestampSize: $(stat -f%z "$backup_file" 2>/dev/null || stat -c%s "$backup_file")SHA256: $(sha256sum "$backup_file" | cut -d' ' -f1)HEAD: $(git ls-remote "$repo_url" HEAD | cut -f1)EOF
# Cleanup old backups cleanup_old_backups "$backup_dir"
return 0 else echo "✗ Backup creation failed" return 1 fi }
# Restore from backup restore_from_backup() { local backup_file="$1" local restore_path="$2"
echo "Restoring from backup: $backup_file"
if [ ! -f "$backup_file" ]; then echo "Backup file not found" return 1 fi
# Verify backup integrity manifest="${backup_file}.manifest" if [ -f "$manifest" ]; then expected_sha=$(grep "SHA256:" "$manifest" | cut -d' ' -f2) actual_sha=$(sha256sum "$backup_file" | cut -d' ' -f1)
if [ "$expected_sha" != "$actual_sha" ]; then echo "Backup integrity check failed" return 1 fi fi
# Extract backup mkdir -p "$restore_path" tar xzf "$backup_file" -C "$restore_path"
# Initialize as git repository cd "$restore_path"/* git init git add . git commit -m "Restored from backup $(basename "$backup_file")"
echo "✓ Repository restored to: $restore_path" }
# Cleanup old backups cleanup_old_backups() { local backup_dir="$1"
echo "Cleaning up old backups in $backup_dir..."
# Keep only recent backups ls -t "$backup_dir"/backup-*.tar.gz 2>/dev/null | tail -n +$((RETENTION_COUNT + 1)) | while read -r old_backup; do echo "Removing old backup: $(basename "$old_backup")" rm -f "$old_backup" "${old_backup}.manifest" done }
# Automated backup scheduler setup_backup_scheduler() { local repos_file="$1"
echo "Setting up automated backup scheduler..."
cat > /usr/local/bin/git-backup.sh << EOF#!/bin/bash# Automated Git repository backups
REPOS_FILE="$repos_file"BACKUP_ROOT="$BACKUP_ROOT"
while read -r repo_name repo_url; do [ "\${repo_name:0:1}" = "#" ] && continue # Skip comments
echo "Backing up \$repo_name..." create_backup_archive "\$repo_name" "\$repo_url"done < "\$REPOS_FILE"
echo "Backup cycle complete at \$(date)"EOF
chmod +x /usr/local/bin/git-backup.sh
# Setup cron for daily backups (crontab -l ; echo "0 2 * * * /usr/local/bin/git-backup.sh") | crontab -
echo "Automated backup scheduler configured" }
# Interactive backup management echo "Automated Backup System Options:" echo "1. Create backup archive" echo "2. Restore from backup" echo "3. Setup backup scheduler"
read -p "Select option (1-3): " option
case "$option" in 1) read -p "Repository name: " name read -p "Repository URL: " url create_backup_archive "$name" "$url" ;; 2) read -p "Backup file: " backup read -p "Restore path: " path restore_from_backup "$backup" "$path" ;; 3) read -p "Repositories file: " repos setup_backup_scheduler "$repos" ;; esac}
automated_backup_system