AzBackup-Restore
Author's notes...
--include-pattern and --exclude-pattern switches, and many others. The azbackup command does however provide the /xfp and /ifp switches which allow filename matching using regular expressions (if that's useful in certain scenarios).
Microsoft has published the sophisticated azcopy command for performing data transfer operations on Azure blobs, tables and files. Despite the power of the azcopy command, whenever I need to use it, I forget where it is, and I'm not sure if it's the latest version, and I always have to study the help to work out the exact parameter syntax required for the simplest operations.
As a software developer, the most common Azure operations I perform are backups from local files to Azure Blobs, with occasional restores. Most importantly for backups, I need rich filtering capabilities so I can skip folders and files which match a variety of conditions on their name, size and extension. The azcopy command unfortunately provides only the /Pattern switch which allows rather constrained filtering with a single value. It was the complexity of the azcopy command combined with its limited filtering capability that eventually led me to write my own lightweight command line utilities for performing file-blob backups and restores.
This repository contains a solution with two C# projects named AzBackup and AzRestore which are command line utilities for transferring files to-and-from Azure Blob Storage. The backup command provides rich filtering options via the /xe, /xd and /xfp switches.
azbackup
OVERVIEW
A command line utility for backing-up files to Azure Blob Storage.
The command has features intended to be helpful for typical software
developers who have complicated requirements to filter out certain files
and folders from backup processing.
SYNTAX
azbackup <source> <target>
[ /dirprefix <prefix> ]
[ /an <account-name> /ak <account-key> ]
[ /s /v /tc /synch /mock /maxkb <kb> /de ]
[ /skiplong /biggest ]
[ /from yyyy-mm-dd ]
[ /xe <file-extension> ... ]
[ /xd <directory-name> ... ]
[ /xfp <file-name-pattern> ... ]
[ /ifp <file-name-pattern> ... ]
[ /log <filename> /logappend ]
[ /nolist ]
[ /mime <url> ]
PARAMETERS
source ...... Source directory for backup.
target ...... Target Azure container name (see notes).
/dirprefix .. A directory prefix for all uploaded Blobs.
/an ......... Azure account name. Default is %AZNAME%.
/ak ......... Azure account key. Default is %AZKEY%.
/s .......... Recursive backup of sub directories.
/v .......... Display verbose processing trace.
/tc ......... Time compare files and blobs and only backup files with
timestamps newer than the blob.
/nometa ..... Do not set blob metadata.
/synch ...... Synchronize the blobs with the files by deleting blobs
that are orphaned and have no corresponding file.
/mock ....... Mock run without altering any files or blobs.
/maxkb ...... Skip files larger than the size in KiB.
/de ......... Dump error details.
/skiplong ... Skip long file names (longer than 260 characters).
/biggest .... Display up to the 10 biggest input files (for debugging).
/from ....... From date for source file modified times. Only files modified
on or after this date will be backed up. Format: yyyy-MM-dd.
/xe ......... Exclude file extensions (without the . prefix).
/xd ......... Exclude directory names.
/xfp ........ Exclude file name patterns (Regex match).
/ifp ........ Include file name patterns (Regex match).
/log ........ Log file.
/logappend .. Append to log file.
/nolist ..... Do not list backed up files or synch deleted blobs.
/mime ....... Specifies the URL of an XML document that maps file extensions
to MIME types for uploaded files. The default document is
loaded from the author's reference file at:
https://orthoprog.blob.core.windows.net/reference/mime-types.xml
NOTES
* All parameters and exclude comparisons are case insensitive.
* The target value is converted to lower-case with illegal container name
characters removed. The special name $web for static websites is accepted
unchanged.
* See: https://dev.azure.com/orthogonal/AzBackup-Restore
* Version 1.2.4.0 (1.2.4.0) Build 2020-08-12 16:08 GMT+10
RETURN CODES
0 ........... Success
1 ........... Help display only
2 ........... Parameter syntax error(s)
3 ........... Unexpected processing exception
Here is a sample of a batch file that is used to backup one of my main development folders. The /xe, /xd and /xfp switches are used to exclude a variety of files and folders.
set XD=bin obj lib output Debug Release packages .hg .vs .git AppPackages ClientBin References
set XE=user suo msi temp zip xap edb chk jrs log mp3 7z ini dll exe nupkg
set XFP=AlbumArt Thumbs\.db Inv.*\.pdf
azbackup C:\dev dev /s /v /tc /synch /xd %XD% /xe %XE% /xfp %XFP%
azrestore
OVERVIEW
A simple command line utility to restore files from Azure Blob Storage.
SYNTAX
azrestore <source> <target>
[ /an <account-name> /ak <account-key> ]
[ /prefix <prefix> ... ]
[ /s /v /tc ]
PARAMETERS
source ...... Source Azure container name.
target ...... Target directory for restore.
prefix ...... Only restore blobs with names matching the prefix(es).
/an ......... Azure account name. Default is %AZNAME%.
/ak ......... Azure account key. Default is %AZKEY%.
/flat ....... Flatten restored blobs into the target folder by removing
any path prefixes.
/v .......... Display verbose processing trace.
/tc ......... Time compare blobs and files and only restore blobs with
timestamps newer than the file.
NOTES
* All parameters and name comparisons are case insensitive.
* https://bitbucket.org/orthoprog/azcmd/wiki/Home
* Version 1.1.0.0 (1.1.0.0) Build 2018-07-19 21:33 GMT+10
RETURN CODES
0 ........... Success
1 ........... Help display only
2 ........... Parameter syntax error(s)
3 ........... Unexpected processing exception
