Understanding history backup
What does that mean backups with history? It means that backups have been done in the previous period (eg, yesterday, 2 days ago, last week, etc.) are not deleted but remain stored. Generally, in the world of IT standards in the corporate backup is to store the latest data backups, 7 daily backups history, history 4 weekly backups, and some historical monthly backups (eg: 12). So that the total period of history is 12 months.
Backup with history is more useful because it can help save data corrupted or deleted but not immediately known. For example you may unknowingly delete a directory that is felt no longer needed because the project is completed. It turned out a month later, the project continued. If you do not store historical backups, so after you delete the directory, daily backups on the run following the latest backup any data will no longer contain the directory data, since the latest backup data follow the latest developments.
Or, say you are using or developing a new version of the program. Apparently after a while just know that these programs contain bugs and little by little destroy or remove a small portion of data. To be sure not to lose valuable data, you want to compare current data with the condition 6 months ago.
Or, you want to go back to the configuration state a week ago before you reinstall the OS. If you backup with history on the directory / etc, it is easy to do. Historical data can simply take the appropriate backups.
Save disk space
Backup with history raises a problem: a swollen disk space. Because we keep a full copy backup every day. Thus, the amount of 100MB of data per week would require backup space 7x100MB = 700MB. Within a month requires 30x100MB = 3GB. Within a year requires 365x100MB = 36GB! What if the data you want backed up even more? How to save disk space?
There are two ways that can be done.
Hard links. The first is by using the 'hard links'. In most file systems, including filesystems are often used in Unix / Linux, a directory entry allowed to refer to the location data (file content) the same as other directory entries. In other words, a copy of the file can have more than one name / location. Example:
$ Ln my data other_location / mydata
This command makes a hard link to my data to lokasi_lain / datasaya. This means that the number of copies of only 1, but referred to two different file name. If we modify one, then both will be modified because in fact they are the same.
Ls-l command to show the number of links that refer to a directory entry:
$ Ls-l my data
-rw-r - r - 2 steven steven my data 149 2007-09-04 17:45
Figure 2 shows that there are two links that refer to this file. If we remove one (or we 'unlink', this is why the delete or delete command in Unix are also called unlink):
$ Rm other_location / mydata
then the number of hard links will be reduced:
$ Ls-l my data
-rw-r - r - 1 steven steven my data 149 2007-09-04 17:45
A file will not be freed until all disknya the directory entries that refer to him missing (in other words, before the hard link to 0). When all was deleted:
$ Rm my data
there was no entry that refers to the contents of the file, and the file becomes inaccessible, and that means the block is occupied by the contents of a file declared free and can be used for other content.
With hard links, we can mengirit disk space. The files that do not change from one version of the backup to the next backup version is the hard links to each other.
Level daily, weekly, monthly. The second way in mengirit disk space is to not save all the history on a daily level. Generally data is already getting old increasingly irrelevant, and should not be stored too 'tight' again. So, to store data over a year, generally not too needed to save 365 pieces daily backups, but just 7 daily backups, 4 weekly backups, and 12 monthly backups. Period covered history remain 12 months, but the number of copies required history is 7 +4 +12 = 23 units. If you just want to keep the 3-month period, the number of copies required history 7 +4 +3 = 14 fruit.
Bagimana how to rotate these levels, if we perform daily backups? Below is the illustration for backups s.d. 3 months. Each number represents the day, the smaller is getting old. To obtain the latest backup (ie the newly done today or at least slow yesterday), you can take the greatest number. Example: in the day-to-1, the latest backup is numbered 1. On day 10, 10 numbered backups, etc.. In contrast, the oldest backup is the smallest number. An asterisk (*) represents a candidate who will be moved to the next level backup.
Day 1:
Daily backups: 1
backup downloads: (not applicable)
monthly backup: (none exists)
Day 2:
Daily backups: 2 1
backup downloads: (not applicable)
monthly backup: (none exists)
Day 7:
Daily backups: 7 6 5 4 3 2 1
backup downloads: (not applicable)
monthly backup: (none exists)
Day 8:
Daily backups: 8 * 7 6 5 4 3 2 1 <- moved to weekly
backup downloads: 1
monthly backup: (none exists)
Day 9:
Daily backups: 9 8 * 7 6 5 4 3 2 <- deleted
backup downloads: 1
monthly backup: (none exists)
Day 15:
Daily backups: 15 * 14 13 12 11 10 9 8 * <- moved to weekly
backup downloads: 8 1
monthly backup: (none exists)
Day 16:
Daily backups: 16 15 * 14 13 12 11 10 9 <- deleted
backup downloads: 8 1
monthly backup: (none exists)
Day 22:
Daily backups: 22 * 21 20 19 18 17 16 15 * <- moved to weekly
backup downloads: 15 8 1
monthly backup: (none exists)
Day 29:
Daily backups: 29 * 28 27 26 25 24 23 22 * <- moved to weekly
backup downloads: 22 15 8 1
monthly backup: (none exists)
Day 36:
Daily backups: 36 * 35 34 33 32 31 30 29 * <- moved to weekly
backup downloads: 29 * 22 15 8 1 <- moved to a monthly
monthly backups: 1
Day 43:
Daily backups: 43 * 42 41 40 39 38 37 36 * <- moved to weekly
backup downloads: 36 29 * 22 15 8 <- deleted
monthly backups: 1
Day 50:
Daily backups: 50 * 49 48 47 46 45 44 43 * <- moved to weekly
backup downloads: 43 36 29 * 22 15 <- deleted
monthly backups: 1
Day 57:
Daily backups: 57 * 56 55 54 53 52 51 50 * <- moved to weekly
backup downloads: 50 43 36 29 * 22 <- deleted
monthly backups: 1
Day 64:
Daily backups: 64 * 63 62 61 60 59 58 57 * <- moved to weekly
backup downloads: 57 * 50 43 36 29 * <- moved to a monthly
monthly backup: 29 1
Day 92:
Daily backups: 92 * 91 90 89 88 87 86 85 * <- moved to weekly
backup downloads: 85 * 78 71 64 57 * <- moved to a monthly
monthly backups: 57 29 1
Day 120:
Daily backup: 120 * 119 118 117 116 115 114 113 * <- moved to weekly
backup downloads: 113 * 106 * 99 92 85 <- moved to a monthly
monthly backups: 85 57 29 1 <- removed
Backup program-with-history
Here is the implementation of the system with historical daily backups using rsync, in about 100 lines of Perl. This script can be found also online at http://people.masterwebnet.com/steven/files/backup-with-history.
1 | #! / Usr / bin / perl-w
|
3 | use strict;
4 | use CWD qw (abs_path);
5 | use POSIX;
6 | use Time:: Local;
|
8 | # --- config
9 | my $ Histories = [-7, 4, 3];
|
11 | # --- subs
12 | sub ESC (
13 | local $ _ = shift;
14 | s /'/'"'"'/ g;
15 | "'$_'";
16 |)
|
18 | # --- main
19 | @ ARGV == 2 or die "Usage: $ 0
20 | my ($ src, $ dst) = @ ARGV;
21 | for ($ src, $ dst) (s !/$!!}
22 | (-d $ src) or die "Src must be an existing directory: $ src \ n";
23 | system "mkdir-p". ESC ( "$ dst / current");
24 | (-d "$ dst / current") or die "Failed to make creating $ dst / current \ n";
|
26 | print "Creating new backup $ src to $ dst / ... \ n";
27 | system "nice-n19 rsync-av - del - force"
28 |. "- link-dest"
29 |. ESC (abs_path ( "$ dst"). "/ current"). ""
30 |. ESC ( "$ src /"). ""
31 |. ESC ( "$ dst / .work /"). "\ n";
32 | warn "rsync command is not successful, you should check \ n" if $?;
|
34 | chdir ($ dst) or die "Can not chdir to $ dst: $! \ N";
35 | system "touch. Current.timestamp";
36 | my $ now = time;
37 | my @ st = stat ( ". Current.timestamp");
38 | my $ tstamp
39 | = POSIX:: strftime ( "% Y-% m-% d \ @% H:% M:% S +00", gmtime ($ st [9] | | $ now));
40 | rmdir "current" or rename "current", "Hist. $ Tstamp";
41 | rename ". Work", "current";
|
43 | print "Deleting backup history that has for too long ... \ n";
44 | for my $ level (1 .. @ $ Histories) (
45 | my $ is_highest_level = $ level == @ $ Histories;
46 | my $ prefix = "Hist". ($ Level == 1?: $ Level);
47 | my $ prefix_next_level = "Hist". ($ Level + 1);
48 | my $ n = $ Histories-> [$ level - 1];
49 | my $ moved = 0;
|
51 | if ($ n> 0) (
52 | print "Only $ n fruit store historical backups L $ level ... \ n";
53 | my @ f = reverse sort grep (! / \. Work $ /) glob "$ prefix .*";
54 | my $ any_tagged = (grep (/ t $ /) @ f)? 1: 0;
55 | for my $ f (@ f [$ n .. @ f - 1]) (
56 | my ($ st, $ tagged) = $ f = ~ /[^.]+ \ .(.+?)( t )?$/;
57 | my $ f2 = "$ prefix_next_level. $ St";
58 | if (! $ Is_highest_level
59 | & &! $ Moved
60 | & & ($ tagged | |! $ Any_tagged))
61 | (
62 | print "Moving the historical level backups: $ f -> $ f2 \ n";
63 | system "mv". ESC ($ f). "". ESC ($ f2);
64 | $ moved + +;
65 | if ($ f ne $ f [0]) (
66 | my $ e3 = ESC ($ f [0]);
67 | system "mv $ e3 $ (e3) t";
68 |)
69 |)
70 | else (
71 | print "Deleting backup history: $ f ... \ n";
72 | system "nice-n19 rm-rf". ESC ($ f);
73 |)
74 |)
75 |)
76 | else (
77 | $ n = - $ n;
78 | print "Just keep a backup history sd L $ level $ n day ... \ n";
79 | my @ f = reverse sort grep (! / \. Work $ /) glob "$ prefix .*";
80 | my $ any_tagged = (grep (/ t $ /) @ f)? 1: 0;
81 | for my $ f (@ f) (
82 | my ($ st, $ tagged) = $ f = ~ /[^.]+ \ .(.+?)( t )?$/;
83 | my $ f2 = "$ prefix_next_level. $ St";
84 | my $ t;
85 | $ st = ~ / (\ d \ d \ d \ d) - (\ d \ d) - (\ d \ d) \ @ (\ d \ d): (\ d \ d): (\ d \ d) \ +00 /;
86 | $ t = timegm ($ 6, $ 5, $ 4, $ 3, $ 2 - 1, $ 1) if $ 1;
87 | $ q & & $ t or do (
88 | print "History of backup formats: $ f, ignored \ n";
89 | next;
90 |);
91 | if ($ t> $ now) (
92 | print "History backups in the future? $ F, ignored \ n";
93 | next;
94 |)
95 | my $ delta = ($ now - $ t) / 86400;
96 | if ($ delta> $ n) (
97 | if (! $ Is_highest_level
98 | & &! $ Moved
99 | & & ($ tagged | |! $ Any_tagged))
100 | (
101 | print "Moving the historical level backups: $ f -> $ f2 \ n";
102 | system "mv". ESC ($ f). "". ESC ($ f2);
103 | $ moved + +;
104 | if ($ f ne $ f [0]) (
105 | my $ e3 = ESC ($ f [0]);
106 | system "mv $ e3 $ (e3) t";
107 |)
108 |)
109 | else (
110 | print "Deleting backup history: $ f ... \ n";
111 | system "nice-n19 rm-rf". ESC ($ f);
112 |)
113 |)
114 |)
115 |)
116 |)
|
118 |
How to use this script:
$ Backup-with-history DIR DIRBACKUP
DIR is the directory you want backed up, eg / home / steven. DIRBACKUP is the desired backup location, eg: / backup / home-steven. Better / backup is on a different hard drive, because the local backup purposes is to avoid loss of data due to hard drive damage. Backup process is complete, the script will form DIRBACKUP directory / current / which contains the latest backup, and a series DIRBACKUP directory / Hist .*, hist2 .*, and which contains hist3 history .* daily backups, weekly, monthly.
To perform a backup process itself is essentially the line 27-31, which relies on rsync. Rsync is going to handle the transfer and synchronization with directory structure of each file metadata (permissions, ownership, date, etc.). With the - link-dest, rsync will also compare the data with the last backup (current /) and do hardlinking if possible. The rest of the script is handling most of the rotation of backup history.
This script is simple. You can add that can be configured (eg: option - exclude rsync, etc.). Good luck!
Tidak ada komentar:
Posting Komentar