VPSファイルのGoogle Driveへの自動バックアップ

% rsync -a <source> <destination>

% rsync -a <source> --compare-dest=<full_backup> <diff_backup>

% sudo apt install rclone

% curl https://rclone.org/install.sh | sudo bash
% rclone --version
rclone v1.51.0
- os/arch: linux/amd64
- go version: go1.13.7

% rclone config
2020/03/10 08:36:03 NOTICE: Config file "~/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> gdrive
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / 1Fichier
   \ "fichier"
 2 / Alias for an existing remote
   \ "alias"
...
12 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
13 / Google Drive
   \ "drive"
14 / Google Photos
   \ "google photos"
15 / Hubic
   \ "hubic"
...
Storage> 13  # Google Driveの13を選択
** See help for drive backend at: https://rclone.org/drive/ **

Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> [Enter] # 空欄でOK
Google Application Client Secret
Setting your own is recommended.
Enter a string value. Press Enter for the default ("").
client_secret> [Enter] # 空欄でOK
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Full access all files, excluding Application Data Folder.
   \ "drive"
 2 / Read-only access to file metadata and file contents.
   \ "drive.readonly"
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ "drive.file"
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ "drive.appfolder"
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ "drive.metadata.readonly"
scope> 1 # コピーと削除も行うので1
ID of the root folder
Leave blank normally.

Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.

Note that if this is blank, the first time rclone runs it will fill it
in with the ID of the root folder.

Enter a string value. Press Enter for the default ("").
root_folder_id> [Enter] # 空欄でOK
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press Enter for the default ("").
service_account_file> [Enter] # 空欄でOK
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n # NoでOK
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n # リモート上でブラウザを立ち上げれない場合はNo
Please go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=20239847832903847.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aeauth%3A2.0%3Aeob&respomse_typo=code&scope=htttps%3asdf2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=rak9ztM0a8da908d7fA
Log in and authorize rclone for access
Enter verification code> ************** # 上記URLでアクセス許可後に表示されたコードをコピー
Configure this as a team drive?
y) Yes
n) No (default)
y/n> n # TeamドライブにアクセスしなければNoでOK
--------------------
[gdrive]
type = drive
scope = drive
token = {"access_token":"*********","token_type":"****","refresh_token":"******","expiry":"****"}
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y # Yesでconfigを~/.config/rclone/rclone.configに保存

% rclone lsd gdrive:

% rclone copy local_file gdrive:remote_folder

% rclone delete gdrive:remote_file

% sudo apt install p7z-full

% mkdir /var/tmp/backup # バックアップ作業フォルダ
% cd /var/tmp/backup
% git clone https://github.com/udon-code/vps_utils.git vps_utils
% ./vps_utils/scripts/backup_to_cloud.py -h
usage: backup_to_cloud.py [-h] -s path [-r <rclone path>] [-d folder] [-i]
                          [-P password] [-z] [--nocompress] [--mysql] [-n]
                          [-v] [-q] [--clean_local_after days] [--cleanall]
                          [--clean_remote_after days]

optional arguments:
  -h, --help            show this help message and exit
  -s path, --src path   Source folder(s). Can be specified multiple times
  -r <rclone path>, --remote <rclone path>
                        Remote path (<rclone remote name>:<remote folder>)
                        (ex. gdrive:backup)
  -d folder, --dst folder
                        Output folder, created a temporary one if omitted
  -i, --incremental     Backup differential from the latest full+diff backup.
                        (note: mySql backup doesnt support differential back)
  -P password, --password password
                        If specified, encrypt zip file with given password
  -z, --zip             Use zip instead of zip (default is 7z)
  --nocompress          Skip compression
  --mysql               Save mysql database (may require root privilege)
  -n, --noexec          dry run
  -v, --verbose         Verbose mode
  -q, --quiet           Quiet mode
  --clean_local_after days
                        Delete old backup files older than specified days(In
                        incremental mode, delete old *_diff after a full
                        backup before specified days)(In fullbackup mode,
                        delete old full backup before specified days)(Delete
                        compressed file when --remote presents)
  --cleanall            Delete everything but the latest archive
  --clean_remote_after days
                        Delete remote file older than specified days

% vi weekly_backup.sh
#!/bin/bash

BACKUP_CMD=`dirname $0`/vps_utils/scripts/backup_to_cloud.py
LOCAL_DST=/var/tmp/backup/work
REMOTE_DST="gdrive:Backup"

echo ""
echo "-----------------------------------"
echo "Start backup ... " `date`

set -x
${BACKUP_CMD} -v \
  -d ${LOCAL_DST} \
  -r ${REMOTE_DST} \
  -P hogehoge \
  --clean_local_after 1 \
  --clean_remote_after 90 \
  -s /var/www \
  -s /home \

set +x
echo ""
echo "End backup ... " `date`
echo "------------------------------------"

% vi daily_backup.sh
#!/bin/bash

BACKUP_CMD=`dirname $0`/vps_utils/scripts/backup_to_cloud.py
LOCAL_DST=/var/tmp/backup/work
REMOTE_DST="gdrive:Backup"

echo ""
echo "-----------------------------------"
echo "Start backup ... " `date`

set -x
${BACKUP_CMD} -v -i \
  -d ${LOCAL_DST} \
  -r ${REMOTE_DST} \
  -P hogehoge \
  --clean_local_after 1 \
  --clean_remote_after 7 \
  -s /var/www \
  -s /home \

set +x
echo ""
echo "End backup ... " `date`
echo "------------------------------------"

% ./weekly_backup.sh     # フルバックアップ
% ./daily_backup.sh  # 差分バックアップ

% crontab -e
10 1 * * 0   /var/tmp/backup/weekly_backup.sh >> /var/tmp/backup/weekly.log
10 1 * * 1-6 /var/tmp/backup/daily_backup.sh >> /var/tmp/backup/daily.log