Differences

This shows you the differences between two versions of the page.

--- rsync_backup [2022/09/09 12:12] – Minor fixes for better understanding. grandmabetty
+++ rsync_backup [2023/09/15 08:37] (current) – Correction: Adjusted links from german to english language grandmabetty
@@ Line 1: / Line 1: @@
 ====== rsync backup to a Synology NAS ======
-This tutorial covers how to backup (manually or automatically) the userdata partition of your Batocera machine to a [[https://www.synology.com/en-us/dsm/solution/what_is_nas|Synology NAS]]. This allows you to recover your data to a previous state in case of data loss.
+This tutorial covers how to backup the userdata partition of your Batocera machine to a [[https://www.synology.com/en-us/dsm/solution/what_is_nas|Synology NAS]]. This allows you to recover your data easily to a previous state in case of data loss.
-The tutorial is based on a Synology NAS which is running on Synology's DSM **version 6.x** or later.
+===== Backup via Synology Active Backup For Business (ABB) =====
-<WRAP center round info>
+Since the release of [[https://www.synology.com/en-us/dsm/feature/active-backup-business/pc|Synology Active Backup For Business]], backing up a Batocera system via //rsync// became much more convenient than ever before. Please check if your NAS model supports //Synology ABB// on this [[https://www.synology.com/en-us/dsm/packages/ActiveBackup|link]].
-This tutorial assumes you have already set up the following:
-  * Static IP address and/or hostname for your Synology NAS (you can do this via your DHCP/DNS server (mostly your router))
-  * Static IP address and/or hostname for your Batocera system (you can do this via your DHCP/DNS server (mostly your router))
-  * Optional: If you want to set up an automated backup (which is also covered in this tutorial), your Batocera system has to be capable of being started remotely (e.g. capable of [[:wake_on_lan|Wake on LAN (WoL)]]).
-</WRAP>
-<WRAP center round info>
+==== Tutorial ====
-There are mainly two different network protocols on how to backup data to as Synology NAS: [[https://en.wikipedia.org/wiki/Server_Message_Block|SMB]] (**S**erver **M**essage **B**lock) and [[https://en.wikipedia.org/wiki/Network_File_System|NFS]] (**N**etwork **F**ile **S**ystem). Both are being covered separately here. In a nutshell, both network protocols come with its adavantages/disadvantages, which you should think of while considering on how to backup your data.
-  * **SMB**: The advantage here is that SMB is easy to understand and easy to set up. It comes with standard user/password authentication. The disadvantages of SMB is that it works best within Windows environments and in some circumstances it has its compatibility problems with Linux symlinks, which according to Batocera are being created mostly when setting up Windows games on WINE. SMB is also known to be less efficient/performant than NFS.
-  * **NFS**: The advantage on NFS is that it has no problems with Linux symlinks. It is also more performant than SMB. For security, NFS provides IP based security and authentication security on shared folders and this is where NFS comes into its disadvantages: Authentication security is a pain to set up for the average user, but fortunately it is an optional thing. Assuming your home network is secure, you should not worry about security too much in that case as you can still use the IP based security anyway so that only the Batocera system can access the according shared backup folder on the NAS and is really limited to that single shared folder.
-In summary:
+=== Backup ===
-  * If you are not planning to install or backup Windows games on WINE and network security is a big thing, SMB is the way to go. \\
-  * If you want to use the full spectrum of current and future emulators, have the possibility to back everything up without issues and network security is not a big deal, NFS is the way to go.
-</WRAP>
-===== Preparing the NAS: General settings =====
+  - Install //Synology ABB// via //Synology Package Manager//
+  - Start //Synology ABB// (if you are installing //Synology ABB// from scratch, you have register the application via the [[https://account.synology.com|Synology online account portal]], don't worry, it's completely free!)
+  - Navigate to: //File Server// → //Add Server//
+  - Choose //rsync server//: \\ [{{:useanasbackup10.png?400|Add a server}}] Click on //Next//
+  - For the following input fields add the following values:\\ //**Server Address:**// ''<my_Batocera_IPAddress_or_hostname>'' \\ **Connection Mode:** Select ''rsync shell mode via SSH'' \\ **Port:** Insert your Batocera instance's SSH port (Default: ''22'') \\ **Account:** ''root'' \\ **Auth Policy:** Choose ''By password'' (or alternatively ''By SSH key'' if you have set up [[access_the_batocera_via_ssh#password-less_authentication|]] from your NAS to your Batocera instance before) \\ **Password:** ''<myBatoceraRootPassword>'' \\ [{{:useanasbackup11.png?400|Insert server data}}] Click on //Done//
+  - Now highlight the //rsync// instance you have just created and click on //Create Task//, then select your preferred backup mode: \\ [{{:useanasbackup12.png?400|Select preferred backup mode}}] Click on //Next//
+  - Select all folders from the Batocera filesystem you want to backup (If you are unsure: Select the whole ''userdata'' folder but no other folders): \\ [{{:useanasbackup13.png?400|Select preferred folders to backup}}] Click on //Next//
+  - For the following input fields add the following values:\\ **Task name:** ''<myCustomTaskName>'' \\ **Local path:** Click on //Browse// and select the preferred path where the backup should be stored on your NAS: \\ [{{:useanasbackup14.png?400|Additional task settings}}] Click on //Next// and then click on //Done//
+  - When asked, confirm if you want to start the backup immediately. If you want to start the backup later manually, highlight the according //rsync// server, click on //Task List// and then click on //Back up//.
-  - Under **SSH**, navigate to **Control Panel** → **Terminal & SNMP** and check "Enable SSH service". Then, for **Port** set the value ''22''.\\ [{{:useanasbackup05.png?500|Enable SSH}}]
-  - Under **rsync**, navigate to **Control Panel** → **File Services** → **rsync** and check "Enable rsync service". Then, for **SSH encryption port** set the value ''22''.\\ [{{:useanasbackup02.png?600|Enable rsync}}]<WRAP center round info> You don't have to worry that port 22 is used by SSH //and// rsync at the same time. This is a common setup and the functionality of each service will not be interfered/disrupted in any way!</WRAP>
-  - [[https://kb.synology.com/en-us/DSM/help/DSM/AdminCenter/file_share_create?version=6|Create a shared folder]] where the Batocera backup data can be stored to. For this example, the folder will be named ''batoceraBackup''.
-  - [[https://kb.synology.com/en-us/DSM/help/DSM/AdminCenter/connection_security_firewall?version=6|Create a firewall rule]] on the Synology NAS so that the Batocera machine can send its data over the network to the Synology NAS. The rule must allow the following two service:
-    * ''Encrypted terminal service (includes encrypted rsync and SFTP)'' [{{:useanasbackup07.png?800|Firewall rule(s)}}]
-  - In addition it is also **strongly recommended** to limit the firewall rule to allow only the static IP address of the Batocera system (that's why it is important to set a static IP for the Batocera system).
-==== Preparing the NAS: Settings for SMB ====
+=== Restore ===
-  - Enable the following services on the Synology NAS via the Synology DSM web GUI as follows:
+<WRAP left round tip 100%>
-    * Navigate to **Control Panel** → **File Services** → **SMB** and check "Enable SMB service".\\ [{{:useanasbackup01.png?300|Enable SMB}}]
+In case you are restoring your data based on a fresh Batocera installation, before restoring, make sure bluetooth service is stopped by executing the following command on your Batocera system: ''/etc/init.d/S32bluetooth stop''. This makes sure your bluetooth controllers are being recognized and working as before. If you don't do this, then you have to set up your bluetooth controllers from scratch again.
-  - [[https://kb.synology.com/en-us/DSM/help/DSM/AdminCenter/file_user_create?version=6|A user (without administrative privileges) will need to be created on the Synology NAS]]. For this example the user will be named ''backup''.
-  - Enable the **read/write** data permission for the user ''backup'' on the user permissions page of the according shared folder "BatoceraBackup":\\ [{{:useanasbackup03.png?900|User permissions}}]
-  - Edit the firewall rule we have just created before on the Synology NAS. In addition to the rule we created earlier, the rule must allow the following service:
-    * ''Windows file server'' \\ [{{:useanasbackup04.png?1000|Firewall rule(s)}}]
-==== Preparing the NAS: Settings for NFS ====
-  - Enable the following services on the Synology NAS via the Synology DSM web GUI as follows:
-    * Navigate to **Control Panel** → **File Services** → **NFS** and check "Enable NFS service" and for "Maximum NFS protocol" choose "NFSv4".\\ [{{:useanasbackup06.png?600|Enable NFS}}]
-  - Now create an according NFS rule as follows: \\ Edit the ''batocerBackup'' shared folder's settings we created earlier and navigate to  **NFS Permissions**. Click on **Create** and for **Hostname or IP** insert the hostname/IP-address of your Batocera system. Leave the other settings as they are: [{{:useanasbackup08.png?600|Shared folder's NFS rule}}] Click **Save** twice.
-  - Edit the firewall rule we have just created before on the Synology NAS. In addition to the rule we created earlier, the rule must allow the following service:
-    * ''Mac/Linux file server'' \\ [{{:useanasbackup09.png?600|Firewall rule(s)}}]
-===== Preparing Batocera: =====
-Now [[https://wiki.batocera.org/access_the_batocera_via_ssh|SSH]] into your Batocera system and execute the following command to create a custom ''scripts'' folder:
-<code>mkdir -p /userdata/scripts</code>
-<WRAP center round tip>
-On the following steps we will cover on how to create three scripts here:
-  * ''romsBackup.sh'' → Which will backup your ''/userdata/roms'' folder, excluding the folders ''/userdata/roms/steam'' and ''/userdata/roms/windows_installers'' as it does not make much sense to backup those folders. When using SMB, to avoid Linux symlink issues as mentioned above, we will also exclude the ''/userdata/roms/windows'' folder. If you want to backup those folders anyway, feel free to adjust the script accordingly.
-  * ''userdataBackup.sh'' → Which will backup your ''/userdata'' folder, excluding the ''/userdata/roms'' folder as it //may// take a very long time to finish, depending on how many games you have to copy over. This way you can save time in case you want to run the scripts manually for whatever reason and backup the rest of your userdata independent from your games. Note that we will also exclude the ''/userdata/saves/flatpak'' folder as empirically there are too many issues when importing it back from a backup. If you want to backup those folders anyway, feel free to adjust the script accordingly.
-  * ''systemBackup.sh'' → Which will only execute the two scripts ''romsBackup.sh'' and ''userdataBackup.sh''.
-We are separating the first two scripts instead of putting them in a single script so you can execute them manually whenever you want to, indepented from the automation steps we are covering below. The third script is exactly for those automation steps.
 </WRAP>
-<WRAP center round important>
+If you want to restore a single file or a full backup, do so by using the //Active Backup For Business (Recovery) Portal// which was installed automatically additionally on your //Synology NAS// when installing //Synology ABB//. It's pretty straightforward: Highlight the according folder(s)/file(s), right-click and choose //Restore// (don't forget to have your Batocera system turned on!): \\ [{{:useanasbackup15.png?400|Restore a backup}}] After successful restore process, reboot your Batocera system.
-**Beware:** The rsync command's ''--delete'' option within the following script ''romsBackup.sh'' does delete every file on the target side which is not available on the source side. We are using this option due to keep the target side consistent with the source side (for example if you have deleted some games on the source side and you want them being deleted on the target side too). If you don't want this to happen then just remove the ''--delete'' option on the script.
-Also note that rsync by default does synchronize only files that have been changed/added since the last synchronization. This will prevent unnecessary syncing for files that have already been synced in the past.
-</WRAP>
-<WRAP center round important>
-For the following mounting steps within the scripts, please note that unlike SMB, for NFS you cannot just mount a remote shared folder as ''<nowiki>//</nowiki><my_IP_address/hostname>/<my_Shared_Folder>''. Instead you will have to insert the full path to the remote shared folder. On a Synology NAS it's always ''<my_IP_address/hostname>:/volume<x>/<my_Shared_Folder>''. The value ''<x>'' in ''volume<x>'' is dependent on the volume number on the Synology NAS. If you have set up only a single volume on your Synology NAS, it's mostly ''volume1''. If you are unsure you can SSH into your Synolog NAS to discover the real path of your shared folder.
-</WRAP>
-Now create the following script file:
-<code>nano /userdata/scripts/romsBackup.sh</code>
-And paste the following content (Replace all placeholders (''<...>'') with the accoring values of your personal environment):
-  * **SMB**: <code>
-#!/bin/sh
-USER=backup
-PASS=<your_password>
-SHARE=//<Synology_NAS_IP_address/hostname>/batoceraBackup
-MOUNT_POINT=/tmp/tempMountFolder
-mkdir -p "$MOUNT_POINT"
-mount -t cifs -o username="$USER",password="$PASS" "$SHARE" "$MOUNT_POINT"
-rsync --exclude=roms/steam --exclude=roms/windows --exclude=roms/windows_installers -Pav --delete -e ssh /userdata/roms "$MOUNT_POINT"
-cd /tmp
-umount -f "$MOUNT_POINT"
-exit 0
-</code>
-  * **NFS**:<code>
-SHARE=<Synology_NAS_IP_address/hostname>:/volume<x>/batoceraBackup
-MOUNT_POINT=/tmp/tempMountFolder
-mkdir -p "$MOUNT_POINT"
-mount -t nfs4 "$SHARE" "$MOUNT_POINT" -o nolock,rw
-rsync --exclude=roms/steam --exclude=roms/windows_installers --delete -e ssh -Pav --no-owner --no-group --no-perms /userdata/roms "$MOUNT_POINT"
-cd /tmp
-umount -f "$MOUNT_POINT"
-exit 0
-</code>
-Save the file and quit the editor.
-Now create the following script file:
-<code>nano /userdata/scripts/userdataBackup.sh</code>
-And paste the following content (Replace all placeholders (''<...>'') with the accoring values of your personal environment):
-  * **SMB**:<code>
-#!/bin/sh
-USER=backup
-PASS=<your_password>
-SHARE=//<Synology_NAS_IP_address/hostname>/batoceraBackup
-MOUNT_POINT=/tmp/tempMountFolder
-mkdir -p "$MOUNT_POINT"
-mount -t cifs -o username="$USER",password="$PASS" "$SHARE" "$MOUNT_POINT"
-cd "$MOUNT_POINT"
-tar cfv "$MOUNT_POINT"/batoceraBackup.tar /userdata --exclude=roms --exclude=saves/flatpak
-cd /tmp
-umount -f "$MOUNT_POINT"
-exit 0
-</code>
-  * **NFS**:<code>
-SHARE=<Synology_NAS_IP_address/hostname>:/volume<x>/batoceraBackup
-MOUNT_POINT=/tmp/tempMountFolder
-mkdir -p "$MOUNT_POINT"
-mount -t nfs4 "$SHARE" "$MOUNT_POINT" -o nolock,rw
-cd "$MOUNT_POINT"
-tar cfv "$MOUNT_POINT"/batoceraBackup.tar /userdata --exclude=roms --exclude=saves/flatpak
-cd /tmp
-umount -f "$MOUNT_POINT"
-exit 0
-</code>
-Save the file and quit the editor.
-Now create the following script file:
-<code>nano /userdata/scripts/systemBackup.sh</code>
-And paste the following content:
-<code>
-#!/bin/sh
-/userdata/scripts/romsBackup.sh
-/userdata/scripts/userdataBackup.sh
-batocera-es-swissknife --shutdown
-</code>
-Save the file and quit the editor.
-Now make all three scripts executable by executing:
-<code>chmod +x /userdata/scripts/*.sh</code>
-You can test your scripts by executing them as follows:
-Backup userdata:
-<code>/userdata/scripts/userdataBackup.sh</code>
-Backup games:
-<code>/userdata/scripts/romsBackup.sh</code>
-Backup userdata and games:
-<code>/userdata/scripts/systemBackup.sh</code>
-If you are planning to only backup your data manually by executing your scripts manually, you can stop here.
-===== Automation part (optional) =====
-Now let's set up the optional automation part for automatic backups of your userdata //and// your games:
-[[https://kb.synology.com/en-global/DSM/tutorial/How_to_login_to_DSM_with_root_permission_via_SSH_Telnet|SSH]] with user ''root'' into your Synology NAS (Unfortunately you have to use a Synology user which is part of the ''administrators'' group, otherwise SSH access will be declined. To avoid any permission issues, we will user ''root'' here.). Then execute the following commands (replace ''<Batocera_IP_address/hostname>'' with the static IP address of your Batocera system) for passwordless login by using SSH [[https://www.ssh.com/academy/ssh/key|secure authentication key]] from your Synology NAS to your Batocera system (if you have already created an SSH key for your ''root'' user in the past on your Synology NAS, skip the first command!):
-<code>ssh-keygen</code> Confirm //everything// without inserting //anything// and by just hitting the ''[Enter]'' key on your keyboard!
-Now execute the following command:
-<code>cat ~/.ssh/id_rsa.pub | ssh root@<Batocera_IP_address/hostname> 'cat >> /userdata/system/.ssh/authorized_keys'</code>
-Insert the Batocera ''root'' user's password (Default password: ''linux''). \\ Now test the passwordless SSH login from your Synology NAS to your Batocera system.
-If passwordless SSH login from your Synology NAS to your Batocera system works, you can now go back to the Synology DSM web GUI and create the according automation task as follows:
-Navigate to ''Control Panel'' → ''Task Scheduler'' → ''Create'' → ''Scheduled Task'' → ''User-defined script'' → ''General'': For ''Task'' insert a custom task name (e.g. ''batoceraBackup'') and for ''user'' choose ''root'' → ''Schedule'': Configure a desired custom schedule → ''Task Settings'': For ''User-defined script'' insert the following (Replace ''<Batocera_MAC_address>'' with the according MAC address of your Batocera system, ''<Synology_network_interface>'' with the according network interface name through which you want to send the magic packet to (the one which is within the same Layer 2 subnet as your Batocera system (e.g. ''eth0'') and ''<Batocera_IP_address/hostname>'' with the static IP address/hostname of your Batocera system.):
-<WRAP center round tip>
-If your Synology NAS does have multiple network interfaces, to find out which network interface name you have to take, from Synology DSM web GUI navigate to: ''Control Panel'' → ''Info Center'' → ''Network''. All of the available network interfaces are listed here with all the unique network/subnet informations. ''LAN1'' is always ''eth0'', ''LAN2'' is always ''eth1'', ''LAN 3'' is always ''eth2'', etc.)
-</WRAP>
-<WRAP center round tip>
-If your Batocera system takes longer than 30 seconds to boot up, feel free to increase the ''sleep'' variable to whatever value you think does fit the real boot time of your Batocera system.
-</WRAP>
-<code>
-#!/bin/sh
-sudo synonet --wake <Batocera_MAC_address> <Synology_network_interface>
-sleep 30
-ssh root@<Batocera_IP_address/hostname> sh ../scripts/systemBackup.sh
-</code>
-Confirm by clicking on ''OK''.
-That's it! Your scheduled backup automation should work now.
-You can test it by running the task manually via the Task Scheduler as follows:
+That's it, you're done!
-Navigate to ''Control Panel'' → ''Task Scheduler'' → Look out for your specific task, highlight it and click on ''Run''.