search

ridgekuhn / arklone-arkos

rclone cloud syncing utility for ArkOS
GNU General Public License v3.0
18 stars 1 forks source link

readme

arklone

rclone cloud sync utility for ArkOS

Watches save directories for RetroArch, and select standalone games.

This project offers no warranties or guarantees. Always make extra backup copies of your data. Use at your own risk!

arklone is released under a GNU GPLv3 license. See LICENSE.md for more information.

rclone, RetroArch, EmulationStation, and ArkOS are the properties of their respective owners.

Table of Contents

  1. Installation
  2. rclone Configuration
  3. First Run
  4. Settings Dialog
  5. Syncing with the Cloud
  6. Advanced RetroArch Configuration
  7. Recommended RetroArch Configuration
  8. Advanced arklone Configuration
  9. Troubleshooting
  10. Developers
  11. FAQ

 

Installation

This module is not yet integrated into ArkOS. See this pull request for updates.

To test, install manually by downloading the installation script, and run it from a terminal:

wget https://github.com/ridgekuhn/arkos/raw/cloudbackups/10092021/install.sh -O installArklone.sh
chmod a+x installArklone.sh
./installArklone.sh
rm ./installArklone.sh

 

Uninstallation

wget https://github.com/ridgekuhn/arkos/raw/cloudbackups/10092021/uninstall.sh -O uninstallArklone.sh
chmod a+x uninstallArklone.sh
./uninstallArklone.sh
rm ./uninstallArklone.sh

 

rclone Configuration

To begin using arklone, you must create an rclone config file. For most cloud providers, this will involve installing rclone to a computer with a web browser, like your desktop or laptop. See the rclone docs for more information on how to do this for your specific provider. Make sure you install the latest version of rclone, (1.56.2 as of this writing) to your desktop or laptop. If you use a package manager like apt, the repository version will be outdated.

Once you have completed this process, copy the rclone.conf file from your computer to your ArkOS device. Your rclone.conf can be located by running:

rclone config file

On your ArkOS SD card, copy rclone.conf to: EASYROMS/backup/rclone/rclone.conf

If you already had rclone installed on your device, your rclone.conf has been moved to EASYROMS for easier access, and symlinked to its original location. arklone will restore the original arrangement if uninstalled.

 

First Run

From EmulationStation, navigate to Options -> Cloud Settings

On first run, you will be greeted by a prompt asking if you'd like to change your RetroArch configurations to the recommended settings.

Obviously, this is recommended!

This will set the following settings in your retroarch.cfg (and your retroarch32/retroarch.cfg):

savefile_directory = "~/.config/retroarch/saves"
savefiles_in_content_dir = "false"
sort_savefiles_enable = "false"
sort_savefiles_by_content_enable = "true"

savestate_directory = "~/.config/retroarch/saves"
savestates_in_content_dir = "false"
sort_savestates_enable = "false"
sort_savestates_by_content_enable = "true"

This will result in savefiles and savestates being stored in the same directory hierarchy as your RetroArch content root, in ~/.config/retroarch/saves

eg, ~/.config/retroarch/saves/nes/TheLegendOfZelda.srm ~/.config/retroarch/saves/nes/TheLegendOfZelda.savestate0

First run screen

 

Settings Dialog

Arklone main menu

 

Syncing with the Cloud

Keeping multiple devices synced can be difficult. arklone tries to do its best, but you should always keep an extra backup copy just in case.

Automatic Syncing

If you enable automatic syncing in the settings dialog, arklone assumes the copy of your data stored in the cloud is the canonical and "always correct" version. On system boot, arklone will run before EmulationStation and attempt to receive updates from the cloud remote. If the remote contains a newer version of a file, it will overwrite the local copy. On this initial sync, arklone only receives updates and does not send anything back.

If the boot sync process succeeds, arklone will begin watching all your save directories, and send updates any time a write is detected, overwriting older versions on the cloud remote. It will also receive updates at the interval set when you enabled automatic syncing.

If the boot process fails or is cancelled by the user, the user is given the chance to try again, or the dirty boot state is set.

Manual Syncing

The manual sync dialog allows you to send or receive any directory which has a path module registered with arklone.

 

Dirty Boot State

If automatic syncing is enabled and the boot sync process fails, the dirty boot state is set. Automatic syncing is disabled for the rest of the session, and will resume after the next boot. On the next boot, the user is shown a warning message about potential data loss before the boot sync proceeds.

To manually reset the dirtyboot state, delete the lock file located at: ~/.config/arklone/.dirtyboot

 

Advanced RetroArch Configuration

This section is for users who wish to have more control over their retroarch.cfg settings and save directories.

Supported RetroArch Configuration

ArkOS includes 64-bit and 32-bit builds of RetroArch. The configuration files are stored at /home/ark/.config/retroarch/retroarch.cfg and /home/ark/.config/retroarch32/retroarch.cfg.

The following settings are supported:

savefile_directory
savefiles_in_content_dir
sort_savefiles_enable
sort_savefiles_by_content_enable

savestate_directory
savestates_in_content_dir
sort_savestates_enable
sort_savestates_by_content_enable

For the next examples, filetype refers to either savefile or savestate.

If filetypes_in_content_dir = "true", it will override the other related settings, and create save data next to the content file.

Otherwise, if sort_filetypes_enable = "true", save data will be organized by libretro core inside filetype_directory. eg, /path/to/filetype_directory/FCEUmm/TheLegendOfZelda.srm

If sort_filetypes_by_content_enable = "true", save data will be organized by the parent directory of the content file. eg, /path/to/filetype_directory/nes/TheLegendOfZelda.srm

If both sort_filetypes_enable = "true" and sort_filetypes_by_content_enable = "true", save data will be organized by the parent directory of the content file, then by libretro core. eg, /path/to/filetype_directory/nes/FCEUmm/TheLegendOfZelda.srm

Known Bugs

ArkOS currently contains a bug which prevents systemd path units from watching subdirectories of exFAT partitions. (See issue #289.) This means that savefiles/savestates can not be watched (and automatically synced) if filetypes_in_content_dir = "true".

Until this bug is resolved, if you wish to store your saves next to the content, you must manually sync your saves from the arklone dialog.

Since the bug only applies to exFAT partitions, advanced users who really want to use automatic syncing and keep savefiles/savestates in the content directories can re-format the EASYROMS partition to FAT32, ext4, etc and edit the mount entry in /etc/fstab.

Recommended RetroArch Configuration

savefile_directory = "~/.config/retroarch/saves"
savefiles_in_content_dir = "false"
sort_savefiles_enable = "false"
sort_savefiles_by_content_enable = "true"

savestate_directory = "~/.config/retroarch/states"
savestates_in_content_dir = "false"
sort_savestates_enable = "false"
sort_savestates_by_content_enable = "true"

 

Advanced arklone Configuration

Arklone has a few settings that can be changed by the user, mostly paths where arklone looks for various files. The user configuration file is stored at ~/.config/arklone/arklone.cfg.

Resetting to "First Run" State

Setting remote to an empty string forces the settings dialog to show the "first run" screen again.

 

arklone.cfg

remote = ""

Changing RetroArch Content Root

Where filetype refers to either savefile or savestate:

If your retroarch.cfg contains the settings filetypes_in_content_dir = "true" or sort_filetypes_by_content_enable = "true", arklone expects your RetroArch content to be organized in a directory hierarchy with one level of subdirectories, where each contains all content for a particular platform. eg, retroarchContentRoot/nes/TheLegendOfZelda.rom

 

arklone.cfg

retroarchContentRoot = "/absolute/path/to/retroarchContentRoot"

arklone also supports select standalone software and "ports". See the systemd/units for a list, and the Path Units section of the developer docs for more info.

Multiple RetroArch Instances

arklone supports multiple instances of RetroArch, in case your distro has both 64-bit and 32-bit builds installed. Set retroarchCfg to a space-delimited list of absolute paths to each retroarch.cfg.

 

arklone.cfg

retroarchCfg = "/home/user/.config/retroarch/retroarch.cfg /home/user/.config/retroarch32/retroarch.cfg"

rclone Filters

arklone passes various filter lists to rclone when a sync script is run. See the Path Units and rclone Filters sections of the developer docs for more info.

 

Troubleshooting

RetroArch Saves Not Syncing

arklone only watches the RetroArch save directories it knew about when it first generated the corresponding path units. If you selected "Set Recommended Settings" on your first run, arklone will automatically generate path units for all your RetroArch content directories which are not empty. If you've added games since then and some of your save directories are not being synced automatically, try manually regenerating them from the arklone dialog menu.

If you change any of the above settings in retroarch.cfg, you must also manually regenerate the path units.

Ports, Standalone Apps, or Other Game Saves Not Syncing

ArkOS is constantly updated with new apps and ports, and we probably haven't caught up to them yet. Please create a new issue so we can include it in a future update.

Logging

To save unnecessary writes to your SD card or hard drive, arklone writes logs to the RAM filesystem at /dev/shm/arklone.log. This file disappears when the system is powered down, but you can view it by opening the arklone settings dialog and selecting "View log file".

 

Developers

Contributions are welcome! Please see the developer docs.

 

FAQ

Can I use it on Windows, MacOS, or other Linux Distros?

Linux

A RetroPie release is planned soon. arklone is written in bash, and relies on tools like apt, dpkg, and inotify-tools. It should theoretically work on any Debian-based distro, as long as your content is organized in the expected directory hierarchy. See Advanced arklone Configuration for more info.

Windows and MacOS

If your cloud provider offers a desktop client, you should install and use that instead.

Can I add my own custom directories?

See the Path Units section of the developer docs.

Can I Use arklone to Sync ROMs or BIOS files?

Not unless you want to set it up yourself. Many users' game libraries are massive and would probably exceed the storage limit on your cloud account several times over. There are also system performance implications for keeping this much data synced on low-power devices, like the ones ArkOS is designed for, where the background sync operations may affect gameplay. It's probably much faster/efficient to transfer your game libraries from device-to-device via USB or over your LAN.

Why Am I Seeing "Directory Not Found" Errors in the Log?

If you have automatic syncing enabled, arklone attempts to download all the different save directories it knows about from the cloud remote. If they don't exist on the cloud remote, these messages are generated for logging and debugging purposes. If there are any actual problems downloading save data from the cloud, you will be presented with a dialog screen, and asked if you want to proceed or view the log file. If you don't see this dialog screen and your device boots straight into EmulationStation, then everything is ok!