Install Backy

Binaries are available from the release page. Make sure to get the correct version for your system, which supports x86_64, ARM64, and i386.

Source Install

You can install from source. You will need Go installed.

Then run:

go install git.andrewnw.xyz/CyberShell/backy@master

Once set, jump over to the config docs and start configuring your file.

Config File Definitions

Commands

The commands section is for defining commands. These can be run with or without a shell and on a host or locally.

See the commands documentation for further information.

commands:
  stop-docker-container:
    output: true # Optional and only when run in list and notifications are sent
    cmd: docker
    args:
      - compose
      - -f /some/path/to/docker-compose.yaml
      - down
    # if host is not defined, cmd will be run locally
    host: some-host 
  backup-docker-container-script:
    cmd: /path/to/script
    # The host has to be defined in the config file
    host: some-host
    environment:
      - FOO=BAR
      - APP=$VAR # defined in .env file in config directory
  shell-cmd:
    cmd: rsync
    shell: bash
    args:
      - -av
      - some-host:/path/to/data 
      - ~/Docker/Backups/docker-data
  script:
    type: scriptFile # run a local script on a remote host
    cmd: path/to/your/script.sh
    host: some-host  
  hostname:
    cmd: hostname

Lists

To execute groups of commands in sequence, use a list configuration.

cmd-lists:
  cmds-to-run: # this can be any name you want
    # all commands have to be defined in the commands section
    order:
      - stop-docker-container
      - backup-docker-container-script
      - shell-cmd
      - hostname
    getOutput: true # Optional and only for when notifications are sent
    notifications:
      - matrix
    name: backup-some-server
  hostname:
    name: hostname
    order:
      - hostname
    notifications:
      - prod-email

Hosts

The hosts object may or may not be defined.

hosts:
  # any needed ssh_config(5) keys/values not listed here will be looked up in the config file or the default config file
  some-host:
    hostname: some-hostname
    config: ~/.ssh/config
    user: user
    privatekeypath: /path/to/private/key
    port: 22
    # can also be env:VAR or the password itself
    password: file:/path/to/file
    # can also be env:VAR or the password itself
    privatekeypassword: file:/path/to/file
    # only one is supported for now
    proxyjump: some-proxy-host

Notifications

The notifications object can have two forms.

For more, see the notification object documentation. The top-level map key is id that has to be referenced by the cmd-lists key notifications.

notifications:
  mail:
    prod-email:
      id: prod-email
      type: mail
      host: yourhost.tld
      port: 587
      senderAddress: email@domain.tld
      to:
        - admin@domain.tld
      username: smtp-username@domain.tld
      password: your-password-here
  matrix:
    matrix:
      id: matrix
      type: matrix
      home-server: your-home-server.tld
      room-id: room-id
      access-token: your-access-token
      user-id: your-user-id

Logging

cmd-std-out controls whether commands output is echoed to StdOut.

If logfile is not defined, the log file will be written to the config directory in the file backy.log.

console-disabled controls whether the logging messages are echoed to StdOut. Default is false.

verbose basically does nothing as all necessary info is already output.

logging:
  verbose: false
  file: path/to/log/file.log
  console-disabled: false
  cmd-std-out: false

Vault

Vault can be used to get some configuration values and ENV variables securely.

vault:
  token: hvs.tXqcASvTP8wg92f7riyvGyuf
  address: http://127.0.0.1:8200
  enabled: false
  keys:
    - name: mongourl
      mountpath: secret
      path: mongo/url
      type:  # KVv1 or KVv2
    - name:
      path:
      type:
      mountpath: 

Commands

The yaml top-level map can be any string.

The top-level name must be unique.

Example Config

commands:
  stop-docker-container:
    cmd: docker
    Args:
      - compose
      - -f /some/path/to/docker-compose.yaml
      - down
    # if host is not defined, command will be run locally
    # The host has to be defined in either the config file or the SSH Config files
    host: some-host
    hooks
      error:
        - some-other-command-when-failing
      success:
        - success-command
      final:
        - final-command
  backup-docker-container-script:
    cmd: /path/to/local/script
    # script file is input as stdin to SSH
    type: scriptFile # also can be script
    environment:
      - FOO=BAR
      - APP=$VAR

Values available for this section (case-sensitive):

cmd

cmd must be a valid command or script to execute.

Args

args must be arguments to cmd as they would be passed on the command-line:

cmd [arg1 arg2 ...]

Define them in an array:

Args:
  - arg1
  - arg2
  - arg3

getOutput

Get command output when a notification is sent.

Is not required. Can be true or false.

host

Host may or may not be defined in the hosts section.

shell

If shell is defined, the command will run in the specified shell. Make sure to escape any shell input.

scriptEnvFile

Path to a file.

When type is script or scriptFile , the script is appended to this file.

This is useful for specifying environment variables or other things so they don’t have to be included in the script.

type

May be scriptFile or script. Runs script from local machine on remote host passed to the SSH session as standard input.

If type is script, cmd is used as the script.

If type is scriptFile, cmd must be a script file.

If type is package, there are additional fields that must be specified.

environment

The environment variables support expansion:

• using escaped values $VAR or ${VAR}

For now, the variables have to be defined in an .env file in the same directory as the config file.

If using it with host specified, the SSH server has to be configured to accept those env variables.

If the command is run locally, the OS’s environment is added.

hooks

Hooks are run after the command is run.

Errors are run if the command errors, success if it returns no error. Final hooks are run regardless of error condition.

Values for hooks are as follows:

command:
  hook:
    # these commands are defined elsewhere in the file
    error:
      - errcommand
    success:
      - successcommand
    final:
      - donecommand

packages

See the dedicated page for package configuration.

Command Lists

Command lists are for executing commands in sequence and getting notifications from them.

The top-level object key can be anything you want but not the same as another.

Lists can go in a separate file. Command lists should be in a separate file if:

1. key ‘cmd-lists.file’ is found
2. hosts.yml or hosts.yaml is found in the same directory as the backy config file

  test2:
    name: test2
    order:
      - test
      - test2
    notifications:
      - mail.prod-email
      - matrix.sysadmin
    cron: "0 * * * * *"

Order

The order is an array of commands to execute in order. Each command must be defined.

order:
  - cmd-1
  - cmd-2

getOutput

Get command output when a notification is sent.

Is not required. Can be true or false. Default is false.

Notifications

An array of notification IDs to use on success and failure. Must match any of the notifications object map keys.

Name

Name is optional. If name is not defined, name will be the object’s map key.

Cron mode

Backy also has a cron mode, so one can run backy cron and start a process that schedules jobs to run at times defined in the configuration file.

Adding cron: 0 0 1 * * * to a cmd-lists object will schedule the list at 1 in the morning. See https://crontab.guru/ for reference.

cmd-lists:
  docker-container-backup: # this can be any name you want
    # all commands have to be defined
    order:
      - stop-docker-container
      - backup-docker-container-script
      - shell-cmd
      - hostname
      - start-docker-container
    notifications:
      - matrix.id
    name: backup-some-container
    cron: "0 0 1 * * *"
  hostname:
    name: hostname
    order:
      - hostname
    notifications:
      - mail.prod-email

Packages

This is dedicated to package commands. The command type field must be package. Package is a type that allows one to perform package operations. There are several additional options available when type is package:

example

The following is an example of a package command:

 update-docker:
    type: package
    shell: zsh
    packageName: docker-ce
    packageManager: apt
    packageOperation: install
    host: debian-based-host

packageOperation

The following package operations are supported:

• install
• remove
• upgrade

packageManager

The following package managers are recognized:

• apt
• yum
• dnf

package command args

You can add additional arguments using the standard Args key. This is useful for adding more packages.

Development

The PackageManager interface provides an easy to enforce functions and options. There are two interfaces, PackageManager and ConfigurablePackageManager in the directory pkg/pkgman. Go’s import-cycle “feature” caused me to implement functional options using a third interface. PackageManagerOptionis a function that takes an interface.

PackageManager

// PackageManager is an interface used to define common package commands. This shall be implemented by every package.
type PackageManager interface {
	Install(pkg, version string, args []string) (string, []string)
	Remove(pkg string, args []string) (string, []string)
	Upgrade(pkg, version string) (string, []string) // Upgrade a specific package
	UpgradeAll() (string, []string)

	// Configure applies functional options to customize the package manager.
	Configure(options ...pkgcommon.PackageManagerOption)
}

There are a few functional options that should be implemented using the ConfigurablePackageManager interface:

// ConfigurablePackageManager defines methods for setting configuration options.
type ConfigurablePackageManager interface {
	SetUseAuth(useAuth bool)
	SetAuthCommand(authCommand string)
}

Notifications

Notifications can be sent on command list completion and failure.

The supported platforms for notifications are email (SMTP) and Matrix.

Notifications are defined by service, with the current form following below. Ids must come after the service.

notifications:
  mail:
    prod-email:
      host: yourhost.tld
      port: 587
      senderaddress: email@domain.tld
      to:
        - admin@domain.tld
      username: smtp-username@domain.tld
      password: your-password-here

  matrix:
    matrix:
      home-server: your-home-server.tld
      room-id: room-id
      access-token: your-access-token
      user-id: your-user-id

Sections recognized are mail and matrix

There must be a section with an id (eg. mail.test-svr) following one of these sections.

mail

matrix

To get your access token (assumes you are using Element) :

1. Log in to the account you want to get the access token for. Click on the name in the top left corner, then “Settings”.
2. Click the “Help & About” tab (left side of the dialog).
3. Scroll to the bottom and click on <click to reveal> part of Access Token.
4. Copy your access token to a safe place.

To get the room ID:

1. On Element or a similar client, navigate to the room.
2. Navigate to the settings from the top menu.
3. Click on Advanced, the room ID is there.

Vault

Vault is a tool for storing secrets and other data securely.

Vault config can be used by prefixing vault: in front of a password or ENV var.

This is the object in the config file:

vault:
  token: hvs.tXqcASvTP8wg92f7riyvGyuf
  address: http://127.0.0.1:8200
  enabled: false
  keys:
    - name: mongourl
      mountpath: secret
      path: mongo/url
      type:  # KVv1 or KVv2
    - name:
      path:
      type:
      mountpath:

Exec

The exec subcommand can do some things that the configuration file can’t do yet. The command exec host can execute commands on many hosts.

exec host takes the following arguments:

  -c, --commands strings   Accepts comma-separated names of commands.
  -h, --help               help for host
  -m, --hosts strings      Accepts comma-separated names of hosts.

The commands have to be defined in the config file. The hosts need to at least be in the ssh_config(5) file.

backy exec host [--commands=command1,command2, ... | -c command1,command2, ...] [--hosts=host1,hosts2, ... | -m host1,host2, ...]  [flags]