February 9, 2017

Install Swift on a single VM with TempAuth (NO OpenStack/SwiftStack client)

This is guide on how to install Swift on a single VM for development/testing purposes. I created this guide after I spent weeks trying to install Swift Object Storage as a standalone package (no SwiftStack or OpenStack clients) and you should have swift up and running within 10 to 30min (varies with internet speed). This setup uses tempauth for authentication.

Prerequisites

  • VirtualBox 64bit. (5.1.10 was used in this installation) [link]
  • Ubuntu Server 16.04.1 image [link]
  • 8 GB for the VM and for the virtual hard drives to be used as storage devices.
  • The username and hostname for the VM MUST be set to “swift” for the scripts in this guide to work.
  • Recommended Install OpenSSH onto your VM to copy/paste commands from this guide.

NB: All commands to be run as sudo.

1. Configure the storage devices

  • Create 3 virtual hard disks (50MB each) and attach them to the VM. This link can be useful if you don’t know how to do this. You just need to create and attach the virtual hard disks (no need to format them). Verify that these storage devices are located at /dev/sdb, /dev/sdc and /dev/sdd. If they aren’t, the scripts in this guide will not work. To do this you can use:

    $ sudo fdisk -l /dev/sdb /dev/sdc /dev/sdd
  • Install supporting utility packages:

    $ sudo apt-get install xfsprogs rsync
  • Format the /dev/sdb,  /dev/sdc and /dev/sdd devices as XFS:

    $ sudo mkfs.xfs /dev/sdb -f
    $ sudo mkfs.xfs /dev/sdc -f
    $ sudo mkfs.xfs /dev/sdd -f
  • Create the mount point directory structure:

    $ sudo mkdir -p /srv/node/sdb
    $ sudo mkdir -p /srv/node/sdc
    $ sudo mkdir -p /srv/node/sdd
  • Edit the /etc/fstab file and add the following to it:

    /dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
    /dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
    /dev/sdd /srv/node/sdd xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
  • Mount the devices:

    $ sudo mount /srv/node/sdb
    $ sudo mount /srv/node/sdc
    $ sudo mount /srv/node/sdd
  • Download the preconfigured file /etc/rsyncd.conf:

    $ sudo curl -o /etc/rsyncd.conf https://raw.githubusercontent.com/tino1b2be/swift-on-vm/v0.1/config_files/rsyncd-sample.conf
  • Edit the /etc/default/rsync file and enable the rsync service:

    RSYNC_ENABLE=true
  • Start the rsync service:

    $ sudo service rsync start

2. Install and Configure Swift

  • Install memcached and swift packages

    $ sudo apt-get install memcached python-memcache swift swift-proxy python-swiftclient swift-account swift-container swift-object
  • Download and run the script to install and configure swift.
    NB!!! I created this script myself. If it fails, open the script and debug by executing each command in the bash terminal.

    $ sudo curl -o /tmp/swift_storage_script https://raw.githubusercontent.com/tino1b2be/swift-on-vm/v0.12/scripts/swift_storgae_script
    $ . /tmp/swift_storage_script
  • The “swift_storage_script” does the following:
    • Installs and configures memcached (used to store tempauth tokens)
    • Installs ad configures the Swift Proxy servers
    • Installs and confiures the Account, Container and Object servers.
    • Sets proper ownership of the config files and also for the recon directory that’s also created.
    • Creates the account, container and storage rings.
    • Restarts the services used by Swift (swift servers and memcached)
  • The script basically does everything on this page from step 4.3 right to the end or on this page.
  • Verify operation

    $ swift stat
                             Account: AUTH_swift
                          Containers: 0
                             Objects: 0
                               Bytes: 0
     Containers in policy "policy-0": 0
        Objects in policy "policy-0": 0
          Bytes in policy "policy-0": 0
                         X-Timestamp: 1444143887.71539
                          X-Trans-Id: tx1396aeaf17254e94beb34-0056143bde
                        Content-Type: text/plain; charset=utf-8
                       Accept-Ranges: bytes

Use Swauth instead of TempAuth

Swauth is an Auth Service for Swift as WSGI Middleware that uses Swift itself as a backing store.

  • Install Swauth:

    $ sudo apt install swauth
  • Modify the pipeline in /etc/swift/proxy-server.conf to replace “tempauth” with “swauth”

    [pipeline:main]
    pipeline = catch_errors cache swauth proxy-server
  • Add a new section for swauth in the same file /etc/swift/proxy-server.conf

    [filter:swauth]
    use = egg:swauth#swauth
    Set log_name = swauth
    Super_admin_key = <swauthkey>
  • Modify the [app:proxy-server] section in the /etc/swift/proxy-server.conf file:

    [app:proxy-server]
    use = egg:swift#proxy
    allow_account_management = true
    account_autocreate = true
  • Restart the swift proxy server:

    $ swift-init proxy restart
  • Initialize the Swauth backing store in Swift:

    $ swauth-prep -K <swauthkey>
  • Verify operation:
    • Create a new account (account = swift, user = swift, password = swift):

      $ swauth-add-user -A http://127.0.0.1:8080/auth/ -K swauthkey -a swift swift swift
    • Try out the new auth system with swift (check the account information):

      $ swift -A http://127.0.0.1:8080/auth/v1.0 -K swift -U swift:swift stat
    • Or a shorter way of doing that is to make use the OpenRC file saved in $HOME/swiftrc (or just create your own) to shorten the command to just:

      $ swift stat

 

RECENT POSTS