January 5, 2017

Installing Swift Object Storge on a single VM

Installing Swift Object Storage on a single VM


UPDATE!!!! : I created a much shorter blog post on how to install Swift that uses TempAuth or Swauth. This one does not use any OpenStack software addons…and it takes only about 30min to complete the setup. Link


I created this doc during a research task on Object Storage when I interned at Praekelt.orgIn this set up we will set up a Swift Object Storage server. This setup will assume that you have basic understanding of how Swift Object storage works. You can use the following links to learn more about it:

We will set up both the Swift Proxy server and the client on a single VM all from scratch. I used OpenStack’s guide and links are included throughout the this setup. Only refer to these links if you get stuck or if you want more details.

Prerequisites

  • VirtualBox 64bit. (5.1.10 was used in this installation) [link]
  • Ubuntu Server 16.04.1 image [link]
  • 8 GB for the VM and 150MB for the virtual hard drive to be used as storage devices.
  • 2 CPUs and 4GB RAM for the VM ( minimum requirements )

NB:

  • All commands must be run as root (or sudo)
  • Any lines that start with # are commands that MUST be run as root.
  1. Setting up the Virtual Machine

    1. In VirtualBox, create a new Linux 64bit VM and install the Ubuntu Server 16.04.1.
    2. Install OpenSSH so you can copy-paste into the VM’s terminal via SSH (you can’t if you inside the VM)
    3. Swift needs at least 3 storage devices to work. To simulate these devices, we will create 3 virtual hard disks which will be used at a later stage. To do this,
      1. go to the VM’s settings > Storage
      2. Under “Controller: SATA”, select “Add new hard disk” > “create new disk” > “VDI” > “Fixed Size” > set it to 50MB and give it a name.
      3. Do this for the other 2 disks, and set the same size for all of them.
  2. Setting up the Environment [link]

    1. Networking [link]

      1. Take note of your network card’s IP address. We will refer to this as HOST_IP.
      2. Set the hostname of the VM to “controller
        Edit /etc/hostname and change whatever is in there to “controller
      3. Edit /etc/hosts. Comment out the 127.0.1.1 entry if it is in there and add:

        # controller
        HOST_IP controller
      4. Restart the VM

        # shutdown -r now
      5. Verify connectivity (ping google’s public DNS or something)

        # ping -c 4 8.8.8.8
    2. OpenStack Packages

      1. Enable the OpenStack repository

        # apt install software-properties-common
        # add-apt-repository cloud-archive:newton
      2. Update the packages (go make some coffee this will take a while)

        # apt update && apt dist-upgrade
      3. Install the OpenStack client. You might need to restart after this:

        # apt install python-openstackclient
    3. SQL Database

      1. Install the packages.

        # apt install mariadb-server python-pymysql
      2. Create and edit “/etc/mysql/mariadb.conf.d/99-openstack.cnf” to have the following:

        [mysqld]
        bind-address = HOST_IP
         
        default-storage-engine = innodb
        innodb_file_per_table
        max_connections = 4096
        collation-server = utf8_general_ci
        character-set-server = utf8
      3. Restart the database service:

        # service mysql restart
      4. Secure the database service by running the mysql_secure_installation script. In particular, choose a suitable password for the database root account.

        # mysql_secure_installation
    4. Memcached

      1. Install the packages

        # apt install memcached python-memcache
      2. Edit the /etc/memcached.conf file and configure the service to use the management IP address of the controller node to enable access by other nodes via the management network:

        -l HOST_IP
      3. Restart memcached service

        # service memcached restart
  3. Identity Service [link]

    The Identity Service is used for authentication (using Keystone). However, TempAuth can also be used instead of keystone

    1. Prerequisites

      Before you configure the OpenStack Identity service, you must create a database and an administration token. To create the database, complete the following actions:

      1. Use the database access client to connect to the database server as the root user:

        $ mysql -u root -p
      2. Create the keystone database:

        mysql> CREATE DATABASE keystone;
      3. Grant proper access to the keystone database:

         mysql> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \
           IDENTIFIED BY 'KEY_DBPASS';
         mysql> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' \
           IDENTIFIED BY 'KEY_DBPASS';

        Replace KEY_DBPASS with a suitable password.

      4. Exit the database access client.
    2. Install and configure components

      1. Install packages

        # apt install keystone
      2. Edit the /etc/keystone/keystone.conf file and complete the following actions:
        1. In the [database] section, configure database access:

          [database]
           ...
           connection = mysql+pymysql://keystone:KEY_DBPASS@controller/keystone

          Replace KEY_DBPASS with the password you chose for the database (in 3.a.iii).

        2. In the [token] section, configure the Fernet token provider:

          [token]
           ...
           provider = fernet
        3. Populate the Identity service database:

          # su -s /bin/sh -c "keystone-manage db_sync" keystone
      3. Initialize Fernet key repositories:

        # keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
        # keystone-manage credential_setup --keystone-user keystone --keystone-group keystone
      4. Bootstrap the Identity service:

        # keystone-manage bootstrap --bootstrap-password ADMIN_PASS \
          --bootstrap-admin-url http://controller:35357/v3/ \
          --bootstrap-internal-url http://controller:35357/v3/ \
          --bootstrap-public-url http://controller:5000/v3/ \
          --bootstrap-region-id RegionOne

        Replace ADMIN_PASS with a suitable password for an administrative user.

    3. Configure Apache Server

      1. Edit the /etc/apache2/apache2.conf file and configure the ServerName option to reference the controller node (you may have to add this entry at the end if you can’t find it):

        ServerName controller
      2. Finalize
        1. Restart the Apache service and remove the default SQLite database:

          # service apache2 restart
          # rm -f /var/lib/keystone/keystone.db
        2. Configure the administrative account

          $ export OS_USERNAME=admin
          $ export OS_PASSWORD=ADMIN_PASS
          $ export OS_PROJECT_NAME=admin
          $ export OS_USER_DOMAIN_NAME=default
          $ export OS_PROJECT_DOMAIN_NAME=default
          $ export OS_AUTH_URL=http://controller:35357/v3
          $ export OS_IDENTITY_API_VERSION=3

          Replace ADMIN_PASS with the password used in the keystone-manage bootstrap command from 3.b.v

    4. Create a domain, project, users and roles

      1. The Identity service provides authentication services for each OpenStack service. The authentication service uses a combination of domains, projects, users, and roles.

        NB: You can repeat this procedure to create additional projects and users.

        This guide uses a service project that contains a unique user for each service that you add to your environment. Create the service project:

        $ openstack project create --domain default --description "Service Project" service
         +-------------+----------------------------------+
         | Field       | Value                            |
         +-------------+----------------------------------+
         | description | Service Project                  |
         | domain_id   | default                          |
         | enabled     | True                             |
         | id          | 24ac7f19cd944f4cba1d77469b2a73ed |
         | is_domain   | False                            |
         | name        | service                          |
         | parent_id   | default                          |
         +-------------+----------------------------------+
      2. Regular (non-admin) tasks should use an unprivileged project and user. As an example, this guide creates the demo project and user.
        1. Create the demo project:

          $ openstack project create --domain default --description "Demo Project" demo
           +-------------+----------------------------------+
           | Field       | Value                            |
           +-------------+----------------------------------+
           | description | Demo Project                     |
           | domain_id   | default                          |
           | enabled     | True                             |
           | id          | 231ad6e7ebba47d6a1e57e1cc07ae446 |
           | is_domain   | False                            |
           | name        | demo                             |
           | parent_id   | default                          |
           +-------------+----------------------------------+

          NB: Do not repeat this step when creating additional users for this project.

        2. Create the demo user:

          $ openstack user create --domain default --password-prompt demo
           
           User Password:
           Repeat User Password:
           +---------------------+----------------------------------+
           | Field               | Value                            |
           +---------------------+----------------------------------+
           | domain_id           | default                          |
           | enabled             | True                             |
           | id                  | aeda23aa78f44e859900e22c24817832 |
           | name                | demo                             |
           | password_expires_at | None                             |
           +---------------------+----------------------------------+

          NB: Note the password you enter here

        3. Create the user role:

          $ openstack role create user
           
           +-----------+----------------------------------+
           | Field     | Value                            |
           +-----------+----------------------------------+
           | domain_id | None                             |
           | id        | 997ce8d05fc143ac97d83fdfb5998552 |
           | name      | user                             |
           +-----------+----------------------------------+
        4. Add the user role to the demo project and user:

          $ openstack role add --project demo --user demo user
    5. Verify operation

      1. For security reasons, disable the temporary authentication token mechanism:

        Edit the /etc/keystone/keystone-paste.ini file and remove admin_token_auth from the [pipeline:public_api], [pipeline:admin_api], and [pipeline:api_v3] sections.
      2. Unset the temporary OS_AUTH_URL and OS_PASSWORD environment variable:

        $ unset OS_AUTH_URL OS_PASSWORD
      3. As the admin user, request an authentication token:

        $ openstack --os-auth-url http://controller:35357/v3 \
           --os-project-domain-name default --os-user-domain-name default \
           --os-project-name admin --os-username admin token issue
         
         Password:
         +------------+---------------------------------------------------------+
         | Field      | Value                                                   |
         +------------+---------------------------------------------------------+
         | expires    | 2016-02-12T20:14:07.056119Z                             |
         | id         |...
         ...
        

        NB : This command uses the password for the admin user

      4. As the demo user, request an authentication token:

        $ openstack --os-auth-url http://controller:5000/v3 \
           --os-project-domain-name default --os-user-domain-name default \
           --os-project-name demo --os-username demo token issue
         
         Password:
         +------------+-----------------------------------------------------------+
         | Field      | Value                                                     |
         +------------+-----------------------------------------------------------+
         | expires    | 2016-02-12T20:15:39.014479Z                               |
         | id         |...
         ...

        NB: This command uses the password for the demo user and API port 5000 which only allows regular (non-admin) access to the Identity service API.

    6. Create OpenStack client environment scripts

      1. The previous section used a combination of environment variables and command options to interact with the Identity service via the openstack client. To increase efficiency of client operations, OpenStack supports simple client environment scripts also known as OpenRC files. These scripts typically contain common options for all clients, but also support unique options. For more information, see the OpenStack End User Guide.

        Create client environment scripts for the admin and demo projects and users. Future portions of this guide reference these scripts to load appropriate credentials for client operations.
      2. Create and edit the admin-openrc file and add the following content:

        export OS_PROJECT_DOMAIN_NAME=default
        export OS_USER_DOMAIN_NAME=default
        export OS_PROJECT_NAME=admin
        export OS_USERNAME=admin
        export OS_PASSWORD=ADMIN_PASS
        export OS_AUTH_URL=http://controller:35357/v3
        export OS_IDENTITY_API_VERSION=3
        export OS_IMAGE_API_VERSION=2

        Replace ADMIN_PASS with password you chose for admin in the Identity service (3.b.v).

      3. Create and edit the demo-openrc file and add the following content:

        export OS_PROJECT_DOMAIN_NAME=default
        export OS_USER_DOMAIN_NAME=default
        export OS_PROJECT_NAME=demo
        export OS_USERNAME=demo
        export OS_PASSWORD=DEMO_PASS
        export OS_AUTH_URL=http://controller:5000/v3
        export OS_IDENTITY_API_VERSION=3
        export OS_IMAGE_API_VERSION=2

        Replace DEMO_PASS with the password you chose for the demo user in the Identity service.

      4. Using the scripts

        To run clients as a specific project and user, you can simply load the associated client environment script prior to running them. For example:

        1. Load the admin-openrc file to populate environment variables with the location of the Identity service and the admin project and user credentials:

          $ source admin-openrc
        2. Request an authentication token:

          $ openstack token issue
           
           +------------+-----------------------------------------------------+
           | Field      | Value                                               |
           +------------+-----------------------------------------------------+
           | expires    | 2016-02-12T20:44:35.659723Z                         |
           | id         | ... 
           ... 
  4. Install Swift Object Storage [link]

    1. Configure Controller Node

      1. The proxy server relies on an authentication and authorization mechanism such as the Identity Service. Other authorisation services can be used but in this setup we will be using OpenStack’s Identity service (which uses keystone). Before we configure the Swift Object Storage service, you must create service credentials and an API endpoint.

        Source the admin credentials to gain access to admin-only CLI commands:

        $ source admin-openrc
      2. To create the Identity service credentials, complete these steps:
        1. Create the swift user:

          $ openstack user create --domain default --password-prompt swift
           User Password:
           Repeat User Password:
           +-----------+----------------------------------+
           | Field     | Value                            |
           +-----------+----------------------------------+
           | domain_id | e0353a670a9e496da891347c589539e9 |
           | enabled   | True                             |
           | id        | d535e5cbd2b74ac7bfb97db9cced3ed6 |
           | name      | swift                            |
           +-----------+----------------------------------+
        2. Add the admin role to the swift user:

          $ openstack role add --project service --user swift admin

          NB: This command provides no output.

        3. Create the swift service entity:

          $ openstack service create --name swift --description "OpenStack Object Storage" object-store
           +-------------+----------------------------------+
           | Field       | Value                            |
           +-------------+----------------------------------+
           | description | OpenStack Object Storage         |
           | enabled     | True                             |
           | id          | 75ef509da2c340499d454ae96a2c5c34 |
           | name        | swift                            |
           | type        | object-store                     |
           +-------------+----------------------------------+
        4. Create the Object Storage service API endpoints (three separate commands):

          $ openstack endpoint create --region RegionOne \
             object-store public http://controller:8080/v1/AUTH_%\(tenant_id\)s
           +--------------+----------------------------------------------+
           | Field        | Value                                        |
           +--------------+----------------------------------------------+
           | enabled      | True                                         |
           | id           | 12bfd36f26694c97813f665707114e0d             |
           | interface    | public                                       |
           | region       | RegionOne                                    |
           | region_id    | RegionOne                                    |
           | service_id   | 75ef509da2c340499d454ae96a2c5c34             |
           | service_name | swift                                        |
           | service_type | object-store                                 |
           | url          | http://controller:8080/v1/AUTH_%(tenant_id)s |
           +--------------+----------------------------------------------+
           
           $ openstack endpoint create --region RegionOne \
             object-store internal http://controller:8080/v1/AUTH_%\(tenant_id\)s
           +--------------+----------------------------------------------+
           | Field        | Value                                        |
           +--------------+----------------------------------------------+
           | enabled      | True                                         |
           | id           | 7a36bee6733a4b5590d74d3080ee6789             |
           | interface    | internal                                     |
           | region       | RegionOne                                    |
           | region_id    | RegionOne                                    |
           | service_id   | 75ef509da2c340499d454ae96a2c5c34             |
           | service_name | swift                                        |
           | service_type | object-store                                 |
           | url          | http://controller:8080/v1/AUTH_%(tenant_id)s |
           +--------------+----------------------------------------------+
           
           $ openstack endpoint create --region RegionOne \
             object-store admin http://controller:8080/v1
           +--------------+----------------------------------+
           | Field        | Value                            |
           +--------------+----------------------------------+
           | enabled      | True                             |
           | id           | ebb72cd6851d4defabc0b9d71cdca69b |
           | interface    | admin                            |
           | region       | RegionOne                        |
           | region_id    | RegionOne                        |
           | service_id   | 75ef509da2c340499d454ae96a2c5c34 |
           | service_name | swift                            |
           | service_type | object-store                     |
           | url          | http://controller:8080/v1        |
           +--------------+----------------------------------+
      3. Install Swift packages

        # apt-get install swift swift-proxy python-swiftclient
      4. Create the /etc/swift directory.
      5. Obtain the proxy service configuration file from the Object Storage source repository:

        # curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/newton
      6. Edit the /etc/swift/proxy-server.conf file and complete the following actions:
        1. In the [DEFAULT] section, configure the bind port, user, and configuration directory:

          [DEFAULT]
           ...
           bind_port = 8080
           user = swift
           swift_dir = /etc/swift
        2. In the [pipeline:main] section, remove the tempurl and tempauth modules and add the authtoken and keystoneauth modules (or just replace is with this… it’s one line):

          [pipeline:main]
           pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server

          NB: Do not change the order of the modules.

        3. In the [app:proxy-server] section, enable automatic account creation:

          [app:proxy-server]
           use = egg:swift#proxy
           ...
           account_autocreate = True
        4. In the [filter:keystoneauth] section, configure the operator roles:

          [filter:keystoneauth]
           use = egg:swift#keystoneauth
           ...
           operator_roles = admin,user
        5. In the [filter:authtoken] section, configure Identity service access:

          [filter:authtoken]
           paste.filter_factory = keystonemiddleware.auth_token:filter_factory
           ...
           auth_uri = http://controller:5000
           auth_url = http://controller:35357
           memcached_servers = controller:11211
           auth_type = password
           project_domain_name = default
           user_domain_name = default
           project_name = service
           username = swift
           password = SWIFT_PASS
           delay_auth_decision = True

          Replace SWIFT_PASS with the password you chose for the swift user in the Identity service.

          NB: Comment out or remove any other options in the [filter:authtoken] section and make sure [filter:authtoken] line is not commented out.

        6. In the [filter:cache] section, configure the memcached location:

          [filter:cache]
          use = egg:swift#memcache
          ...
          memcache_servers = controller:11211
    2. Configure the Storage Devices [link]

      1. This would normally be done on another server or VM but in our case both the swift server and the storage node will be on the same machine (VM)

        Remember those virtual hard disks we created earlier. This is when they come into use. Verify that these storage devices are located at /dev/sdb , /dev/sdc and /dev/sdd. To do this you can use:

        # fdisk -l /dev/sd*

        This command will list all devices starting with /dev/sd and you should 4 of them, sda being the main hard drive and sdb/c/d being the storage devices we are going to use. If these are different on your PC then take note of the names of the devices (or check if they are available in the VM (back to section 1.c)

        Although Object Storage supports any file system with extended attributes (xattr), testing and benchmarking indicate the best performance and reliability on XFS.

      2. Install supporting utility packages:

        # apt-get install xfsprogs rsync
      3. Format the /dev/sdb,  /dev/sdc and /dev/sdd devices as XFS:

        # mkfs.xfs /dev/sdb
        # mkfs.xfs /dev/sdc
        # mkfs.xfs /dev/sdd
      4. Create the mount point directory structure:

        # mkdir -p /srv/node/sdb
        # mkdir -p /srv/node/sdc
        # mkdir -p /srv/node/sdd
      5. Edit the /etc/fstab file and add the following to it:

        /dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
        /dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
        /dev/sdd /srv/node/sdd xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
      6. Mount the devices:

        # mount /srv/node/sdb
        # mount /srv/node/sdc
        # mount /srv/node/sdd
      7. Create or edit the /etc/rsyncd.conf file to contain the following:

        uid = swift
        gid = swift
        log file = /var/log/rsyncd.log
        pid file = /var/run/rsyncd.pid
        address = HOST_IP
        
        [account]
        max connections = 2
        path = /srv/node/
        read only = False
        lock file = /var/lock/account.lock
        
        [container]
        max connections = 2
        path = /srv/node/
        read only = False
        lock file = /var/lock/container.lock
        
        [object]
        max connections = 2
        path = /srv/node/
        read only = False
        lock file = /var/lock/object.lock

        Replace HOST_IP with the IP address of the management network on the storage node.

      8. Edit the /etc/default/rsync file and enable the rsync service:

        RSYNC_ENABLE=true
      9. Start the rsync service:

        # service rsync start
    3. Configure Storage Node components [link]

      1. Install packages:

        # apt-get install swift swift-account swift-container swift-object
      2. Obtain the accounting, container, and object service configuration files from the Object Storage source repository:

        # curl -o /etc/swift/account-server.conf \
        https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/newton
        # curl -o /etc/swift/container-server.conf \
        https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample?h=stable/newton
        # curl -o /etc/swift/object-server.conf \
        https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample?h=stable/newton
      3. Edit the /etc/swift/account-server.conf file and complete the following actions:
        1. In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory:

          [DEFAULT]
          ...
          bind_ip = HOST_IP
          bind_port = 6002
          user = swift
          swift_dir = /etc/swift
          devices = /srv/node
          mount_check = True

          Replace HOST_IP with the IP address of the VM.

        2. In the [pipeline:main] section, enable the appropriate modules:

          [pipeline:main]
          pipeline = healthcheck recon account-server
        3. In the [filter:recon] section, configure the recon (meters) cache directory:

          [filter:recon]
          use = egg:swift#recon
          ...
          recon_cache_path = /var/cache/swift
      4. Edit the /etc/swift/container-server.conf file and complete the following actions:
        1. In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory:

          [DEFAULT]
          ...
          bind_ip = HOST_IP
          bind_port = 6001
          user = swift
          swift_dir = /etc/swift
          devices = /srv/node
          mount_check = True

          Replace HOST_IP with the IP address of the VM.

        2. In the [pipeline:main] section, enable the appropriate modules:

          [pipeline:main]
          pipeline = healthcheck recon container-server
        3. In the [filter:recon] section, configure the recon (meters) cache directory:

          [filter:recon]
          use = egg:swift#recon
          ...
          recon_cache_path = /var/cache/swift
      5. Edit the /etc/swift/object-server.conf file and complete the following actions:
        1. In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory:

          [DEFAULT]
          ...
          bind_ip = HOST_IP
          bind_port = 6000
          user = swift
          swift_dir = /etc/swift
          devices = /srv/node
          mount_check = True

          Replace HOST_IP with the IP address of the VM

        2. In the [pipeline:main] section, enable the appropriate modules:

          [pipeline:main]
          pipeline = healthcheck recon object-server
        3. In the [filter:recon] section, configure the recon (meters) cache and lock directories:

          [filter:recon]
          use = egg:swift#recon
          ...
          recon_cache_path = /var/cache/swift
          recon_lock_path = /var/lock
      6. Ensure proper ownership of the mount point directory structure:

        # chown -R swift:swift /srv/node
      7. Create the recon directory and ensure proper ownership of it:

        # mkdir -p /var/cache/swift
        # chown -R root:swift /var/cache/swift
        # chmod -R 775 /var/cache/swift
    4. Create and distribute initial rings

      1. Create Account Ring

        1. Change to the /etc/swift directory.
        2. Create the base account.builder file:

          # swift-ring-builder account.builder create 10 3 1
        3. Add each storage node to the ring:

          # swift-ring-builder account.builder \
            add --region 1 --zone 1 --ip HOST_IP --port 6002 \
            --device DEVICE_NAME --weight DEVICE_WEIGHT

          Replace HOST_IP with the IP address of the management network on the storage node. Replace DEVICE_NAME with a storage device name on the same storage node. Since we only using one storage node with three devices, the HOST_IP will remain the same for all three devices. The three commands you will then enter are:

          # swift-ring-builder account.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6002 --device sdb --weight 100
          # swift-ring-builder account.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6002 --device sdc --weight 100
          # swift-ring-builder account.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6002 --device sdd --weight 100
        4. Verify the ring contents:

          # swift-ring-builder account.builder

          The results must be a table showing the three devices. Make sure the IP address is HOST_IP and the ports are 6002.

        5. Rebalance the ring:

          # swift-ring-builder account.builder rebalance
      2. Create Container Ring

        1. Change to the /etc/swift directory.
        2. Create the base container.builder file:

          # swift-ring-builder container.builder create 10 3 1
        3. Add each storage node to the ring:

          # swift-ring-builder container.builder \
            add --region 1 --zone 1 --ip HOST_IP --port 6001 \
            --device DEVICE_NAME --weight DEVICE_WEIGHT

          Replace HOST_IP with the IP address of the management network on the storage node. Replace DEVICE_NAME with a storage device name on the same storage node. Since we only using one storage node with three devices, the HOST_IP will remain the same for all three devices. The three commands you will then enter are:

          # swift-ring-builder container.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6001 --device sdb --weight 100
          # swift-ring-builder container.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6001 --device sdc --weight 100
          # swift-ring-builder container.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6001 --device sdd --weight 100
        4. Verify the ring contents:

          # swift-ring-builder container.builder

          The results must be a table showing the three devices. Make sure the IP address is HOST_IP and the ports are 6001.

        5. Rebalance the ring:

          # swift-ring-builder container.builder rebalance
      3. Create Object Ring

        1. Change to the /etc/swift directory.
        2. Create the base object.builder file:

          # swift-ring-builder object.builder create 10 3 1
        3. Add each storage node to the ring:

          # swift-ring-builder object.builder \
            add --region 1 --zone 1 --ip HOST_IP --port 6000 \
            --device DEVICE_NAME --weight DEVICE_WEIGHT

          Replace HOST_IP with the IP address of the management network on the storage node. Replace DEVICE_NAME with a storage device name on the same storage node. Since we only using one storage node with three devices, the HOST_IP will remain the same for all three devices. The three commands you will then enter are:

          # swift-ring-builder object.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6000 --device sdb --weight 100
          # swift-ring-builder object.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6000 --device sdc --weight 100
          # swift-ring-builder object.builder add \
            --region 1 --zone 1 --ip HOST_IP --port 6000 --device sdd --weight 100
        4. Verify the ring contents:

          # swift-ring-builder object.builder

          The results must be a table showing the three devices. Make sure the IP address is HOST_IP and the ports are 6000.

        5. Rebalance the ring:

          # swift-ring-builder object.builder rebalance
    5. Finalise Installation

      1. Obtain the /etc/swift/swift.conf file from the Object Storage source repository:

        # curl -o /etc/swift/swift.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample?h=stable/newton
      2. Edit the /etc/swift/swift.conf file and complete the following actions:
        1. In the [swift-hash] section, configure the hash path prefix and suffix for your environment.

          [swift-hash]
          ...
          swift_hash_path_suffix = HASH_PATH_SUFFIX
          swift_hash_path_prefix = HASH_PATH_PREFIX

          Replace HASH_PATH_PREFIX and HASH_PATH_SUFFIX with unique values (make them simple this is just a demo)

          Warning: Keep these values secret and do not change or lose them.

        2. In the [storage-policy:0] section, configure the default storage policy:

          [storage-policy:0]
          ...
          name = Policy-0
          default = yes
      3. Ensure proper ownership of the configuration directory:

        # chown -R root:swift /etc/swift
      4. Restart the Object Storage proxy service including its dependencies:

        # service memcached restart
        # service swift-proxy restart
      5. Start the Object Storage services:

        # swift-init all start
      6. NB: The storage node runs many Object Storage services and the swift-init command makes them easier to manage. You can ignore errors from services not running on the storage node.
    6. Verify Operation

      1. Source the demo credentials:

        $ source demo-openrc
      2. Create a test file:

        $ touch test.file
      3. Show the service status:

        $ swift stat
                                 Account: AUTH_ed0b60bf607743088218b0a533d5943f
                              Containers: 0
                                 Objects: 0
                                   Bytes: 0
             X-Account-Project-Domain-Id: default
                             X-Timestamp: 1444143887.71539
                              X-Trans-Id: tx1396aeaf17254e94beb34-0056143bde
                  X-Openstack-Request-Id: tx1396aeaf17254e94beb34-0056143bde
                            Content-Type: text/plain; charset=utf-8
                           Accept-Ranges: bytes
      4. Create container1 container:

        $ openstack container create container1
         +---------------------------------------+------------+------------------+
         | account                               | container  | x-trans-id       |
         +---------------------------------------+------------+------------------+
         | AUTH_ed0b60bf607743088218b0a533d5943f | container1 | 8cd68-0056f00a4a |
         +---------------------------------------+------------+------------------+
      5. Upload a test file to the container1 container:

        $ openstack object create container1 test.file
         +------------+------------+----------------------------------+
         | object     | container  | etag                             |
         +------------+------------+----------------------------------+
         | test.file  | container1 | ee1eca47dc88f4879d8a229cc70a07c6 |
         +------------+------------+----------------------------------+
      6. List files in the container1 container:

        $ openstack object list container1
         +-----------+
         | Name      |
         +-----------+
         | test.file |
         +-----------+
      7. Delete test.file on your machine.

        $ rm test.file
      8. Download a test file from the container1 container:

        $ openstack object save container1 test.file
      9. Verify that the file has been downloaded onto your machine.

        $ ls test.file
         test.file

        Congratulations you have successfully installed OpenStack’s Swift Object Storage Service!

  5. Troubleshooting

If you ran into any problems and you are trying to debug the service. Here’s where you can start:

    1. Do not ignore errors while setting up. They escalate very easily (talking from experience)
    2. Restart the machine (this usually works).
    3. Look at the logfile in /var/log/syslog and look for any anomalies and unexpected behaviour.
    4. Make sure that services used by Swift are all running. These are the Swift proxy server, account server, container server and object server. You can restart these services using “# sudo swift-init all start” and take not of any errors or warnings.
    5. Make sure that the Identity service is working properly. Refer back to 3.e.
    6. Error 503: You may get this error when trying to verify if swift is running properly (when you enter “swift stat”. This error usually means that you are successfully connecting to the swift proxy server BUT that proxy server cannot successfully connect to the storage nodes or identity service for authentication. First step is to restart the system.
      1. Verify that the ip addresses and port numbers in account-server.conf, container-server.conf,object-server.conf and proxy-server.conf are correct (NB: (bind-ip is commented out in proxy-server.conf)
      2. Verify that the details under [filter:authtoken] in proxy-server.conf are correct.
    7. Error 404: This error means that you can’t even connect to the proxy-server and this is usually a network setup error.
      1. Make sure the IP address you have been using as HOST_IP is correct. If it isn’t, go through the whole setup again using the correct IP address for HOST_IP.
      2. Verify that the hostname of the machine is “controller”.
      3. Check that /etc/hosts has an entry for “controller” and is it mapped to the VM’s IP address.
    8. Look at OpenStack’s troubleshooting guide. It’s not very useful sadly.
    9. If all else fails, Google is your friend 🙁

RECENT POSTS