In this lab, we will configure lrbd to use iSCSI access to Ceph rbd images.
As always don't forget read the full documentation:
https://www.suse.com/documentation/ses-3/bookstorageadmin/data/cephiscsiiscsi.html

This ugly ascii represents the lab:

                                                 +-----+
                                 +-------------> |     |LINUXCLIENT
                                 |               |     |192.168.100.40
SES3 CLUSTER                     |               |     |
   +-----+                       |               |     |
   |     |                    +--+--+            |     |
   |     |                    |     |            |     |
   |     |                    |     |            +-----+
   |   +-----+                |     |GATEWAY
   |   | |   |                |     |192.168.100.30
   |   | |   +------------->  |     |            +-----+
   +-----+   |                |     |            |     |
       |     |                +--+--+            |     |WINCLIENT
       |   +-----+               |               |     |192.168.100.27
       |   | |   |               |               |     |
       +-----+   |               +-------------> |     |
           |     |                               |     |
           |     |                               +-----+
           |     |
           |     |
           +-----+

In SES3 Admin, create a rbd image, 5 GB per example

sesadm:~ # rbd -p rbd create linuximage --size 5120  
sesadm:~ # rbd -p rbd list  
linuximage  
sesadm:~ #  

In gateway install ceph_iscsi

gateway:~ # zypper in -t pattern *ceph_iscsi*  

Now, I assume we are familiar with the concepts that we talk in the previous post, so let's configure lrdb

lrbd -e

{
    "auth": [
        {
            "target": "iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644", 
            "authentication": "none"
        }
    ], 
    "targets": [
        {
            "target": "iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644", 
            "hosts": [
                {
                    "host": "gateway", 
                    "portal": "portal1"
                }
            ]
        }
    ], 
    "portals": [
        {
            "name": "portal1", 
            "addresses": [
                "192.168.100.30"
            ]
        }
    ], 
    "pools": [
        {
            "pool": "rbd", 
            "gateways": [
                {
                    "target": "iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644", 
                    "tpg": [
                        {
                            "image": "linuximage"
                        }
                    ]
                }
            ]
        }
    ]
}

Where
target = Gateway iscsi-name
hostname = Gateway uname -n
addresses = Gateway IP addresses
image = Ceph RDB image

Run lrbd to apply the changes and verify the config

gateway:~ # targetcli ls  
o- / ............................................................ [...]  
  o- backstores ................................................. [...]
  | o- fileio ...................................... [0 Storage Object]
  | o- iblock ...................................... [0 Storage Object]
  | o- pscsi ....................................... [0 Storage Object]
  | o- rbd ......................................... [1 Storage Object]
  | | o- linuximage ............... [/dev/rbd/rbd/linuximage activated]
  | o- rd_mcp ...................................... [0 Storage Object]
  o- ib_srpt .............................................. [0 Targets]
  o- iscsi ................................................. [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644  [1 TPG]
  |   o- tpg1 ............................................... [enabled]
  |     o- acls .............................................. [0 ACLs]
  |     o- luns ............................................... [1 LUN]
  |     | o- lun0 .......... [rbd/linuximage (/dev/rbd/rbd/linuximage)]
  |     o- portals ......................................... [1 Portal]
  |       o- 192.168.100.30:3260 .................. [OK, iser disabled]
  o- loopback ............................................. [0 Targets]
  o- qla2xxx .............................................. [0 Targets]
  o- tcm_fc ............................................... [0 Targets]
  o- vhost ................................................ [0 Targets]
gateway:~ #  

In the client (linux), try to discover the targets of gateway and login

linuxclient:~ # iscsiadm -m discovery -t sendtargets -p gateway.ses.lab  
192.168.100.30:3260,1 iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644  
linuxclient:~ #  
linuxclient:~ # iscsiadm -m node -p gateway.ses.lab --login  
Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644, portal: 192.168.100.30,3260] (multiple)  
Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644, portal: 192.168.100.30,3260] successful.  
linuxclient:~ #  

Confirm

linuxclient:~ # lsscsi  
[0:0:0:0]    cd/dvd  QEMU     QEMU DVD-ROM     1.2.  /dev/sr0
[2:0:0:0]    disk    SUSE     RBD              4.0   /dev/sda
linuxclient:~ #  
linuxclient:~ # multipath -ll  
3600140536e1b05849e131b48c9390dd0 dm-0 SUSE,RBD  
size=5.0G features='0' hwhandler='0' wp=rw  
`-+- policy='service-time 0' prio=1 status=active
  `- 2:0:0:0 sda 8:0   active ready running
linuxclient:~ #  

Now you are able to format and use your new iSCSI volume

linuxclient:~ # mkfs.xfs /dev/mapper/3600140536e1b05849e131b48c9390dd0  
log stripe unit (4194304 bytes) is too large (maximum is 256KiB)  
log stripe unit adjusted to 32KiB  
meta-data=/dev/mapper/3600140536e1b05849e131b48c9390dd0 isize=256    agcount=9, agsize=162816 blks  
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=1310720, imaxpct=25  
         =                       sunit=1024   swidth=1024 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0  
log      =internal log           bsize=4096   blocks=2560, version=2  
         =                       sectsz=512   sunit=8 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0  
linuxclient:~ #  
linuxclient:~ # mount /dev/mapper/3600140536e1b05849e131b48c9390dd0 /mnt  
linuxclient:~ # df -h  
Filesystem                                     Size  Used Avail Use% Mounted on  
/dev/vda2                                       48G  1.6G   45G   4% /
devtmpfs                                       1.9G  8.0K  1.9G   1% /dev  
tmpfs                                          1.9G     0  1.9G   0% /dev/shm  
tmpfs                                          1.9G  9.5M  1.9G   1% /run  
tmpfs                                          1.9G     0  1.9G   0% /sys/fs/cgroup  
/dev/mapper/3600140536e1b05849e131b48c9390dd0  5.0G   33M  5.0G   1% /mnt
linuxclient:~ #  

Done!

Now we will add and configure a new image to use with a M$ Win client. (10 GB, only to notice at a glance the difference between the previous one)

sesadm:~ # rbd -p rbd create winclient --size 10240  

Add the new image at the lrdb Gateway configuration
For this lab, I'll use one portal (interface) and one gateway with no authentication, so only need add the image at the pool in the config file
This is the bottom of my config file with the new image added

  "pools": [
    {
      "pool": "rbd",
      "gateways": [
        {
          "target": "iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644",
          "tpg": [
            {
              "image": "linuximage"
            },
            {
              "image": "winclient"
            }
          ]
        }
      ]
    }
  ]
}

Now save the config with lrbd and check the changes

gateway:~ # targetcli ls  
o- / ..................................................................... [...]  
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rbd ................................................. [2 Storage Objects]
  | | o- linuximage ........................ [/dev/rbd/rbd/linuximage activated]
  | | o- winclient ............................ [/dev/rbd/rbd/winclient activated]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ....................................................... [0 Targets]
  o- iscsi .......................................................... [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.gateway.x86:sn.50ebf9fc6644 ......... [1 TPG]
  |   o- tpg1 ........................................................ [enabled]
  |     o- acls ....................................................... [0 ACLs]
  |     o- luns ....................................................... [2 LUNs]
  |     | o- lun0 ................... [rbd/linuximage (/dev/rbd/rbd/linuximage)]
  |     | o- lun2 ....................... [rbd/winclient (/dev/rbd/rbd/winclient)]
  |     o- portals .................................................. [1 Portal]
  |       o- 192.168.100.30:3260 ........................... [OK, iser disabled]
  o- loopback ...................................................... [0 Targets]
  o- qla2xxx ....................................................... [0 Targets]
  o- tcm_fc ........................................................ [0 Targets]
  o- vhost ......................................................... [0 Targets]
gateway:~ #  

As you can see now, we have a new LUN pointed to the new image
In the Win Client using the iSCSI Initiator, connect to the Gateway and "login", then add and format the new iSCSI volume

Done!

BONUS

If you are curious, with this lab done you can easily test the CEPH thin provisioning

"Digital for storage and quickness. Analog for fatness and warmth."

Billy Vera © 2017. All Rights Reserved.

The opinions expressed in this blog are purely my own and are not in any way endorsed by my employer.

Proudly published with Ghost using Ghostium Theme