You know iSCSI and VMware.. I bet not… Part 3

22 04 2014

Alright Part 1 we went over coming up with a plan for the networking, Part 2 we prepared our host to make sure it was ready for ESXi and had the latest firmware. So now it’s time for Part 3… Let’s get this party started. So here is what the host plans look like.

· 2 C-Series Servers with 6 network adapters each

o 2- NICs for Mgmt traffic

o 2-NICs for Prod traffic

o 2- NICs for iSCSI traffic

So keeping with Part 1 and the plan here is what I have set for IP Scheme’s and layout. Keep in mind that for iSCSI to work properly the recommended setup is if possible segment iSCSI traffic to its own switches. Also iSCSI traffic should not be routable. Finally with iSCSI each VMKernal Port for ISCSI needs its own unique subnet. For instance if you have 4 ports you wanted to assign to iSCSI then you will need to have four distinct subnets similar to this:

ISCSI 1- 192.168.11.10

ISCSI 2- 192.168.12.10

ISCSI 3- 192.168.13.10

ISCSI 4- 192.168.14.10

**Also note that while my lab I only have 10/100 switches in no way shape or form am I recommending to run this over a 10/100 switch. This should be ran over 1g or 10g connections period!**

Something special to note as well when even considering iSCSI “iSCSI should be considered a local-area technology, not a wide-area technology, because of latency issues and

security concerns. You should also segregate iSCSI traffic from general traffic. Layer-2 VLANs are a particularly good way to implement this segregation.” Best Practices for Running VMware vSphere®on iSCSI, Cormac Hogan “

Ok enough of that and let’s get back to business and feel free to reference Cormac Hogan’s white paper on the iSCSI Best Practices located here.

Host 1

vSwtich0

MGMT IP- 10.1.32.46 VLAN 132

· VMNIC0- Infra switch 1 port 1 (Primary for Mgmt)

VMotion- 10.1.35.46- VLAN 135

· VMNIC1-Infra switch 2 port 1 (Primary for vMotion)

vSwitch1

Production- VLAN 136

· VMNIC2- Infra switch 1 port 2

· VMNIC3- Infra switch 2 port 2

vSwtich2

ISCSI 1

· VMNIC4- ISCSI switch 1 port 1

· 10.1.33.46 – VLAN 133

vSwitch3

ISCSI 2

· VMNIC 5- ISCSI switch 2 port 1

· 10.1.42.46 – VLAN 142

Host 2

vSwitch0

MGMT IP- 10.1.32.47 VLAN 132

· VMNIC0- Infra switch 1 port 25

VMotion- 10.1.35.47- VLAN 133

· VMNIC1-Infra switch 2 port 25

vSwitch1

Production- VLAN 136

· VMNIC2- Infra switch 1 port 26

· VMNIC3- Infra switch 2 port 26

vSwitch2

ISCSI 1

· VMNIC4- ISCSI switch 1 port 25

· 10.1.33.47 – VLAN 133

vSwtich3

ISCSI 2

· VMNIC 5- ISCSI switch 2 port 25

· 10.1.42.47 -VLAN 142

Above is our plan and layout made so next we need to load up ESXi then start our configuration. So since I am using two Cisco C200 M2’s I will go and download the latest Customized ISO from VMware’s website. Since I am loading 5.5 you can find the customized ISO here. For those of you asking what gives Customized ISO? Well the Customized ISO contains all the approved drivers from Cisco based on the correlated firmware for their host, other manufactures put theirs here too Dell keeps there on their own website. You can also do it the long way and load the standard version of ESXi on the host and then load up all the drivers manually. If you want to make your own customized ISO’s one of my co-workers Jeremy Waldrop did a great blog post on how to do that here.

Ok so I will not bore anyone with how to load ESXi pretty straight forward download the Customized ISO either burn it to CD or mount the ISO through the CIMC and install the OS on the local drive on the host. If you want to do iSCSI Boot from SAN well you’re on your own there. This blog is not going to cover it, maybe later but at this time just say no to iSCSI BSAN J. So here is what this looks like in the wild.

Host 1

vSwtich0

MGMT IP- 10.1.32.46 VLAN 132

· VMNIC0- Infra switch 1 port 1 (Primary for Mgmt) notice that I put VMNIC1 as standby this forces the management traffic across VMNIC0 until VMNIC0 become unavailable and then will use VMNIC1

VMotion- 10.1.35.46- VLAN 135

· VMNIC1-Infra switch 2 port 1 (Primary for vMotion) when you put VMNIC1 on standby and then you create the next VMkernal port it will automatically set VMNIC1 as the primary port and VMNIC0 as standby

vSwitch1

Production- VLAN 136

· VMNIC2- Infra switch 1 port 2

· VMNIC3- Infra switch 2 port 2

vSwtich2

ISCSI 1

· VMNIC4- ISCSI switch 1 port 1

· 10.1.33.46 – VLAN 133

vSwitch3

ISCSI 2

· VMNIC 5- ISCSI switch 2 port 1

· 10.1.42.46 – VLAN 142

So your first host would look something like this as a Base to start from.

Host 2

vSwitch0

MGMT IP- 10.1.32.47 VLAN 132

· VMNIC0- Infra switch 1 port 25 (Primary for Mgmt) notice that I put VMNIC1 as standby this forces the management traffic across VMNIC0 until VMNIC0 become unavailable and then will use VMNIC1

·

VMotion- 10.1.35.47- VLAN 133

· VMNIC1-Infra switch 2 port 25 (Primary for vMotion) when you put VMNIC1 on standby and then you create the next VMkernal port it will automatically set VMNIC1 as the primary port and VMNIC0 as standby

vSwitch1

Production- VLAN 136

· VMNIC2- Infra switch 1 port 26

· VMNIC3- Infra switch 2 port 26

vSwitch2

ISCSI 1

· VMNIC4- ISCSI switch 1 port 25

· 10.1.33.47 – VLAN 133

vSwtich3

ISCSI 2

· VMNIC 5- ISCSI switch 2 port 25

· 10.1.42.47 -VLAN 142

Ok so now we have our base config done now we need to enable the software adapter for each of the host so we can get multipathing working correctly for iSCSI. So let’s go to our first host and enable the software iSCSI adapater.

To do this click on Storage Adapters and then click on Add

You only get this option when you click add just click OK

The next thing that pops up is an acknowledgement click OK

Once the install completes you will see ISCSI Software Adapter show up like this

***You will need to do this for all your host that you will be running ISCSI on.****

Once you get all the ISCSI Software Adapters installed on your host then we need to configure a few things on these ISCSI Software Adapters. Before we go into that lets just have a quick talk about Jumbo Frames. Depending on the manufacture of your array you are connecting to they will give you their recommended best practices for Jumbo Frames. I know for EMC they tell you to use Jumbo Frames only if you are doing large block IO or if you are doing 10 G ISCSI. I have worked with Dell a fair amount and their stance is Jumbo Frames end to end for their arrays regardless of I/O. If you are asking my personal stance on Jumbo Frames I say the increase is minimal in 1 G solutions so my personal opinion is unless the workload dictates that you enable Jumbo Frames then don’t make this any harder on yourself.

Prudent use of the KISS principle advised…. Keep It Simple Stupid J

Ok so let’s setup this ISCSI software adapter. Click on the Adapter and then go to Properties

First from this screen lets click the Advance Tab and disable Delayed Ack globally for this iSCSI Software Adapter

Scroll all the way to the bottom of the Advanced menu and remove this check mark

So should be like this J

Click OK

Personally to make things a bit more user friendly I would recommend going into Configure and providing an Alias on IQN so you know what belongs to what host similar to this.

Then this pops up.

Put an Alias that makes sense. For me since this is Host 1 I will just call this MS1-ISCSI-VMHBA35

Next we will click on The Network Configuration Tab.

Then Click Add. This will present you with the ports that are eligible candidates for doing this Port Binding.

Click the first one in the list and hit OK. Then you will see this.

Next we need to add the second path to this port binding. So we click Add

We select our second ISCSI port and click OK.

We now have our two ISCSI vSwitches using the Port Bindings so now we have failover setup correctly from the host perspective. When you click Close on the above screen you will be presented with this screen.

If this is a new setup and nothing has been configured to run on this host then go ahead and click Yes. If you are doing this on a production host that has running VM’s on it then click No and make sure you are in maintenance mode before you do a rescan using this ISCSI software adapter.

So at this point we have the Software adapter configured for mulitpathing you will need to do this on all host that you are going to be running ISCSI on your environment. One thing to note is that I recommend a reboot after enabling ISCSI Software Adapter and Delayed Ack to ensure all the settings stay. You may also be asking about CHAP and whether to use or not this really depends on your environment and what your requirements dictate. I will say this if you have CHAP setup and if for some reason you do not have your ISCSI network isolated CHAP is an extra layer of security to keep someone from erroneously attaching a LUN to the wrong server or worse wiping out a LUN without realizing what is in on the LUN.

Whewww that was a lot to go over for Part 3 so let’s take a breather and take that in and prepare for Part 4. So stay tuned for the upcoming post as we go over setting up our storage array and configuring the discovery targets on each of our host to discover the ISCSI LUNS presented to each host.

Advertisements

Actions

Information

One response

26 04 2014
Newsletter: April 26, 2014 | Notes from MWhite

[…] in part 2 with host stuff – meaning firmware updates everywhere, and in part 3 he talks about connections.  So a very good series so […]

Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s




%d bloggers like this: