Imaging VMware ESX Guest using ImageX
VMware ESX has the ability to clone templates, and this is a great feature when using iSCSI or FC datastores. When leveraging NFS datastores however, you lose thin provisioning on those NFS datastores. One way to get around this is to use a third party imaging software.
Microsoft provides a free imaging package titled imageX. You can read about on technet from here. A quick synopsis is that, unlike other imaging software, ImageX focuses on the files instead of disk blocks. This allows ImageX to leverage a Single Instance Store (SIS). Image X will compress the first image at around 33 to 50% of the on disk size, and will store the image in a file with a WIM extension. The WIM file holds the SIS and also indexes of which blocks of data are associated with the image. The benefit of the SIS and imageX can be found when appending a second image to the WIM. imageX will create a second index in the WIM file. It will then start imaging the machine, it will find a file and compare that to the SIS. If the file is found, it will add a pointer to the new index and move on. If the file is not found, imageX will add the file into the SIS and then add a pointer in the new index and move on.
To use imagex, you will need to download the Windows Automated Installation Kit (WAIK). After installing the WAIK, you can follow the instructions found here, on svrops.com, to create a WinPE boot CD. Before you create the CD using he OSCDIMG, you will need to inject the Network and SCSI drivers required for ESX. To do this, you will first need to get the correct drivers. You can either scour the internet looking for them, or you can just select install VMware tools from a guest VM. On the Guest VM, open my computers, Open the CDrom, \program files\VMware\VMware Tools\Drivers\ and copy the Vmxnet and SCSI folders. Now, on the machine that you installed the WAIK on, run the following commands
peimg /inf=DRIVE:<location you copied the network driver>vmxnet.inf /image=DRIVE:<mount location of wimfile>
peimg /inf=DRIVE:<location you copied the network driver>vmscsi.inf /image=DRIVE:<mount location of wimfile>
you can also use the same commands to inject other drivers, if you are going to leverage the boot CD across other hardware also. After you’ve injected all of the drivers that you plan on, go ahead and complete the instructions found on svrops.com. Now, after the OSCDIMG command, you will have a bootable WINPE iso. One point, make sure that you when you run the unmount command, ensure that you use the /commit command. If not, all of your changes will be lost, and you get to do it again.
Image Capture
So now that you have a bootable winpe iso, with imagex and the correct drivers, what do you do. First, build a gold image(s) of Windows XP, vista, 2003 and/or 2008. Patch everything with the latest service packs, security patches, etc… Also, its best to build the initial gold image with multi processors. That way you can use the same image for single/multi processors with out needing to change the HAL. next sysprep the gold image. You can find instructions for running sysprep here. After running the sysprep, start the VM and mount the CDrom as the iso created.
After booting, you will need to mount a network share, use a command similar to: Net use m: \\<servername>\share.
Create a new VM manually. It doesn’t need to be identical, but ensure that the hard disks are large enough to hold the uncompressed data from the gold image. Next, boot to the ImageX WinPE iso created earlier. After booting up, you will need to run diskpart, you can find websites online that detail everything about diskpart, but to create a basic C drive, you will need to run the following commands.
Diskpart.exe
Select disk 0
Create partition primary size=<size of disk in Mbytes>
Select partition 1
Format fs=NTFS label=”Sys” Quick
Active
Now, for each additional disk, select disk <disk> and run all of the same commands as above except, change the label to a description of the drive. Also, run the active command on the sys drive. After you have configured all of the drives type exit to get back to the command prompt. From the command line, run the net use command again. Next, change to X:\program files\imagex\. Type imagex /apply m:\<wimfile>.wim <index number> c: /verify. After the image is applied, you can rerun the imagex /apply command change the index number and the drive. After complete, unmount CDrom and reboot the VM. You should now be greeted by the Windows mini-setup.
While not as fast as VMware builtin clone from template, it does allow you to continue to leverage the thin provisioning inherent in NFS datastores. In our environments, it takes roughly 20 minutes to build a Windows 2003 VM, versus about 10 minutes to build the same VM from template.
Troubleshoogling
This weekend was one of those weekends, spent more on Google, looking for potential fixes, than with our families. One problem was a firewall was put in place between two sites and they were not correctly configured to allow extended DNS to pass. The firewalls used DNS guard and a limit of 512 bytes on UDP traffic. DNS resolution traffic uses UDP and would normally fit under the 512 limit. Windows 2003 DNS leveraged extended DNS. The Pix firewall would stop the UDP traffic when it exceeded 512 Bytes and DNS Guard would terminate the session. We were able to isolate the issue through nslookup and reviewing the firewall logs. In nslookup, when the server was changed to one of those behind the firewall, we would get request timed out. When we started looking at the firewall logs, we saw a second DNS request packet coming from the server, using the same session number. After some Google research, we were able to conclude that our issue had to do with the UDP packet size and DNS guard. By increasing the size allowed for UDP to 4096, we eliminated the errors seen in both the firewall logs and nslookup.
Well after we put that problem to bed, we ran into other issues that one by one, we were either able to find an exact fix for or information that point us in the right direction, on Google. Today, a coworker and I were laughing about it, when we started throwing out phrases. Google troubleshooting, trouble Googling; but the one we like the best was troubleshoogling. It is amazing the amount of data related to troubleshooting that can be found by having the right search criteria in Google, which is the hardest part of troubleshoogling.
Microsoft Licensing 3 – Clusters
In this post, I’ll discuss licensing when working in a clustered Microsoft environment. I’ll pick some of the more common Microsoft Apps and detail what is required to properly license them. Windows 2003 and 2008 support eight node clusters. In a two node cluster, you can technically have them configured in an active/active configuration. However, this is not considered best practice by Microsoft. They recommend running in an Active/Passive configuration. Three, four and five node configurations must have one passive node; the other nodes can be active. In the cluster, you can only have up to four active nodes, so nodes five through eight must be passive. All nodes must be licensed with either Windows Server Enterprise or Windows Server Datacenter. In most cases, Windows Server Enterprise makes the most financial sense. Enterprise has an MSRP of $4,000 per server up to eight physical processors versus $3,000 per physical processor using the Datacenter SKU. If there are more than 8 physical processors, you must use Datacenter.
Hyper-V cluster
The Datacenter SKU makes the most sense as the basis of your Hyper-V cluster in most Hyper-V environments. With the free unlimited guest OS licensing on each server, the breakeven point is 8 guest OS across the 2 node cluster. You receive the right to run 4 instances of Windows OS with each license of Enterprise. In a cluster, during a failover situation, more than 4 VMs might be running on the single node. Therefore, you would need to buy an additional 4 licenses of Standard. That would put Enterprise and datacenter, both at $12,000 for a 2 node cluster. Above 8 guests on the cluster, or when you are running copies of Enterprise on the guest VMs. If you have four physical processors, you would need to run roughly 20 guest VMs to break even.
This is a good point to add a quick discussion about virtualized environment hardware. Should you buy bigger 4 or 8 processor machines with multi-core or go wide in your cluster with dual processor boxes. When I’m designing clusters for a virtualized environment whether VMware Infrastructure or Hyper-V, I go wide first, and then scale up. My reasoning for this is simple, in both the VMware licensing model and Microsoft’s Datacenter licensing model; it is per socket costs. If you have 4 cluster nodes of dual processors or 2 cluster nodes of 4 processors. Both VMware and Microsoft will charge the same cost. Normally though the hardware costs to purchase two quad processor servers would be higher than four dual processor servers. You also gain the ability to have a higher utilization rate when going wide. In a two node cluster, you can only run at 50% capacity. But with a 4 node cluster, all four nodes can run at 75%. Another issue is that when running the larger hardware, a single physical server failure will shutdown basically 50% of your environment until they restart on the other node. When going wide, only about 25% of your environment will go down.
SQL Server
With SQL server, you would more than likely use the Windows 2003 Enterprise license. Unless you’re servers have more than eight processors, and if they do, you probably need this blog entry to explain licensing. Starting in SQL 2005, SQL enterprise is no longer a requirement for a SQL cluster. The Enterprise license now gives you additional features such as data warehousing. Microsoft is generous with SQL licensing in the cluster. You do not need to purchase a license for the passive node. License the active node with either per device or processor licenses and go. Again, in the same scenario as with Hyper-V, the licensing is the same whether you have two eight processor servers versus eight dual processors servers. Again, if you need more processing power on your database, you probably don’t need this blog. I should be talking to youJ.
Exchange
In a clustered environment of Exchange, you must run Exchange Server Enterprise per the Exchange Server 2007: Platforms, Editions and Versions web page. Also, you need one copy of Exchange for each node in the cluster. You do not get the benefit of not licensing passive nodes like you would on the SQL cluster. Exchange Enterprise licenses have an MSRP of $4,000 per server. As previously stated, and supported by Dell and IBM tests, Exchange does not scale well above 2 processors. So again take the cluster wide.
There are whitepapers by Dell and VMware that Exchange actually scales better in the Virtual environment than in physical. On a quad processor quad core IBM server, VMware was able to scale to 16,000 mailboxes. This was done with eight dual vCPU VMs each hosting 2,000 users. A blog discussing this can be found here. Dell wrote a similar paper on a dual quad core server.
My suggestions here; skip Microsoft clustering; get a couple of dual quad core processor servers and two licenses of VMware ESX 3.5. Load the servers up with as much ram as they will take and buy Exchange 2007 standard at $700 per VM. Build two Exchange servers using Standby Continuous Replication (SCR) between the two. Configure a Rule in the VMware cluster to put the two servers on different Physical ESX hosts. The VMware HA will protect you from a physical hardware failure; SCR will minimize the impact of an OS or application failure on the primary Exchange server. Of course you will still need additional servers for the other functionality in Exchange 2007: Edge transport, Client Access, Hub transport and Unified messaging. But with the cost savings of not buying additional servers, you can build standalone VMs to provide each piece of the Exchange environment.
SharePoint Server
With SharePoint Server, the best play would be to run a network load balance cluster for the SharePoint front ends and place databases on SQL cluster above. This will be a significantly cheaper solution as it will not required cluster able hardware and would only required Windows Server Standard instead of Windows Server Enterprise. It would also provide as high if not better uptime as a clustered front end.
SQL Server Pricing: http://www.microsoft.com/sqlserver/2005/en/us/pricing.aspx
Exchange Server licensing: http://www.microsoft.com/exchange/howtobuy/default.mspx
Exchange licensing comparison: http://technet.microsoft.com/en-us/library/bb232170.aspx
Microsoft licensing 2
In a previous post, I spoke about licensing Window Server in a virtualized environment. Today, I’ll be addressing Client Access License or CAL, especially revolving around web facing. By web facing, I mean any server that services requests from any machine not owned by the firm. The underlying Windows OS license is not impacted whether it is web facing or private. Windows Standard is Windows Standard. To connect to a Windows server however, the machine in question needs to be accounted for with the purchase of a CAL. Purchasing a single license of Windows 2008 Standard gets you 5 shiny core CALs, Enterprise gets 25 CALs. In the datacenter licensing model, no CALS are extended’ but as seen in the previous post, found here, the savings more than make up for a few missing CALs. One gotcha in Windows CALs is that they are OS specific. If your firm purchased Windows 2003 CALs, you would need to upgrade to Windows 2008 CALs to connect to a 2008 server.
What do you get with the Core CAL? Here is a quote lifted from the Microsoft website discussing the Core CAL. “The Microsoft Core CAL Suite encompasses four fundamental Microsoft server products that provide your people with identity management, directory services, enterprise communication (e-mail, calendar functions, and scheduling), collaborative workspaces, and asset management. ” The interesting part of that quote is you now get an enterprise communication, previously known as an Exchange CAL. You also get a SharePoint CAL, and a Systems Center Configuration Manager Cal: Previously SMS. Pretty good, for a MSRP of approximately $40 a CAL, you get access to the standard functionality found in a Microsoft based IT environment: Active Directory, file and Print, Exchange, SharePoint and SCCM.
There is also an Enterprise CAL suite. The Enterprise CAL gets everything found in the Core CAL, plus Office communicator standard and enterprise, Rights Management and System Center Operations Manager (SCOM, previously known as MOM), Exchange 2007 Enterprise and Forefront Security. The extra functionality will cost you slightly around $125 per machine. Microsoft recommends that if you are going to roll out two or more of the systems found in the Enterprise CAL, it’s in your best interest financially to purchase enterprise CALs. You don’t have to specifically run one or the other either. If you only have 20 users running Office communicator or SCOM, buy 20 Enterprise CALs and buy the remaining CALs as Core. In larger numbers , however, this could become a burden on management as you have to ensure that you carry enough Enterprise CALs. You can also buy CALs specifically for each of the products in the Enterprise CAL. Again, this might become a management nightmare when dealing with large numbers of clients.
Now on to web facing servers: if you know that only users from your firm using devices that are properly accounted for in the CAL count. Then you have nothing to worry about. However, if anyone connects to a web facing server using anything other than a device with a CAL, you might need to an additional CAL type. I say might because Microsoft recently added a limited use External Connector to their Windows 2008 Web Server SKU. Microsoft now allows for up to 50 concurrent connections to their Web server product. Now this only affects web servers, if you have a terminal Server or file, file and print or need more than 50 concurrent connections to a web site. You would need to purchase an additional product. Bring on the Windows Server 2008 External connector License: Big name, easy functionality. If machines are connecting to a server other than a 2008 Web Server OS and they are not accounted for in your CAL count; you need an external connector. Luckily, Microsoft doesn’t stick it to us that bad, and they actually dropped the price from the 2003 version. The external connector license now costs $2000 MSRP.
Another area, where Microsoft gets a little extra money for web facing servers is through SQL licensing. Again, if you know exactly how many machines are connecting to a SQL server, you can buy that number of SQL CALs. You can purchase a Server plus 5 CAL suite of SQL Standard for $1849 with an additional CAL cost of $162 per client device. Now if you can’t guarantee that you know exactly how many users are going to connect through to your SQL server, such as in a public facing web server, you would need to license you SQL server by processor. This allows an unlimited number of connections, but it is significantly higher; $6,000 per processor. If you have a four processor SQL server, it would cost you $24,000. OK, you thinking but I only have one device connecting to it; the web server. Ah, good thought but Microsoft has already blocked that play. They believe that the web server is not the client; it is only the middle man in the delivery chain. You need to license the individual client machine; IE Joe public sitting at his Vista laptop sitting his living room watching the latest episode of Heroes on DVR. Ahem, sorry moving on.
So hopefully that gives you a basic understanding of Client Access licensing. A post on Cluster licensing will follow.
Microsoft Windows 2008 pricing: http://www.microsoft.com/windowsserver2008/en/us/pricing.aspx
Microsoft Core CAL: https://www.microsoft.com/calsuites/core.mspx
Microsoft Enterprise CAL: http://www.microsoft.com/calsuites/enterprise.mspx
Microsoft SQL Licensing: http://www.microsoft.com/sqlserver/2005/en/us/pricing.aspx
Microsoft licensing
We are beginning to go a usage audit to true-up our Microsoft licensing. For the most part, the licensing is straight forward. Use a product, get a license. Don’t use the product, don’t get a license. But where confusion creeps in is around items such as virtualization, Public web access, Clustering. In this blog I’ll discuss Microsoft licensing in the virtualization arena. I’ll write another entry on public web access and clustering within the next day.
Licensing in the virtualization arena:
You have three options for licensing the Windows Server operating system. The first is that you buy a license for each virtual machine based on whether it is running Windows 2003 Standard or Enterprise: Easy enough. Option two is a bit more tricky, according to the Microsoft licensing for Virtualization web page, you can run “…you to run up to four software instances at a time in virtual operating system environments (OSEs) on a server under a single server license.” The third option is to purchase a license of Windows 2003 Datacenter, which is licensed per socket, for each of your Physical Hypervisor Hosts. This allows you to run an unlimited number of Windows Server based guest VMs on that particular host.
Lets look at a quick cost benefit analysis of each licensing type. We will use a two node cluster of dual processor Quad core servers. We will exclude networking, storage, electrical and cooling consumption. Those would be similar under any of the three licensing options. I also won’t even begin to do a hardware cost comparison between physical and virtual as there is enough information on the web to make an accountant cry about how much you will save virtualizing your environment. We will use a Server vCPU to pCPU ratio of 5:1, which should give us roughly 40 vCPUs. Given that we need the overhead to allow a hardware failure, we will not account for the second Host node. We’ll break down the license usage as 34 Windows 2003 Standard and 6 Windows 2003 Enterprise Guests.
License option 1: (one license for each Guest VM)
| MSRP | Amount | Option 1 Cost | ||
| Std |
$1,000 |
34 |
$34,000 |
|
| Ent |
$4,000 |
6 |
$24,000 |
|
|
$58,000 |
||||
As you can see in the graphic above, the MSRP of those 40 servers would be approximately $58,000.
License option 2: (Windows Server Enterprise – 4 free on the same server)
| MSRP | Amount | Option 2 Cost | ||
| Ent |
$4,000 |
12 |
$48,000 |
|
Here is where it can a little dicey, the license states that you can run 4 instances of the OS on one server. When you license in the two node environment, especially when using a product such as VMware Infrastructure DRS; you can not be sure how many VMs will reside on one physical host at any one time. It might be 20-20 or it might be 22-18, etc.. While it would look like you only need 10 Enterprise licenses to cover those 40 servers, you would probably need at least 1 extra for each node to ensure that you never have more guest VMs running on one node. Even with purchasing two extra licenses of Windows Enterprise, you still save $6,000 over the one-license per guest option. Another benefit is that you can run either Standard or Enterprise and still be in the good graces of Microsoft.
License option 3: (Windows Server Datacenter – run what you brung!)
| MSRP | Amount | Option 3 Cost | ||
| DataCenter |
$3,000 |
4 |
$12,000 |
|
Here is where Microsoft licensing in the virtualized arena begins to shine. Microsoft DataCenter licensing has an MSRP of $2,999 per physical processor. Not per core, per physical socket. That means that for each node in the cluster, we need $6,000 worth of Microsoft OS, to cover everything. This licensing option also allows us the opportunity to load whatever OS, the business unit needs. Or, we just standardize on Windows Enterprise for the Virtualized servers and not worry about any features that are disabled on the standard version.
Microsoft Licensing for Virtualization: https://www.microsoft.com/licensing/highlights/virtualization.mspx
Virtualization Server Sprawl
When a firm is contemplating virtualization, there are many positives: server consolidation, improved DR, reduced energy consumption, reduced infrastructure costs, etc…. There are also negatives: increased risk due to single point of failure, additional complexity, server sprawl…
Server sprawl has long been a part of the Windows Server realm, due to the overwhelming mindset of one application – One Server. One of the few things that kept server sprawl in check was the cost to procure another server. The firm would need to purchase a new server, this normally entails
- Research (find systems that meet the application requirements)
- Verify datacenter can absorb additional server (are there enough network ports, are power and cooling sufficient, is there space in the rack?)
- Requesting quotes (must work with multiple vendors to ensure best value proposition)
- Select quote (best value proposition, not always lowest cost)
- Submitting the PO to purchasing (wait out the steeping period)
- Purchasing department orders the server
- Waiting for vendor to ship (anywhere from a week to a month)
- Request storage and networking ports from the groups responsible.
- Waiting for equipment to arrive in datacenter (we work in a union facility where we are not allowed to move equipment, this usually takes a week)
- Submit change request to rack mount the server and bring it on the network (this usually must happen after hours when the server is being installed in racks that have production servers)
Now, the administrator can begin configuring the server.
- Install OS
- Install Service Packs
- Patch
- Install VirusScan
- Install backup software
- Install monitoring software
- Configure monitoring
- Install support features for the application
- Patch support features
- Install application
- Test
- Verify
Now, what does this have to do with server sprawl? After virtualization, skip most of the steps required to purchase a server. Instead of the approximately two months to purchase new hardware, as soon as the request is made, the VM can built almost immediately and at almost zero cost to the firm. Of course there are costs, OS licensing, backup, virusscan, monitoring, storage. But the portion of the costs that are immediately seen by the purchasing department are only for the application.
When your firm decides to move forward on your virtualization project, do not underestimate the concept of server sprawl. My previous two firms after the virtualization infrastructure was in place, experienced significant server sprawl. Projects that were originally slated for two servers became 10. Applications that were on the bubble, immediately became a go. Applications that would have previously been denied due to limited use by only a few in the company, became feasible.
IWUG and Essential Business Server presentation
Last night’s Indiana Windows Users Group (IWUG) was a web conference presented by Nick King, A Microsoft Technical manager for the Essential Business Server team. While we were sitting enjoying Dinner , Nick was sitting in Redmond Washington.
What is Essential Business Server (EBS)? A Product sku designed for medium business that have one or two IT Pros on staff. It fills the gap between Small Business Server and just purchasing an assortment of Microsoft products. EBS comes in two flavors, Standard and Premium. Standard includes 3 licenses of 64bit Windows Server 2008 Standard, 2 licenses of Exchange 2007, 1 license of System Center Essentials 2007, 1 license of Forefront Security for Exchange, and 1 license of Forefront Threat Management Gateway Medium Business Edition (formerly ISA Server). Premium edition includes all of the licenses in the standard version plus and additional license of Server 2008 Standard and one license of SQL 2008 Standard. More Information can be found on Microsoft’s website at the EBS home page.
EBS is more than just a collection of Microsoft SKUs. Microsoft has created and administrative portal that leverages Power Shell to create a new status at a glance portal. It also provides wizards allowing you to create users, groups, computers, SharePoint sites, etc… all from a single point. One question, I had while watching the demo was, “wasn’t this the grand vision of the MMC?” it was supposed to be the only place the administrator went to manage their server. Oh, well another Windows version, another way to manage it. Moving on. EBS is a good deal cost wise, according to the Nick, the EBS skus, represent a 30% savings, over purchasing the components individually. Standard has a list price of $5472 and Premium has a list of $7163.
EBS also helps with the rollout for administrators who rolling out either their first AD environment or are upgrading from Small Business Server (SBS). Microsoft has reduced the install screens from roughly 120 down to approximately 30. Also, the management portal is tabbed to help focus the administrators on exactly what they need. According to the presenter, the portal allows the admin to perform 85% of all their Microsoft infrastructure work without leaving the portal.
What questions did I come away with?
First is that the requirements are 64 bit processors for all 3 servers in the standard sku, the SQL server in premium includes either 32 or 64 bit. All new servers come with 64 bit procs, but many smaller companies might not want to purchase new hardware, just to move to this version.
Second, the memory requirements are 4 GB each for the management and the Exchange server and 2GB for the Security Server. If you have to purchase new hardware, no big deal, but some older hardware might require more capital expenditures.
Overall, the Essential business server appears to be a good deal. We use a general rule of $4K per server for an HP Proliant DL380. If you need to purchase new hardware, you are looking at $12K for hardware and $5K for the ESB standard. This is roughly $17K before we even add in general file storage or backups. Add in an additional $6K to jump to the Premium sku and additional server.
The presenter’s blog can be found at http://blogs.technet.com/nking/
Microsoft Remote Desktop on IPhone
As a Windows administrator, there are many times where you are out and get the emergency call that a service is down and you need to come into the office. You need to drop everything and run to the office or the datacenter. Smart phones, specifically the Apple Iphone can help out. An application WinAdmin allows you to RDP into your server or desktop and work on the issue.
sounds good, what do I need to do?
- Have an Iphone or Ipod Touch
- Either connect through wireless broadband or WiFi
- Download WinAdmin through Apple Store on your unit
- Configure and connect through VPN client if outside of internal network
- Start WinAdmin and add new server
- connect to the server and work.
About mordtech
Experience:
I’m an IT professional with over ten years of Microsoft Server support. I’ve also supported Checkpoint and VMware. I’m currently employed by a large IT outsourcing corporation in the US.
I’ve implemented Windows Active Directory, Checkpoint VPN/Firewalls, VMware Infrastructure, Sharepoint/Project Server amongst others.
Personal:
I’m married and a father of two boys, which is by far the most trying job of my life: the boys, not the marriage. We reside in the midwest and aren’t looking forward to the fast approaching winter.
sitemeters
