Archive for the ‘Virtual Machine’ Category


At a central location, a Windows Server hosts a handful of Hyper-V virtual machines and also hosts the QB data files.  Users who are remote access the virtual machines over a VPN using Remote Desktop to get the experience of running QB locally; it performs abysmally over a VPN when the data is at one end and the desktop at another.

As a side note, performance for both users on the LAN where the server is and the virtual machines starting degrading substantially; by putting the Hyper-V machines and virtual disks, and the QB files on a SSD, performance jumped abut 10-fold; it’s as though everyone is using a high speed workstation.

What Happened?

I cannot say for certain what went south but one of the virtual machines would no longer run QB and gave the 3371 error.  Uninstalling QB and re-installing did not change that.  What I did notice, however, was that Edge would not resolve any URLs and appeared to Windows 10 to be offline, but there was network connectivity. The start button didn’t work, and there were a big handful of updates pending download and install.

I finally did a network reset from the settings page, rebooted and then had to make some adjustments to the virtual machine settings.  Primarily, this was to change the MAC address back to the value it was in the DHCP reservations; I wanted all the virtual machines to have a fixed IP address for the users.  That worked, and after a few reboots, the installs started to download, but they all remained pending.  i kept restarting but nothing budged.  Finally, I set the time for a restart attempt a few minutes ahead, and that got the list of updates to install when it rebooted.  Edge worked again.  But QB did not, even after a remove/reinstall.

What Did Work

The ultimate fix was very easy.

  1. I opened Control Panel and chose File Explorer Options.  On the View tab, I selected Show hidden files and folders and deselected the four Hide check boxes, accepting the error warning for the last one.
  2. I closed Control panel and opened File Explorer. I could now see ProgramData and clicked on it, eventually navigating to ” C:\ProgramData\Intuit\Entitlement Client\v8“.
  3. In that folder there is an ECML file, and I renamed it to .emcl.old.
  4. I closed File Explorer and then ran QB.
  5. QB then asked what version type (this was manufacturing and wholesale), and that setup ran.
  6. Then QB opened without an error, and all is well again.

I should add that updates show current and Windows 10 is back to behaving.

Hyper-V Followup

The virtual machines are just running QB; users use their own desktops for other work, so the VMs are pretty static.  What I am now doing is making copies of the .vhdx files for the machines in a separate folder (on the SSD, but it doesn’t really matter; could be anywhere including the cloud).  I actually had a copy for this particular machine but it was about four months old, and I had a feeling that I could fix the problem so I pressed on.  But if a VM went south again soon, I just have to replace the .vhdx file with the saved copy.  Sure beats dealing with a handful of physical machines.  Oh, yeah, I didn’t even have to be there to do this.


 Virtual Machines Have Prevented Disasters More than Once

Let me tell you about the latest situation where I look like a hero instead of a fool.  How can you not love someone who does that for you?

I have a client who is still on SBS 2011 for a variety of reasons.  Let’s sum them up by saying it is unlikely to change for a while.

A few years back, I suggested making SBS a virtual machine and running on a Hyper-v host.  The OS for the host was Windows Server 2008 R2.  Earlier this year, both of the USB drives they were using for Windows Backup (one for the host, one for SBS) had to be replaced.  The new drives were formatted NTFS with 4096 byte sectors.  If you don’t know already, Windows Backup in 2008 R2 and earlier can’t create a backup on these targets.  So for a while, the system has been running without backups.

Why not Just Upgrade?

Duh, why didn’t I think of that?  I did, but an inplace upgrade to 2012 R2 kept failing.  Finally, we located a 2012 Standard .iso and license for sale and grabbed that.  The in place upgrade went well, very well.  Much to our chagrin, the product key would not activate even though it was valid, and the client got his money back.  But there we were with an OS that was about to die from lack of activation.

So I tried the upgrade from 2012 to 2012 R2, and it succeeded.  Problem solved, right?

You Probably Picked the Wrong Answer

When it finally rebooted after the upgrade, a colorful blue screen appeared with the message


The scarce number of articles I found relating to this indicated that it was an invalid product key (I knew that) and re-installing would fix it.  So in goes the DVD and we boot from it.  Enter the correct 2012 R2 product key, select upgrade, and we are instructed to remove the disk and reboot.  When we do the same nasty error above appears.

So I Said…

Let’s just forget about the host system and create a new one.  I didn’t format any disks, just let the installation go.  It completed fine, booted into 2012 R2, and I added the Hyper-v role, reset the static IP address, and got down to setting up a new virtual machine.

I pointed to the existing .vhd for SBS and the virtual disks for Exchange, SharePoint, and data that were on physical drives on the host and attached them through ISCSI controllers, started the virtual machine and bingo!  There is was.  Almost.

I Nearly Cried when It First Started

Active directory showed NO users, NO joined computers, and SBS Console said the OS was not operative.  While I played around for a bit, then looked again after about 5-10 minutes, AD values were back!!  But there was no Internet connection and nothing looked right on the SBS Console network tab.

First I ran Connect to the Internet, and that restored the connection.  Then I ran Fix my network, and suddenly there was the trusted certificate and all the other goodies.  It has been running like a champ ever since.

One Last Thing

The 4096 sector drive for SBS still doesn’t work, so I am trying a WD drive that emulates 512 sectors,  I am going to create another post about this, although there is a lot of information on this topic out there already.

So, to sum it up, your Hyper-v host machine is disposable.  You can trash it and provided you don’t lose the date, you can reconstruct your workhorse servers.  I highly recommend this approach.

Even better, use an iSCSI device (like QNAP, which I love as well) and keep all your virtual information on the drives apart from your computer.  That means the entire platform is disposable.

The Scenario

I needed a Windows 7 OS to run an older version of an application on 32-bit, but I had converted my system to Windows 8 64-bit.  Since I had plenty of memory, I decided to run Windows 7 32-bit as a virtual machine using Hyper-v in Windows 8.  Since it is just like Hyper-v in Windows Server 2012, this was effortless.  I created the machine, installed Windows 7 32-bit on it, installed Office (needed for the app) and the app itself.

All was pretty heavenly, just clicked on the taskbar icon for the VM, and the app was at my fingertips.

Then something went wrong.  The data files could not be located on a network share.  It wasn’t the share but the whacky behavior of the network adapter on the virtual machine.  It could get a DHCP address from the server (release and renew worked), but it was otherwise isolated.  Kaput.  Couldn’t get to a thing.

First Attempt to Remedy

My thought was that the network adapter attached to the VM was faulty.  I could ping, but I either got destination not reachable or request timed out, often a mix of the two.  So I shut down the machine, went to settings, removed the network adapter and tried to add another one.  No luck.  It kept giving me a failure that it couldn’t add anything in the machine’s current state.

Tried to Create a New Virtual Machine

This seemed to indicate a problem with the instance of the virtual machine, and if I couldn’t remove devices, maybe I could use the existing .vhdx in a new machine.  I also decided to be clever and store it on a network share that was being backed up by Windows Server Backup.  Well, not so clever.  It wouldn’t allow me to stash the virtual machine there nor allow me to use the .vhdx copy I had placed there.  So back to storing om local drives.

But creating the virtual machine kept failing with an error that contained

“The requested operation cannot be performed on a file with user-mapped section open”

What the heck was that?  After some snooping, I realized it might be a permissions or blocking error.  Eliminating permissions, I decided to try and eliminate blocking. I had Trend Micro Worry Free Advanced running on the domain server, with an agent on this desktop, so I disabled it.

Sure enough, creating the virtual machine using the old .vhdx from the failed machine worked. And when it started, all was well with the network.

What I Learned

  1. You can re-use a virtual disk from an old virtual machine on a new virtual machine.
  2. Anti-virus software can block virtual machine creation and maybe ongoing operation.
  3. I needed material for a new post.

I hope this helps some of you expand your virtual Hype4r-v horizons.  It is a great tool.


I guess it had to happen, and that is why we do backups and more.  But sometimes it is not quite enough and this was one of them.

I was logged onto a test server where I am creating a SharePoint application.  This server is a virtual W 2008 R2 connected via a VPN to a virtual SBS host. While I was doing some work, I noticed the VPN connection dropped.  Strange, I thought.  I remoted to the host machine for the SBS server and saw in Hyper-v Manager that SBS was stopped.  I checked on the drive where the .vhd files are stored and noticed that it was absent from the host server.  Oh, just a dropped iSCSI volume – something that happened a few times before (more about that later).  Simple enough, just reconnect.

Imagine my surprise when I couldn’t reconnect to the iSCSI target LUN!   I am going to skip over the details of why, for now, and get to the bigger point.  I was able to do a power cycle on the iSCSI box, and of course I could re-connect to the target.  Now, just restart the SBS virtual machine.

Horror Show Part I

The machine would not start.  Hyper-v reported a disk corruption and could not use the .vhd.  Well, that was a minor set-back.  I had a USB drive attached to the iSCSI box that did daily backups of the LUN.  I’ll just restore that to another iSCSI target I would create, just in case.

It took about an hour and lo and behold, a new and freshly restored LUN with the .vhd files.  I’ll attach it and start the virtual SBS 2011.

Horror Show Part II

You already knew, I bet, that it wouldn’t be that easy.  Diagnosis?  The LUN to USB is a copy and replace, so only one backup to restore.  That backup occurred AFTER a disk failed and was in rebuild on a RAID 6.  Doing the power cycle caused some data corruption, seemingly limited to the SBS 2011 .vhd.  Hyper-v failed to boot with a missing boot file.  Sigh.

Sounds like being up that creek.  There was no Windows Server backup of that volume.

But, we had installed AppAssure’s Replay backup. Unfortunately, some disk issues had caused it to fail and we were investigating the cause.  As it turns out, engineers were evaluating the logs that morning from some I/O stress testing we did.  They called to say the iSCSI target was generating lots of retrys, and a call to the vendor of he iSCSI box helped us determine bad sectors on a drive that went into rebuild that eventually died and …. you get the idea.

Even though Replay was disabled when this happened, it had created lots of backups during the previous two weeks. I picked the most recent backup of the volume and mounted it.  Sure enough, there was my precious .vhd file.  I coped it to a new location ( a lengthy process), modified the machine settings to point to the new .vhd, and fired it up.

Horror Show Starts to End

YES! It started fine.  Problem solved!

Of course not,  All services started but Exchange mailboxes were not accessible.  Neither the mailbox nor the public store properly mounted.  Here are some of the things I did to solve the problem:

  1. Ran eseutil.exe with /mh option.  This reported that the database was in a dirty shutdown mode.
  2. Ran eseutil .exe with /r option.  This ran successfully, but a subsequent eseutil /mh still reported dirty shutdown.
  3. Ran eseutil.exe with /p option.  Ran successfully, then eseutil /mh reported clean shutdown.

In order to run eseutil.exe with the various options, try this article.  To keep from typing long and tedious paths to the database and to the log file folder, stash them somewhere on your desktop and copy and paste them at the appropriate parts of the command line.  Remember these points:

  • Enclose paths and file names that contain spaces in double quotes.  I forgot and used single quotes and it couldn’t find the files.
  • Run cmd as administrator.
  • Set the path to the database location for convenience.

If you have not moved Exchange files, the default location  is “C:\Program Files\Microsoft\Exchange Server\V14\Mailbox\<mailbox name>”; otherwise it is exactly the same path but a different drive letter.  Get the mailbox name form Exchange Management Console Organization Configuration Mailbox.  NO .edb for the path name, please!

The log files will be in the original path on drive C: unless you manually moved them.

Public database is at “C:\Program Files\Microsoft\Exchange Server\V14\Mailbox\PublicFolderDatabase” or on another drive letter if moved and the log files on drive C:

For the /r option, the log prefix is by default E00 for the mailbox and E01 for public folders.

And yes, I had to repair each.  I repaired them at the same time using two open CMD windows.

It will take a long time if you have a sizable mailbox.  A nearly 30GB one took around 5 hours.

The End, Almost

Once they mounted, mail began to flow and all appeared well.  Except… several iPhones were no longer receiving email.  And ActiveSync was reporting errors on them.

Fixing this was easy, but it deserves a separate post.


If you don’t know your locations precisely for eseutil, then rely on EMC.  Click on either the mailbox or public database that is unmounted, then chose Move Database Path from the actions pane.  In the first dialogue box that appears, the path to the database and the log files is there for you to copy and paste.  Remember you just want the database path or the path name unless you run eseutil from another path entirely.

When you run a repair with the /p option, you might loose bits of your mailbox.  I have done it before without loss, and this time the same thing.  But it is better than having nothing.

yes, Virginia, there is a Santa Claus who can help you convert your physical SBS server to a virtual machine to run under Hyper-v.  But dear, they don’t tell you nearly enough, so pay careful attention to this.  Especially if you want to user the same server box to be your new Hyper-v host machine.

Preliminary Work

The assumption is that you have SBS installed on a physical machine.  You would like to use that physical machine as a Hyper-v host.  How can you convert SBS to a virtual machine?  And what does it take to host Hyper-v on that box?

If your SBS is 2008 or 2011, you needn’t worry about whether the server can support 64-bit because both of those are 65-bit operating systems.  If, however, your starting point is SBS 2003, you need to verify that it can run a 64-bit OS.  If your server is from around the time when SBS 2003 was first released, I suggest you look at a new server to save yourself a lot of frustration, time and eventually money.  You can purchase a lovely box from Dell, HP and others for about $1500.  Then  you can do a migration to SBS 2011 between two physical boxes.  See my post about setting up SBS 2011 as a virtual machine and do the migration to it.

Once you are convinced your current server box will support the Hyper-v host os, check on how much memory you you have.  I would recommend16GM as the minimum necessary for a successful system and as much more as you can add past that to accommodate the number of virtual machines you plan on hosting.

I have a few virtual machines for SBS running between 8GB and 16GB and all give more than satisfactory performance.  They run along with 4-8 other virtual machines (other 2008 R2 servers, Windows 7, and XP operating systems). More memory is better of course  but you may find the marginnal benefit of 1GB more memory to not show up in the other vm performance .

Now you have determined a proper machine to host Hyper-v with sufficient memory.  It is my very strong suggestion that you now purchase new disks for the OS hosting environment. Why? Because if something goes wrong in the conversion, you don’t want to have clobbered your original system disks to be able to go back to a physical machine.  I know the cost and complexity of new system disks will vary from installation to installation, but disk prices have dropped so much that the cost of even a pair of 500GB (RAID 1 scenario) is under $100.  What is your time for recovery worth:?

The last thing you will need is a disk that can house the .vhd created on the physical installation to be copied to the new os environment for the virtual machine.  An external USB drive is perfect for this, but an internal hard disk that can remain mounted when you create the new hosting environment works well, too.

Cleaning Up the System

Before you convert to a .vhd, do as much cleanup on your SBS system as possible.  Not only will this make the conversion faster, it will make it easier.  And in some cases, it can mean the difference between success and failure.  The conversion utility will convert up to 127GB, so if you have too much data on your system disk, it may not convert at all.

Here are some suggestions:

  1. Move your Exchange data files to another hard disk.  As a rule. I never leave them on the system volume.
  2. Do the same with your SharePoint data files.  Same rule for moving it to another disk.
  3. Move any user profiles to another disk.
  4. If you have any data files that are shared, move them as well and update the share.
  5. Empty the recycle bin.
  6. Delete any temporary files.

In short, anything that is on your system disk that is not essential to the system either delete or move.  Note: if files are small, weigh the inconvenience of moving them against the conversion impact.  A few kilobytes won’t make a difference but a few gigabytes might.

Disk2vhd Utility

You will need this free and downloadable utility to convert your system disk to a .vhd file (click to download it).  It is straightforward to use and can work with a system that is online.  I will share with you some of the tricks I have picked up to make it easier for you.

Disk2vhd works by taking a snapshot of the files to be converted.  What you don’t want to happen is have those files updated while the conversion is running, possibly causing data consistency issues.  Exchange is a case in point, so before you start the conversion, stop Exchange services.  Do the same for SharePoint and any other applications that might be updating files as they run,

If you have been following closely about the preparations, you will have detected my overall approach:  isolate the system to the c: drive, convert it to a .vhd, bring up the new os and create a virtual machine from it, and have all of the data files remain on media that are mounted onto the host operating system.

With that in mind, when you run the conversion utility, you will be just converting the system disk.


In the above example, I have isolated Exchange, SharePoint and user data and even have a separate drive to store the .vhd file.  (Remember, I am a big iSCSI fan and this is a piece of cake to do).

Much  more important, however, is to note that I have selected not only drive C: but the System Reserved area as well.  If you don’t select the System Reserved area, the resulting .vhd will not be recognized as bootable.  When you try to attach it as a virtual drive and start the machine, you will get the following messages:

disk error has occurred
Ctrl+alt+del to continue

But of course there is nothing to continue to.

Choose your destination for the .vhd, then click Create.  It can take thirty minutes to over an hour, so entertain yourself while it runs.

The Hyper-v Host

Now shut down the SBS system, remove the system disk(s), insert the new disk(s) for the host os, and install the os.  If you have a license for Windows 2008 R2, use it by all means.  If you don’t own or want to pay for a license, you can download Windows 2008 Hyper-v for free.  Be aware that it has only a limited interface and that you will need to do enable remote execution of Hyper-v manager on it to be effective.  Still, it is free.

Install the os and launch Hp yer-v manager.  Create a new virtual machine following the wizard.  But instead of creating a new .vhd, instead point to the .vhd you created with the disk2vhd utility.

Before you start the virtual machine, adjust the settings as follows by highlighting that vm and clicking on settings.

  1. Memory – use dynamic setting.
  2. Processor – use the number of virtual processors that match the host cpu.
  3. Network adapter – you can use the same adapter as the host machine or an additional one.
  4. iSCSI Controller – Use this to add disk drives attached to the host system.  For example, if you have USB drives (for backup, e.g.) connect them to the virtual machine.  Same for data drives.  Note – if you are using iSCSI targets, the virtual machine can connect directly to them without adding them as hardware.

Now start the virtual machine.  What you will discover is that it doesn’t start right up like you would think.  Instead, it appears to be “hung” without the desktop appearing.  This could take from five to thirty minutes.  What is happening is that SBS is adding new devices to the system as they have been attached through the virtual interface.  When that is complete, you will get a message that new hardware has been added and you should reboot.

After the reboot, check the network settings to make sure they are correct.  You may have to run fix my network or connect to the internet wizard from the SBS console.  Check to make sure all services are properly started and that Exchange connected to the databases.

Your virtual SBS is now complete and you can add other virtual machines as necessary.

I was pulling my hair out, wondering why the clock on client machines in a SBS 2011 domain were off about 5 minutes fast.  Surely the server time was the issue, and I kept resetting it only to see it go back to its five minutes fast setting again.  So I rolled up my sleeves, dug into all the command options of w332tm and registry settings, and they all looked fine.  My server was set to use

Why didn’t this work?

I finally paid attention to the result of this command:

w32tm /query /status

I slapped my forehead and also did another command

w32tm /query /source to amplify the results I was looking for

Here is what I got

Since I had set the time server to, I was puzzled why that time server did not show up and what the heck was VM IC Time?  Almost instantly it hit me: the hyper-v SBS 2011machine was getting its time from the host, and my settings were not overriding that.

Sure enough, I looked at the host machine (not part of the domain) and it was not using an authoritative time server and was – gasp! – about 5 minutes fast.  A few simple things fixed it all.

On the Hyper-v Host

From either the control panel or time in systray, set the computer clock.  Click on the Internet Time tab and if it is not set to synchronize, click Change Settings…. In the dialogue box that appears, click the Synchronize box and leave the server as or change to another value.  Click update now button, then OK to close.  Click OK again.

On the SBS 2011 Server

Open a command prompt in elevated mode (run as administrator).  Enter this command

w32tm /resync

You should see the clock change to match the value on the hyper-v host machine.

On Client Machines

You will see the time change when the client machines follow the time synchronization schedule, but you can do it immediately if you want.  Use the procedure above for the SBS Server but on your client machines.

That’s it.

There’s Another Way……

This made me think, there’s gotta be another way to deal with this.  Sure enough, a bit of snooping led me to play with the hyper-v host settings, and here is a way to let your SBS 2011 hyper-v machine throw off the shackles of the host time server.

On the hyper-v management console, click on the SBS 2011 server and choose settings.  On the left hand pane under Management, click on Integrated Services and clear the check mark next to Time Synchronization then OK or Apply.  You can actually do this while the hyper-v machine is running.

Once you have done this, log onto the SBS 2011 server, run command prompt and type

w32tm /query /source

You should now see the source changed to or whatever other time server source you have configured.

Either method you choose will work, but I prefer the change in Integration Services to make your SBS server less dependent on its host.

I am currently doing a migration from Windows 2003 (not SBS) to SBS 2011 for a client.  The didn’t want to invest in new hardware since their server was a fairly recent vintage and needed only a bit more memory.  My plan for them is to:

  • Physically locate their server in my office on a new network segment;
  • Set up their router to get a static IP address from my network;
  • Configure parent DNS to point to this new network segment;
  • Run Migration Prep Tool on the old server;
  • (all of the above done
  • Create an answer file for the migration and store it on the desktop of the host OS machine that runs hyper-v (see additional comments later);
  • Install Virtual Clone software on the parent OS of the hyper-v machine;
  • If you have an .iso of the SB 2011 install disk, make sure it is somewhere in the disk space  of the parent OS hyper-v machine and mount it on the virtual drive.  If you have a physical DVD, insert it in the drive on the parent OS machine.
  • Create the virtual machine for SBS 2011.  Make sure you assign a network connection so it can talk to the local network as well as have Internet access;
  • Capture either the virtual or real drive with the SBS 2011 install and start the virtual machine.

It is really a good thing to not have the old server as on the LAN.  If it is that address, change it and be sure to change your port forwarding on the router to match.

When you create the answer file, use for the new server and whatever IP address the old one had. The user administrator will be the system user, and you can add a new one and make the old one inactive.

While the initial part of the install is running, take a made a CD or DVD with the answer file on it.  When the dialogue for migration appears and the SBS 2011 install says it can’t locate the answer file, eject the SBS 2011 install disk, insert the CD with the answer file and try again.  If you used a virtual drive, just capture the physical drive from the Clipboard menu on the Hyper-v connection to the new SBS 20111 server.

Using Virtual Clone was imperative for me in running some things on the old server; it had no DVD drive.  If that were the case on the new server, how would you get the answer file to work?  One way is to create the CD with the answer file on it, then use a program like Magic ISO to convert the CD to an .iso image.  Then you can mount the .iso on a virtual drive  on the parent os and capture if on the hyper-v console.

Once you have the new SBS 2011 running as a virtual machine, then just do all of the migration steps you normally would with two physical machines.