Scenario

I am in the process of migrating a SBS 2011 server to Windows 2012 R2.  It is mostly, but not entirely done, and some essential tasks have been deferred until time permits.  Both of these servers are Hyper-V VM instances.  The host server and both VM servers use iSCSI targets for a number of key disks.  The virtual machines and disks reside on such a volume.

In spite of a dedicated UPS for the host server and the iSCSI device, they both power recycled for some reason late last week.  I always takes the iSCSI much longer to reboot than the host server, and I expect a few minutes of delay until the VMs start.  However, when I checked later, the 2012 R2 server was not restarted but reporting a failure and asking to do a repair.  A few times trying that made no difference.

How I Fixed This

I selected the tools option on the failure start screen and tried starting in safe mode.  No luck, it still failed.  I also tried low video resolution, same problem.  Then to my delight selecting  directory services restore mode allowed a successful boot.  That made me realize that the NTDS database was probably corrupted.  NOTE:  you will have to logon with a local administrator account.  AD does not start and none of the credentials in it are available.

The first thing I tried was to navigate to the database folder, C:\Windows\NTDS.  I copied the folder contents to C:\Windows\NTDS\Save after creating that folder, then from an elevated command prompt, ran ntdsutil and then the following commands

  • files <enter>
  • info <enter>  This will list the files for the database and logs
  • compact to <full path>  You probably want to create a new folder and provide path to it.
  • quit twice to return to the command prompt

Ideally, you will have a new and well formed NTDS.DIT file in that directory, and you should copy it to C:\Windows\NTDS and overwrite the corrupted file.  Don’t worry about losing anything since you have a copy saved.

Now reboot your computer and it should start normally.

WELL MINE DID NOT!

I was so focused on getting my server back that I can only vaguely recall that the compact command did not work, saying there were log files that had not been applied.  Well, it thought that is what compact would do.  Or maybe it did and the server still did not restart properly.

In any case, I switched to using Esenttutl instead of ntdsutil.

Run an elevated command prompt and type

  • esentutl /g c:\windows\ntds\ntds.dit
  • esentutl /r c:\windows\ntds\ntds.dit

The first is an integrity check, and mine predictably failed.  The second is a recovery command, and that, too, failed with a JET database engine error. So I ran the repair option, /P, instead of /R on the command line.  Voila!  It completed successfully and I reboot to a normal windows server.

So What Was That All About?

In general, Windows databases do not update directly but write transaction log files.  Later, these log files are “played back” and make the actual transactions update the database itself. When an unexpected shutdown occurs, as in my case, it is possible that the database does not close properly and has a corrupted element somewhere in it.

Esentutl is also used for Exchange databases if they become corrupted, and it has saved me many times with SBS errors.  While I was hoping the /R recovery function would work, I was not particularly worried about the /P repair option, and it did work.

You might ask yourself, why didn’t I just restore the directory from the last backup?  Remember those tasks not yet done?  Er, server backup was the next item on the to-do list.  Happy to say it has now been done.

Advertisements

Scenario

The client machine on which this happened was Windows 10 with Office 2016 (Pro Plus from Office 365) installed.  It started happening suddenly according to the user.  When Word, for example, would launch, an error text displayed that work files couldn’t be opened.  Additionally, if Word, etc. were trying to open existing files, they never loaded and an error was displayed that the file could not be opened.

Red Herring

I first thought it might be a permissions error, so I looked at permissions on folders and saw that the user not only had them but was also a local administrator.  To trouble shoot some more, I tried to give another domain user privilege, but THAT gave me a RPC error when I tried to add one of the enumerated users.  So I concentrated on domain membership.  For a while.  In fact, I was about to remove the client from the domain and re-add it, but thankfully did not.  It would not have solved the problem.

Next I decided to look at environment variables.  In particular, TEMP and TMP.  Both were correct with %USERPROFILE%\AppData\Local\Temp.  I reset them anyway but no luck.  Same errors.

What Did Work

The problem was registry settings.  I launched REEGEDIT and navigated to

HKEY_CURRENT_USER\Software\Microsoft\Windows\Current Version\Explorer\User Shell Folders

Look at Cache and Cookies.  On the client machine, these had values that began with C:\Users\<username>|Appdata\Local\Microsoft\…  I changed them to %USERPROFILE%\AppData\… and closed Regedit.

That made everything work.

I checked my own computer, which has a similar configuration, and the settings that did not work on the client machine worked fine on my own.  I am not sure why this is the case.  But never look success as your enemy, more like a virtuous being you will never understand,


Introduction

Guess I am like the cobbler’s kids who didn’t get their shoes when everyone else did.  I am the last of my clients, save one, to migrate off SBS and onto Windows 2012 R2 Standard.  I thought you could benefit from some of the issues I ran into, and solved.

Here’s my scenario.  Have a Hyper-V server running SBS 2011 as a virtual machine.  Created a new virtual machine and installed Windows 2012 R2, did updates.  Unfortunately, it sat for several months while I finally got a few days to do he migration.  More about that later.

Long ago, I migrated email and SharePoint to Office 365, so I had disabled services and IIS application pools on SBS.  My starting point was to fire them up again in order to remove them.  I did not want AD to migrate with all of those extra objects.

Removing Exchange Server 2008 R2

This did not start off well.  When I launched EMC, it failed to connect to the SBS server.  I ended up putting it aside for a few days but came back to it.  I had tried to guess or remember which services should be started, but I seemed to have failed.  I also only started the Exchange-related application pools.  I easily tracked down an article describing which services start automatically, fired them up, and enabled those that should start manually.  BTW it describes all of SBS services.

That did the trick, and EMC successfully opened.  But I knew that in order to uninstall Exchange, I had to remove the mailboxes.  Fortunately, I had a small number of them.  And to make it easy, I used Exchange PowerShell commands to do this.

Get-Mailbox | Disable-Mailbox
Get-Mailbox -Archive | Disable-Mailbox -Archive
Get-Mailbox -Arbitration | Disable-Mailbox -Arbitration

First, please note that my scenario had with SBS – a single mailbox database and server.  That is why there are no qualifying parameters on the commands.

If your first thought is to use EMC to remove mailboxes, CAUTION!  That method removes both the mailbox AND the user from AD.  If you do want to remove some users and their mailboxes, do that but otherwise use Disable-Mailbox.  There is a Remove-Mailbox command but it also removes both user and mailbox.

So what the first command does is get a list of mailboxes which are piped to the next command.  The second does the same thing but gets the archive mailboxes.  You will then not be surprised the third command gets the arbitration ones.

I tried just getting archive and arbitration mailboxes, but the id names were too long to display in their entirety, so piping was essential.  And easier. And faster.

I then tried to uninstall Exchange but got two failures.  The first block came from Trend Micro Worry Free Advanced that it was using the database and the second was the Offline Address Book in Public Folders.  I uninstalled the messaging agent for Trend Micro but getting rid of the OAB was harder.

First I tried to simply delete the public database but it was not empty (I knew that from the OAB warnings).  I then tried to create a new, empty one to mount but then Exchange would not let me create a second one.  So under Tools, I chose Public Folders and expanded the tree and selected OAB then the firs entry and deleted that.  It was the only entry I could delete.

Luckily, I was then able to uninstall Exchange.

Removing SharePoint

Could not have been easier.  Just uninstall from Control Panel.

Back to Windows 2012 R2

I had already joined the domain and installed Active Directory Services, so I was ready to promote it to a domain controller by starting the configuration on that role.  Just added to the existing domain and it just worked.

I needed a few more roles on this new server, and then trouble started.  I tried to add both Remote Access and Windows Update at the same time and the installation failed.  Separately they also failed.  Again and again.

I cheeked for updates and found plenty and installed and rebooted.  Still no luck in adding roles or features.  I finally found this article which pointed me to a fix.  Note that I modified both policies that it refers to.  After the gpupdate, installation of roles worked fine.

I had not run into this issue before with 2012 R2, so I think it is related to both the GPO settings from SBS 2011 and that AD is at 2008 R2 levels and cannot be promoted until SBS is removed from the domain.

But this puts me well on my way to being where I want to be.

Now I just have to move Worry Free Advanced and get client machines set up under Windows Essentials role.


Trying to Launch Either Program Failed

I had upgraded a desktop from Windows 8.1 to Windows 10.  I was more than pleased at the results.  I liked Windows 10 features a lot better, and it seemed to perform better on my existing hardware than its predecessors.

A few weeks ago, however, neither Quicken 2016 nor Quick books 2016 would properly open.  The splash screen would appear and the application would start, then fail with a can’t continue message and terminate.  Very frustrating to say the least.

What Didn’t Work to Fix It

I tried a repair on QuickBooks.  I tried uninstalling and re-installing each program.  Same thing.

I looked at the event logs and discovered an event that referenced wpfgcs_v0400.dll.  I convinced myself that was the culprit.  Turns out I was right.

What Did Work

I was lucky enough to have a virtual machine running a new install of Windows 10.  I located the .dll file at C:\Windows\Microsoft.NET\Framework\v4.0.30319\WPF and notice the date was about 6 months newer, so I copied that file and went to the corresponding directory on the failing machine to paste it in.

Not so fast.  I didn’t have permissions on that directory.  I had to take ownership of the directory, disable inheritance, keeping the inherited security, and then add myself with full control.  That allowed me to overwrite the .dll with the newer one.

As if by magic, both Intuit products started working again.

If You Run into this Problem…

You may not have access to an updated .dll.  Contact me if you need it and I will send you a copy.

 


 Virtual Machines Have Prevented Disasters More than Once

Let me tell you about the latest situation where I look like a hero instead of a fool.  How can you not love someone who does that for you?

I have a client who is still on SBS 2011 for a variety of reasons.  Let’s sum them up by saying it is unlikely to change for a while.

A few years back, I suggested making SBS a virtual machine and running on a Hyper-v host.  The OS for the host was Windows Server 2008 R2.  Earlier this year, both of the USB drives they were using for Windows Backup (one for the host, one for SBS) had to be replaced.  The new drives were formatted NTFS with 4096 byte sectors.  If you don’t know already, Windows Backup in 2008 R2 and earlier can’t create a backup on these targets.  So for a while, the system has been running without backups.

Why not Just Upgrade?

Duh, why didn’t I think of that?  I did, but an inplace upgrade to 2012 R2 kept failing.  Finally, we located a 2012 Standard .iso and license for sale and grabbed that.  The in place upgrade went well, very well.  Much to our chagrin, the product key would not activate even though it was valid, and the client got his money back.  But there we were with an OS that was about to die from lack of activation.

So I tried the upgrade from 2012 to 2012 R2, and it succeeded.  Problem solved, right?

You Probably Picked the Wrong Answer

When it finally rebooted after the upgrade, a colorful blue screen appeared with the message

MUI_NO_VALID_LANGUAGE

The scarce number of articles I found relating to this indicated that it was an invalid product key (I knew that) and re-installing would fix it.  So in goes the DVD and we boot from it.  Enter the correct 2012 R2 product key, select upgrade, and we are instructed to remove the disk and reboot.  When we do the same nasty error above appears.

So I Said…

Let’s just forget about the host system and create a new one.  I didn’t format any disks, just let the installation go.  It completed fine, booted into 2012 R2, and I added the Hyper-v role, reset the static IP address, and got down to setting up a new virtual machine.

I pointed to the existing .vhd for SBS and the virtual disks for Exchange, SharePoint, and data that were on physical drives on the host and attached them through ISCSI controllers, started the virtual machine and bingo!  There is was.  Almost.

I Nearly Cried when It First Started

Active directory showed NO users, NO joined computers, and SBS Console said the OS was not operative.  While I played around for a bit, then looked again after about 5-10 minutes, AD values were back!!  But there was no Internet connection and nothing looked right on the SBS Console network tab.

First I ran Connect to the Internet, and that restored the connection.  Then I ran Fix my network, and suddenly there was the trusted certificate and all the other goodies.  It has been running like a champ ever since.

One Last Thing

The 4096 sector drive for SBS still doesn’t work, so I am trying a WD drive that emulates 512 sectors,  I am going to create another post about this, although there is a lot of information on this topic out there already.

So, to sum it up, your Hyper-v host machine is disposable.  You can trash it and provided you don’t lose the date, you can reconstruct your workhorse servers.  I highly recommend this approach.

Even better, use an iSCSI device (like QNAP, which I love as well) and keep all your virtual information on the drives apart from your computer.  That means the entire platform is disposable.

Is this a remake of an old movie?

Posted: December 12, 2015 in SBS 2011

Source: Is this a remake of an old movie?


OneDrive for Business and/or SharePoint Libraries Synced

If you had your OneDrive for Business library and/or other SharePoint libraries synced and then you upgraded from Office 2010/2013 to Office 2016, there is an excellent chance that syncing will stop.  It may be coincidence that this showed up along with an upgrade to Windows 10, but the primary issues seem to be with Office 2013.

Try This First

Open task manager and see if OneDrive for Business is running, or look in systray for the blue clooud icon indicating it is trying to sync.  If it is not running, go to the start menu (Windows 7 or 10) or look in Applications (Windows 8/8.1) and open OneDrive for Business.  You won’t see anything unless you look in systray again.

If it is running, either because you started it or it was already running, right-click on the systray icon and see if it brings up the context menu.  Hopefully it will, and you should choose repair. Let it run to see if that fixes your problem by then seeing if things start to sync.

I was not so lucky.  I got no context menu when I right-clicked.  More than that, I should 13,000+ files syncing (size of all my synced libraries) and it never changed.

What Worked for Me

I tried several things, none of which actually did any good towards fixing the sync problem, but for your information, and in hopes maybe they would work for you, here they are:\

  • Started OneDrive for Business, restarted the computer and tried again.  Nope.
  • Ran quick repair on Office (installed from Office 365 BTW).  Nope
  • Ran full repair on Office.  Nope.
  • Opened Office 365 and went to each library and tried sync icon.  Nope.

If you too got none of those to work what you should do next is uninstall Office 2016.  Don’t panic about settings like Outlook profiles, signatures, etc.  They won’t disappear on you.

For the next step, rename the synced folders on my local drive.  In my case, the local copies were all on Drive C:, so navigated toC:\Users\<myprofile>.  If you synced ShareSharePoint  libraries you will see a folder Named SharePoint.  DO NOT DELETE THESE FOLDERS OR FILES!!  Rename it to have an extension of .old (you can do anything you want to get rid of the original name, but I would just add some sort of extension and will explain why later).  Doing this AFTER removing Office 2016 doesn’t give you an error that the files are in use.

Now rename the OneDrive for Business folder.  It, too, in in your user profile, with a file name of One Drive – <domain> where domain is your Office 365 user domain.  Add .old to it as well, for example.

Now re-install Office 2016.  If your source is Office 365, open the portal, click on the gear wheel, and choose Office 365 Settings, then either Software (if your site has not been recently upgraded) or Installs (if it has).  Click to download and install Office 2016.  If you didn’t change your computer name, then it already knows you are authenticated on that computer.

When the install is finished, start OneDrive for Business.  You may get error messages that it can’t find the libraries to sync.  If you do, then right click on the systray icon and choose Stop Syncing a Folder, then select the folder(s) and stop synching all of them.

Now, start with the SharePoint libraries you want to sync.  Open the Office 365 portal, navigate to the sites, then to each library and choose sync.  When all have finished, you can now turn your attention to OneDrive.

Here is what I had to do to avoid getting an error on sync that the file could not be opened.

  1. Stop OneDrive for Business.  You can do this in task manager, or right-click on the icon in systray and choose Exit.
  2. Open the Office 365 portal and navigate to your OneDrive for Business.
  3. Click on the sync icon, then allow it to sync.

It took a bit for my files to come across as I had a large amount in OneDrive for Business, but it finally caught up.

But Then…

I started seeing that horrible red circle with the white X appear on the folder and file names in the local folder.  I thought that very strange since all the files were synced before and there should have been no errors.  So I looked at sync errors – right-click on the icon in systray and choose that option – and saw that file after file was asking for credentials.

This is what Microsoft support describes as a known issue.  There is an update to fix it, and if you open an Office 2016 product, click on File then select Account, you will see an update button.  Click to apply the update, then reboot your computer.

While you are there, however, make sure that your Office 2016 is connected to

  • Office 365 SharePoint
  • Office 365 OneDrive for Business
  • OneDrive if you have a personal account that you also use.

I also had to do a bit of tweaking to finally get sync going again for OneDrive for Business.  I stopped the sync, did a repair, then started it again.  It did not seem to get things moving.  So I removed OneDrive for Business from the sync sites, went back to Office 365 and synced it again.  Finally, when it started reporting “need credentials,” I went off to do something else.  When I came back several hours later everything had synced and no errors were reported.

Go figure.

Files Updated Locally but Never Synced to Office 365

You may have a  situation similar to mine in that I had opened and updated files locally as well as created new ones, but they never got synced to the cloud.  So the final step is to “sync locally” with those changes.  That is why it was important to keep the old local copies because those copies hold the updated files and folders.

To “sync” them, I used xcopy.exe from a command prompt.  Add the parameters “/e /s /i /d” to copy only newer files (/d with no additional parameters) and add any missing files and folders.  Assuming that your files are synced on Drive C:, the default location, try these steps:

  1.   Open cmd prompt window.
  2.   Navigate to C:\users\<profilename>.
  3.   Look for the folder SharePoint and the one you renamed to SharePoint.old.
  4.   Enter the command xcopy sharepoint sharepoint.old /e /s /d /i
  5.   This should copy the files that are newer to the local syced folders and then in turn up to Office 365.
  6.   Repeat the command for OneDrive for Business new folder and the old, renamed one.  Be sure and use ” around the directory name, as in xcopy “onedrive for business.old” “onedrive for business” /e /s /d /i.  Replace the directory names in this example with the actual directory names on your drive.

When everything has synced to Office 365 and you are comfortable with the files that are local and in the cloud, you can safely delete the old, renamed folders from your drive.