Categories
Exchange 2010 Office365 Uncategorized

Interesting migration issue with Office 365

We’re currently running a migration to Office 365 from an internal Exchange environment.  Rather than use directory synchronisation or go to a Single Sign On environment (SSO), this is a 1 time migration to the new O365 world.
The idea was that instead of “polluting” the new world with rubbish legacy information/setup created over probably 15 years operation since Exchange 2000 days (through 2003, 2007 and 2010); as well as have an impact on the internal setup/migrating by configuring a hybrid setup we would go for a  new setup that we knew would be fresh.
Fortunately (or possibly not, depending on your point of view!) we had an email domain fully registered and configured, but not used in real life (it was just to protect against misuse by someone else). We decided to add that to the O365 world and get a fully working setup, and then we could just add the other \”real-life\” domains to it to complete the migration.
To set up the O365 trial we did a number of things:
  • Removed the SMTP domain from within Exchange setup (and from the message hygiene service we used)
  • Added it to the O365 trial account
  • Created our mailboxes in O365 and had everything working OK.
  • By now we had removed the aliases in place on the internal exchange system user mailboxes (note: user mailboxes!) for the new domain.  Everything was going swimmingly.
It was at this point 3 further things needed to be tested on O365 before “signing off” that all was good for the 1-time transfer over and migration of data.
  • Mail Enabled Public Folders for List Server subscriptions
  • Use of O365 for automatically mailing of messages from devices and services
  • Use of O365 Shared Mailboxes for accounts such as Admin, Webmaster, Postmaster and so on to cover off public required (or sensible) aliases without using an O365 licence.
The first two were sorted quite quickly using publicly available information (although watch out for mail enabled Public Folders needing anonymous “Create” rights to enable mail to be delivered).
But the last was proving a right PITA.  I had created a Shared Mailbox for the webmaster and whenever I added it to my O365 mailbox my outlook account would constantly prompt for my password to connect.
Eventually (after deleting and recreating profiles, deleting the contents of the Outlook folder (OST and so on) in my home directory, and deleted windows credentials through the control panel); I put in a support call.
Before getting the call and whilst away at a Microsoft IT Camp, I recreated the shared mailbox and found that on my Surface the profile worked fine.  So it was machine specific…
Raj called and we tried a number of things (no caching, new profile, delete the profile from the registry) and so on.  Then we spotted something.  In the depths of the profile registry entries I spotted a reference not (as in the image below) to the ExchangeLabs O365 folders, but to my internal exchange server

Raj agreed this was something to work on, and he came up very quickly with a fix.  The creation of a registry key in AutoDiscover to prevent SCP lookup on the network.  This would prevent the outlook client from searching through our network for a resolution of the webmaster shared mailbox identity and location.
We deleted and recreated the profile, logged in and it all worked.  Hurrah.
As a final test I sent an email from my phone (connected over 3G, not the internal network) and found that my email did not hit the shared mailbox, but when Raj sent a test email, it worked.  Hmm….  This was a further clue as to why SCP lookup was causing the problem.
A quick look at the Exchange Console (I say quick, but given I had Microsoft support on the line it took a bloody age to load!); and I discovered that the webmaster@ alias was still registered on the administrator mailbox.  So, what had been happening is that the SCP lookup in the profile creation when I started outlook was finding the alias in my network, and thereby causing a conflict with the O365 provisioned shared mailbox with the same email alias.
Remediation was then straightforward:
  • delete the now unused smtp alias (and a few others I found) for the domain now on O365 from recipients on the internal network (there were a few admin like accounts that still had them)
  • regenerate the Offline Address Book
  • allow the OAB to distribute
  • remove the ExcludeSCPLookup key from AutoDiscover
  • test again
And all was well.
Lessons?
  1. Don’t assume a completely standalone move, even with unused domains, to O365 will go smoothly 🙂 – just because it’s completely separate
  2. If you have unused domain names on your exchange infrastructure then do check carefully whether aliases have been created for them
  3. If you have (for legitimate reasons like us, or not) disabled automatic address generation on mailboxes, then you really need to take some care over legacy SMTP aliases on mailboxes  (the PowerShell code listing later on will help you find them – run on an exchange server and look at the output file – the location of which is coded at the end of the PowerShell)
  4. Microsoft O365 tech support are good.  It was nice to prove that before we committed over to the service.
Get-Mailbox -ResultSize Unlimited |Select-Object DisplayName,ServerName,PrimarySmtpAddress, @{Name=“EmailAddresses”;Expression={$_.EmailAddresses |Where-Object {$_.PrefixString -ceq “smtp”} | ForEach-Object {$_.SmtpAddress}}} | Export-CSV c:\\smtp.csv -NoTypeInformation

Categories
Exchange 2010 Microsoft Uncategorized

The joys of Exchange 2010 Service Pack 3 installation

Having just completed this process on a few of several servers.  Some news from the frontline…

  1. If you have Backup Exec in the mix, then stop the service before you start installing SP3 on any of your boxes.
  2. You may want (or need) to do the schema update to Active Directory before you start.  Either way, it’d be a good idea to ensure the upgrade is done and replicated before proceeding too far down the install
  3. You may get a warning about obtaining a Hotfix from  support.microsoft.com/kb/2550886 during preparation on servers with a MBX role – grab it now, and have a look.  But it only applies to improved Failover Cluster performance in stretched DAG’s across datacentres.  So you may well not need it.
  1. You may wish to stop your exchange services before running setup.exe, I have seen it fail to wait long enough to stop services properly during the upgrade.  The SP3 install actually disables them before stopping (if you run services.msc you can see it happening).
  2. Lastly, when the setup is close to finishing and attacking the server roles, make sure Microsoft Exchange Active Directory Topology service is running, if not then the AD communication will fail and you will have a big red cross on your screen, that may have an impact on your cardiac health!  This can be monitored for and the services started by hand when the installation moves onto “Restoring services”.  So far I have seen this issue on MBX servers.  I decided to do it for all servers irrespective as the upgrade rolled out
Web pages that may help you:
So, onto further servers!
PS1: Oh, yes, you really mustn’t ignore a good backup as a really good idea – especially for those cardiac moments!
PS2: version changes in the management console (at the site on which I’m working) moved from 14.2 (Build 247.5) to 14.3 (Build 123.4)

Categories
Exchange Exchange 2010 Outlook 2010 Uncategorized

Public Folder replication from Exchange 2007 to 2010 failing, it\’s not a pretty fix

If, like me, you have an infrastructure that has a considerable history, then it will have been Exchange 2003 in the past.  The work detailed below was a late night oepration and my documentation skills were not up to the usual levels.  But hopefully you can get enough from this to fix your setup (now you\’ve found this article!)

I\’ve previously blogged about the migration to 2007, and the fun in preparing for 2010, but having made the transition I found that the Public Folder hierarchy would not replicate, let alone the replicate the content.  I was getting errors 1020 on the MS Exchange Store Driver.  In addition on trying to add the PF hierarchy to a new Public Folder Mailbox using AddReplicaToRecursive.PS1 I was getting a lot of errors.

After some investigations and work, I have an answer – your mileage might vary, but:

Firstly, I am indebted to the following pages for providing insight and information on what to look at and try!
Public Folder Mayhem (pointer to an ADSIEDIT required)
The inevitable TechNet article (explanation of that ADSIEDIT – and note NO PLANNED FIX)
and zerohoursleep (another pointer to the same problem)

The first clue from the Public Folder Mayhem was the empty Exchange 2003 legacy object in Active Directory.  The object is
CN=Servers,CN=First Administrative Group,CN=Administrative Groups,CN=,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=NetBIOS name>,DC=TLD>
this object (which had permissions like this – scuse the obfuscation of some details):



That object was then very carefully documented – permissions, ACL details, ownership, attributes using various PowerShell commands (including Quest\’s AD addon Get-QADObjectSecurity) and LDP.  These were all filed away.
Some people in that article considered deleting the entire First Administrative Group.  I chose not to, as there are three other leaves in there (Advanced Security, Folder Hierarchies, and Routing Groups).  I saw no point in deleting more than might be necessary, even if those leaves are (or are going to be) unnecessary.  Note that the TechNet article supports this approach.  If you want to delete it – use the interface for Exchange not ADSIEDIT.

So I deleted the object from ADSIEDIT, and replicated around the network (checking that all was good).  This all worked fine.

Now to get the PF replication under way.

Previously I had tried to use .\\AddReplicaToPFRecursive.PS1 (with parameters -TopPublicFolder \\ -ServerToAdd MBX server NetBIOS name> as supplied by the Exchange Team to speed up the process. This still failed.

Over to the new PF hosting MBX server and the following quickly got things in place:
Get-MailboxServer -identity MBX server NetBIOS name> | update-publicfolderhierarchy 
this gets the heirarchy into the new server

Even now adding the replicas
.\\AddReplicaToPFRecursive.PS1 -TopPublicFolder \\ -ServerToAdd MBX server NetBIOS name>
failed!

Bryant\’s fail safe to restart all exchange services on both MBX servers was then applied.
It then worked.

So I now had a PF hierarchy and content on the Exchange 2010 server.

A few days later I was investigating something else, and noticed that at a Microsoft Outlook client having opened Exchange Connection Status (right click the Outlook icon on the System Tray, Connection Status) that the PF connection was to the old MBX box (image below shows results after the fix, but you get the picture).


One last trick is required though:  you need to associate the new PF database to the new Mailbox data.  So go to your Exchange Admin tools, right click properties on the 2010 Mailbox database for users.  Client Settings tab
Browse to the new PF store and select it thus:

Now, finally I have Public Folders replicated to Exchange 2010, and users connecting to the new PF location.

just some tidying up, and then i can start to decommission the 2007 world!

Categories
Exchange 2010 Microsoft Outlook 2003 Uncategorized

Just a reminder, if you are connecting #Outlook 2003 to #Exchange 2010

Then to get a connection you are going to have to turn this:
Into this:
Otherwise the connection and the name resolution on creating the profile, WILL fail.
Categories
Exchange Exchange 2010 Outlook 2010 Uncategorized

Think you are clever #Outlook2010 ? No. (Take 2) #fail

You may have seen my annoyance earlier with Outlook 2010 and it\’s \’helpful\’ \’this is not the latest email\’ message here: http://corylus.blogspot.com/2010/06/think-you-are-clever-outlook2010-no.html
 
It\’s just got worse.
 
I received an email recently from someone i was trying to find and contact for my mother.  They replied yesterday, and I forwarded the email immediately to mum.  So today I replied to the person to let him know that I’d done so, and that she’d be in touch.  Guess what?
 
This is not helpful!  I don’t think it should squawk at me when I am replying to the latest message in the conversatino between me and the recipient.
 
 
 

Categories
Exchange Exchange 2010 Outlook 2010 Uncategorized

Think you are clever #Outlook2010 ? No #fail

I was answering an email, and got a strange error message (see the first marked area in the image)
 
So I clicked on the message and it brought up an entirely unrelated message, but the key was the subject (also marked).  It seems Outlook looks for all messages with the same subject IRRESPECTIVE of addressees.
 
This attaches to my concerns about Exchange 2010 expressed a while back – http://corylus.blogspot.com/2010/04/why-is-exchange-2010-problem-upgrade.html  It seems to reinforce my concerns that the linked messages are connected by a crude text comparison of the subject, rather than a more sophisticated tracking system.
 
 

Categories
Bad Decision Exchange Exchange 2010 Uncategorized

So by now I am ready to do Exchange 2010

Firstly, I cannot commend too highly the script that I found at Automated prerequisite installation Exchange Server 2010

This script, one you have a fully patched Windows Server 2008 R2 installation gives you a quick setup to add the necessary pre-requisites on the server to start the Exchange 2010.  If you already have the Update Rollup 3 for Exchange 2010 and the 64bit filter pack then copy these to c:\\innervation on your server so that the downloads are not done again.

Run the script and you can quickly install everything you need before you hit the Exchange 2010 DVD.
If you read my blog then you will have already seen in my previous post that I have lost loads of time already on this project so was running out of time to do a clean Exchange 2010 and get back to ‘real’ work.
So having done the pre-reqs the install got going and seemed to run smoothly.  Immediately after install I hit the server with Update Rollup 3, but however on reboot of the new server, despite everything starting fine in the services arena, the console would not find the other servers in my network, although it could see various settings and my mailboxes.  Hmm.

I ran some quick diagnostics and got nowhere.  For now I just wanted to bail out.  So I decided to uninstall to preserve my sanity and start again another day.

Uninstall is interesting though – read this – Error when uninstalling Exchange 2010 you need to uninstall the Rollup first.

So this weekend, I am retiring hurt 🙂
Categories
Bad Decision Exchange Exchange 2007 Exchange 2010 Uncategorized

The delights of Exchange Server 2007 SP2 installation

Updated 1/6 13:40 to clarify a point made by email
I thought I’d document (albeit in not too much detail) the delights I discovered this weekend.


Being self-employed my network is a bit of test and dev environment – at times I am bleeding edge running beta code, at others a bit behind the curve.  The latter is true on my production Exchange infrastructure – but as email is so critical – that was OK.

But by now, although Exchange 2007 is working and on the “if it ain’t broke, don’t fix it theory” I had left well alone the upgrade to Exchange 2010 was overdue, and I thought I’d make some progress on it.  But…

First off, Exchange 2010 requires that you upgrade your environment to Exchange 2007 SP2 as a minimum for 2010 to install.  I’d not done that largely as to find a period of downtime where I will not impact my business email or my wife’s business’s co-located email setup has been tricky for some months due to the long term illnesses and now recent deaths of our fathers.  But now, having downloaded (again – just to be sure) the modest (!) 800MB+ service pack file I set to the task (along with the Update Rollup 4 that is the latest for Exchange 2007).

The installation went well, and the SP2 install also applies all the schema changes required for your network to support Exchange 2010 as well, so that would save on that server install.  Good.  I started with my HUBCAS server (I have a separate MBX server).

However as soon as I rebooted the server after the update things went a tad wrong.  Several services refused to start – Microsoft Exchange File Distribution, Service Host, Transport and Transport Log Search.  Active Directory Topology was experience lots of errors, and the whole thing stank.  It began to feel like my last problem (Exchange 2007 slow start up fixed); so I quickly checked out the servers and confirmed that no spurious IP addresses existed in DNS or on the Domain Controllers.

So, some further investigation was required.  As is my usual practice I archived and cleared the event logs, and rebooted the server.  This gives me a clean eventlog to check through, and also from a clean boot so I know that the event list I am looking at is directly the fault of the issue, and not clouded by other stuff (FWIW – for years I have also fully rebooted servers [if possible] before upgrades and archive/clear event logs then, so that a) I know the machine boots OK, b) the event logs don’t overflow or show me old rubbish).

So the errors were still piling up, the services were still not starting.  I was seeing events 2114 2604 1014 from the AD Topology, and found these 2 links:
http://support.microsoft.com/kb/944752/en-us
http://social.technet.microsoft.com/forums/en-US/exchangesvrdeploy/thread/56df0d06-2a86-419a-bf5c-42bd2d105892/
These pages report that post Update Rollup 5 on Exchange 2007 the performing of the certificate revocation checks that the managed code performs during service boot can time out and cause services to fail to start.  That looked good as I do have some connectivity issues being over 6km from the BT exchange for my broadband, so went down that route.  I changed the HOSTS files as suggested for both the IPv4 and IPv6 address (127.0.0.1 and ::1); but that had no impact.

So then I reverted the HOSTS changes and instead assigned specific IPv6 addresses to the servers and tried that – no good – again.

By this time (given that I clear logs and reboot between each test) I was starting to get more than a bit annoyed with all the time I was losing.  I headed over to eventid.net instead to research some of the event id’s. Eventually… after searching for ages over various event id’s I came across this page:  Notes on 2114 error in MSExchangeDSAccess .  Joe Richard’s comment was something that intrigued me, so I hopped over to a DC and discovered that the my site’s IP range (and assignment to the site) in Active Directory Sites and Services was missing.  SHOCK HORROR. *Updated – however although all was working well recently, I\’ve no idea (for now) when this disappeared; I\’d really like to blame the SP2 AD upgrade :-).  If i find out, I\’ll add more

I quickly fixed this (and replicated around the network), but disappointingly the services did not play ball.  I restarted DNS everywhere, but no dice.  Ever the optimist (!), I took the decision to reboot the DC’s and then the exchange boxes to force everything through (I could not be bothered to stop/start services until it played.

Hallelujah.  The services all came up cleanly

So some 8 or 9 hours later I was finally on track to install an Exchange 2010 server…