Exchange · Office 365

Exchange Hybrid MRS vs. MigrationService Migrations

I would like to share some Exchange 2010 hybrid migration facts with you that we figured out.

First, again many thanks to Michael Van Horenbeeck! He helped me discuss this with a customer. I’m always very happy to work with him. And many thanks to Ben Winzenz and Jeff Kizner as well, I’m very grateful for your help.

In short: a customer is trying to keep about 65k mailboxes in sync to ensure a short cutover time. We are using a maximum of 1,500 mailboxes per batch, 5 batches per week, and switching 7,500 mailboxes with an overall data of about 5TB per week. For some technical details, we are using Azure ER (800 Mbit) for migration with 4 TMG as a proxy and some kind of F5 load balancing in between, PAW is activated, and two migration endpoints with each 100 sync/complete in parallel. We did some networking measuring and move request statistics and we had an average migration velocity of 18.6GB/h for batches starting the first incremental sync (0% to 95%) which is great. Of course, the migration velocity depends on the number of batches, mailboxes, mailbox items, network workload, etc.

We suspected if we keep a large number of mailboxes in sync might affect throttling or the overall migration velocity.

Another important part is how long MRS will sustain the synced mailboxes. The documentation figures if a migration batch has no administrator interaction (nobody touching it) for 90 days, it will be automatically stopped. 30 days later, again if no administrator interaction, it would then be deleted. We have had some batches longer in a synced state for testing purposes, but we don’t want to fall into this limit in our production “migration” environment.

However, our problem was the delay in between when the request gets marked as complete, and when the batch status on the portal or Get-MigrationBatch gets updated and shows complete. We are using the status of the migration batch because there is some automated logic behind the migration for different pre and post hybrid migration tasks.

An example: a batch was started (CompleteAfter) 01/29 00:01 AM and showed completed in the portal and with Get-MigrationBatch at 01/29 13:00 PM. Nearly at the same time the email report from Exchange Online was sent as well.

I did some research with Ben’s help with the following cmdlets:

$moves = Get-MoveRequest -batchname “MigrationService:20180130T000100_MB_X-X” | Get-MoveRequestStatistics

$moves | select identity, completeafter, finalsynctimestamp, completiontimestamp

Interesting to see that the CompletionTimestamp flag for every move request (1,863 mailboxes in the batch) was finished only about 2 hours later.

Identity CompleteAfter        FinalSyncTimestamp   CompletionTimestamp

——– ————-        ——————   ——————-

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:15 AM 1/29/2018 2:22:08 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:35 AM 1/29/2018 2:23:23 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:41 AM 1/29/2018 2:22:40 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:51 AM 1/29/2018 2:23:03 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 3:13:16 AM 1/29/2018 3:14:24 AM

Another approach from Ben was to check the activity on the migration endpoints:

Get-MigrationEndpoint | fl *active*

ActiveMigrationCount       : 0

ActiveIncrementalSyncCount : 0

ActiveMigrationCount       : 0

ActiveIncrementalSyncCount : 0

20180212T000100_MB_X-X                     Syncing   ExchangeRemoteMove 96

20180221T000100_MB_X-X                     Synced   ExchangeRemoteMove 98

20181231T000100_MB_X-X                    Syncing   ExchangeRemoteMove 1322

20180215T210000_MB_X-X                     Synced   ExchangeRemoteMove 5

20180213T210000_MB_X-X                     Synced   ExchangeRemoteMove 5

20180214T210000_MB_X-X                     Syncing   ExchangeRemoteMove 14

20180221T000100_MB_X-X                     Synced   ExchangeRemoteMove 101

20180207T114420_MB_X-X                     Completed ExchangeRemoteMove 1

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 98

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 95

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 94

20180208T093847_MB_X-X                     Syncing   ExchangeRemoteMove 1

But it seems this is not accurate enough.

If I remember correctly, FastTrack is using move requests instead of batch migrations (or migration service) as well. In my customers case we have to use the migration batch and therefore the migration service as our migration plan. But this is only based on the organizational migration strategy from the customer, which means add, delete, change switch date and time, etc. must be done a lot for some batches. And every batch contains a single business unit so that all of them will be switched at the same date and time. Other features like health checking, reports, etc. are also very important for us.

At least Jeff gave us some interesting information: While MRS is actively processing moves and updating its status in real time with moving the data, migration service processes its queue serially. So as long as it has batches and migration users in its queue, it has to work its way through them all. Spending a slice of time on each one. Following on that, the status of the migration batch itself will only change once all of the migration users in the batch have been updated. Though its improved, managing status on large scale migrations using get-migrationbatch and get-migrationuser is generally a slow process.

I have some stats for you as well:

Name : MB_X-X

StartTime :

EndTime : 01.02.2018 12:53:50

MigrationDuration : 12 day(s) 11:27:39

MailboxCount : 287

TotalGBTransferred : 129,73

PercentComplete : 95,00

MaxPerMoveTransferRateGBPerHour : 2,41

MinPerMoveTransferRateGBPerHour : 0,00

AvgPerMoveTransferRateGBPerHour : 0,47

MoveEfficiencyPercent : 74,83

AverageSourceLatency : 4.056,92

AverageDestinationLatency :

IdleDuration : 0,00%

SourceSideDuration : 31,34%

DestinationSideDuration : 13,02%

WordBreakingDuration : 6,09%

TransientFailureDurations : 16,01%

OverallStallDurations : 26,70%

ContentIndexingStalls : 0,00%

HighAvailabilityStalls : 0,00%

TargetCPUStalls : 26,70%

SourceCPUStalls : 0,00%

MailboxLockedStall : 0,00%

ProxyUnknownStall : 0,00%

Thanks to everyone who helped me with this scenario, highly appreciated!

Exchange · General · Microsoft Ignite · Office 365

Microsoft Ignite 2017: BRK3155 – Thrive in as an enterprise organization in Microsoft Exchange Online

Large enterprise customers often have unique and specialized requirements for adoption Exchange Online. This session showcases the lifecycle of an enterprise customer leveraging features designed just for them. Demo and some several new features will be covered, such as Mailbox Plans, Client Access Rules, on send event APIs, and a first look at technology for mergers and divestitures.

The full recorded session can be found here:

This blog post covers a summary of the session. Continue reading “Microsoft Ignite 2017: BRK3155 – Thrive in as an enterprise organization in Microsoft Exchange Online”

Exchange · Office 365 · Microsoft Ignite

Microsoft Ignite 2017: BRK3154 – The epic Exchange preferred architecture debate

What is the best, DAS or SAN? Are SSDs on the way in or are slow spindles here to stay? Should you give up and migrate to the cloud? What about virtualization? This session covers the various Exchange architectures that can be deployed on-premises and hybrid.


  • Exchange 2016 now supports up to 192GB of memory
  • Item Recovery Enhancements20170927_163343930_iOS

Sample questions:

  1. Should I follow the PA? Yes. If it is possible, follow the PA. This is the tested and best practices solution from Microsoft running in Exchange Online. This simplifies the operation process in case of outage, failures, etc.
  2. Should I deploy SSD? No. Jeff Guillet mentioned a good example for this. If you are using SSD’s for Exchange, it is like you are driving a Ferrari on a gravel road.
  3. Should I virtualize Exchange? Yes and no. Of course, you can virtualize Exchange. Be sure that you use the calculator for virtualization 1:1 as you would do it for physical servers. Physical servers are more easier to manage and deploy because virtualization needs some more things to do.
  4. What should I do if I plan to have a hybrid deployment with O365? Follow the same approach as you would do it for an on-premises environment. If all your mailboxes are migrated to Exchange Online, use a single Exchange server for recipient management purposes only.
  5. What size mailbox should I deploy? Are 1GB mailboxes valid anymore? You already get 25GB mailboxes for free today and simple JBOD storage is a very low cost factor.
  6. Third-party archiving solutions or keep in Exchange? The Microsoft perspective is that archive mailboxes should be retain in Exchange. In case of big mailboxes and Outlook OST slider, there is no reason to use archive mailboxes anymore. If you are having strict compliance regulations for archiving, you can use Exchange, Exchange Online, or of course third-party archiving solutions.
Exchange · Microsoft Ignite · Office 365

Microsoft Ignite 2017: BRK4029 – Inside Exchange Online

Matt Gossage and Ananth Sundararaj show how Exchange Online works. The engineering leaders who design and build the infrastructure reveal the secrets of deep neural networks, machine learning, substrate, shards, and much more. They also share how these mystical creatures actually impact IT pros and users of Exchange and Outlook.

Continue reading “Microsoft Ignite 2017: BRK4029 – Inside Exchange Online”

General · Keynote · Microsoft Ignite

Microsoft Ignite 2017 – Keynote summary notes

      • Day 1 was the traditionally Keynote day of Microsoft Ignite 2017
      • Julia:
        • Automatically translation of the Keynote into 12 languages based on AI
      • Satya:
        • Digital Transformation of every life and industry
        • Enabling Digital Transformation20170925_131603154_iOS
        • Microsoft 365 describes the modern workplace
          • Mixed Reality and Hololens and Oasis Headsets as part of Microsoft 365
          • The frontier of new collaboration
      • Raanah (Modern Workplace)
        • Demonstration how Ford is using Mixed Reality with Hololens with Rachel and Terry
        • With Microsoft Teams, the team is alerted that someone created a note with Hololens
        • They will be able to join a call for all people around the world an all are seeing the same experiences with Hololens
        • Empowering people to collaborate in the same shared experience anywhere around the world20170925_132223481_iOS 1
      • Microsoft 365 and Microsoft Graph
        • It represents or it captures the knowledge for your people in their relationship with other people inside and outside your organization
        • First integration with Microsoft 365 and Graph was LinkedIn
        • For example, you can see all the others in your network who are attending Ignite
        • Using latest technologies with AI for Microsoft Graph
      • Li-Chen Miller (Modern Workplace)
        • Shows an example with LinkedIn Graph with Microsoft 365 Graph
        • Search
          • More intelligent, more personal, and more contextualized for search across Microsoft 365
          • Bing for Business and a direct connection to Office 365, Delve, and Azure Active Directory can transform all information in one place20170925_133623640_iOS
          • Bing for Business use machine reading comprehension and deep learning models for searching specific documents, policies, files, and other stored information
          • Cortana can connect to Office 365 and automatically set the auto email responder, if you tell her, you want to schedule vacation for a specific date and time
          • Bing for Business is available for a private preview today
      • AI-First and Mixed Reality applications, for example Trimble, using both Azure and mixed reality
      • Tetra Pak ins in a food packaging and processing for liquid food industry and is using Mixed Reality and Hololens
      • Land O’Lakes
      • University of Cambridge
      • Epic is using Azure, 200 mio records are stored in Azure. Detect long before medical problems happens.
      • Intelligent Cloud and intelligent Edge applications is happening today.