Exchange · Office 365

Exchange Hybrid MRS vs. MigrationService Migrations

I would like to share some Exchange 2010 hybrid migration facts with you that we figured out.

First, again many thanks to Michael Van Horenbeeck! He helped me discuss this with a customer. I’m always very happy to work with him. And many thanks to Ben Winzenz and Jeff Kizner as well, I’m very grateful for your help.

In short: a customer is trying to keep about 65k mailboxes in sync to ensure a short cutover time. We are using a maximum of 1,500 mailboxes per batch, 5 batches per week, and switching 7,500 mailboxes with an overall data of about 5TB per week. For some technical details, we are using Azure ER (800 Mbit) for migration with 4 TMG as a proxy and some kind of F5 load balancing in between, PAW is activated, and two migration endpoints with each 100 sync/complete in parallel. We did some networking measuring and move request statistics and we had an average migration velocity of 18.6GB/h for batches starting the first incremental sync (0% to 95%) which is great. Of course, the migration velocity depends on the number of batches, mailboxes, mailbox items, network workload, etc.

We suspected if we keep a large number of mailboxes in sync might affect throttling or the overall migration velocity.

Another important part is how long MRS will sustain the synced mailboxes. The documentation https://support.office.com/en-us/article/Manage-migration-batches-in-Office-365-d164b35c-f624-4f83-ac58-b7cae96ab331 figures if a migration batch has no administrator interaction (nobody touching it) for 90 days, it will be automatically stopped. 30 days later, again if no administrator interaction, it would then be deleted. We have had some batches longer in a synced state for testing purposes, but we don’t want to fall into this limit in our production “migration” environment.

However, our problem was the delay in between when the request gets marked as complete, and when the batch status on the portal or Get-MigrationBatch gets updated and shows complete. We are using the status of the migration batch because there is some automated logic behind the migration for different pre and post hybrid migration tasks.

An example: a batch was started (CompleteAfter) 01/29 00:01 AM and showed completed in the portal and with Get-MigrationBatch at 01/29 13:00 PM. Nearly at the same time the email report from Exchange Online was sent as well.

I did some research with Ben’s help with the following cmdlets:

$moves = Get-MoveRequest -batchname “MigrationService:20180130T000100_MB_X-X” | Get-MoveRequestStatistics

$moves | select identity, completeafter, finalsynctimestamp, completiontimestamp

Interesting to see that the CompletionTimestamp flag for every move request (1,863 mailboxes in the batch) was finished only about 2 hours later.

Identity CompleteAfter        FinalSyncTimestamp   CompletionTimestamp

——– ————-        ——————   ——————-

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:15 AM 1/29/2018 2:22:08 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:35 AM 1/29/2018 2:23:23 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:41 AM 1/29/2018 2:22:40 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 2:20:51 AM 1/29/2018 2:23:03 AM

XXXXX    1/29/2018 12:17:19 AM 1/29/2018 3:13:16 AM 1/29/2018 3:14:24 AM

Another approach from Ben was to check the activity on the migration endpoints:

Get-MigrationEndpoint | fl *active*

ActiveMigrationCount       : 0

ActiveIncrementalSyncCount : 0

ActiveMigrationCount       : 0

ActiveIncrementalSyncCount : 0

20180212T000100_MB_X-X                     Syncing   ExchangeRemoteMove 96

20180221T000100_MB_X-X                     Synced   ExchangeRemoteMove 98

20181231T000100_MB_X-X                    Syncing   ExchangeRemoteMove 1322

20180215T210000_MB_X-X                     Synced   ExchangeRemoteMove 5

20180213T210000_MB_X-X                     Synced   ExchangeRemoteMove 5

20180214T210000_MB_X-X                     Syncing   ExchangeRemoteMove 14

20180221T000100_MB_X-X                     Synced   ExchangeRemoteMove 101

20180207T114420_MB_X-X                     Completed ExchangeRemoteMove 1

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 98

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 95

20180221T000100_MB_X-X                     Syncing   ExchangeRemoteMove 94

20180208T093847_MB_X-X                     Syncing   ExchangeRemoteMove 1

But it seems this is not accurate enough.

If I remember correctly, FastTrack is using move requests instead of batch migrations (or migration service) as well. In my customers case we have to use the migration batch and therefore the migration service as our migration plan. But this is only based on the organizational migration strategy from the customer, which means add, delete, change switch date and time, etc. must be done a lot for some batches. And every batch contains a single business unit so that all of them will be switched at the same date and time. Other features like health checking, reports, etc. are also very important for us.

At least Jeff gave us some interesting information: While MRS is actively processing moves and updating its status in real time with moving the data, migration service processes its queue serially. So as long as it has batches and migration users in its queue, it has to work its way through them all. Spending a slice of time on each one. Following on that, the status of the migration batch itself will only change once all of the migration users in the batch have been updated. Though its improved, managing status on large scale migrations using get-migrationbatch and get-migrationuser is generally a slow process.

I have some stats for you as well:

Name : MB_X-X

StartTime :

EndTime : 01.02.2018 12:53:50

MigrationDuration : 12 day(s) 11:27:39

MailboxCount : 287

TotalGBTransferred : 129,73

PercentComplete : 95,00

MaxPerMoveTransferRateGBPerHour : 2,41

MinPerMoveTransferRateGBPerHour : 0,00

AvgPerMoveTransferRateGBPerHour : 0,47

MoveEfficiencyPercent : 74,83

AverageSourceLatency : 4.056,92

AverageDestinationLatency :

IdleDuration : 0,00%

SourceSideDuration : 31,34%

DestinationSideDuration : 13,02%

WordBreakingDuration : 6,09%

TransientFailureDurations : 16,01%

OverallStallDurations : 26,70%

ContentIndexingStalls : 0,00%

HighAvailabilityStalls : 0,00%

TargetCPUStalls : 26,70%

SourceCPUStalls : 0,00%

MailboxLockedStall : 0,00%

ProxyUnknownStall : 0,00%

Thanks to everyone who helped me with this scenario, highly appreciated!

Exchange · General · Microsoft Ignite · Office 365

Microsoft Ignite 2017: BRK3155 – Thrive in as an enterprise organization in Microsoft Exchange Online

Large enterprise customers often have unique and specialized requirements for adoption Exchange Online. This session showcases the lifecycle of an enterprise customer leveraging features designed just for them. Demo and some several new features will be covered, such as Mailbox Plans, Client Access Rules, on send event APIs, and a first look at technology for mergers and divestitures.

The full recorded session can be found here: https://www.youtube.com/watch?v=pN6lsxKRrJQ&t=1503s

This blog post covers a summary of the session. Continue reading “Microsoft Ignite 2017: BRK3155 – Thrive in as an enterprise organization in Microsoft Exchange Online”

Exchange · Office 365 · Microsoft Ignite

Microsoft Ignite 2017: BRK3154 – The epic Exchange preferred architecture debate

What is the best, DAS or SAN? Are SSDs on the way in or are slow spindles here to stay? Should you give up and migrate to the cloud? What about virtualization? This session covers the various Exchange architectures that can be deployed on-premises and hybrid.

Announcements:

  • Exchange 2016 now supports up to 192GB of memory
  • Item Recovery Enhancements20170927_163343930_iOS

Sample questions:

  1. Should I follow the PA? Yes. If it is possible, follow the PA. This is the tested and best practices solution from Microsoft running in Exchange Online. This simplifies the operation process in case of outage, failures, etc.
  2. Should I deploy SSD? No. Jeff Guillet mentioned a good example for this. If you are using SSD’s for Exchange, it is like you are driving a Ferrari on a gravel road.
  3. Should I virtualize Exchange? Yes and no. Of course, you can virtualize Exchange. Be sure that you use the calculator for virtualization 1:1 as you would do it for physical servers. Physical servers are more easier to manage and deploy because virtualization needs some more things to do.
  4. What should I do if I plan to have a hybrid deployment with O365? Follow the same approach as you would do it for an on-premises environment. If all your mailboxes are migrated to Exchange Online, use a single Exchange server for recipient management purposes only.
  5. What size mailbox should I deploy? Are 1GB mailboxes valid anymore? You already get 25GB mailboxes for free today and simple JBOD storage is a very low cost factor.
  6. Third-party archiving solutions or keep in Exchange? The Microsoft perspective is that archive mailboxes should be retain in Exchange. In case of big mailboxes and Outlook OST slider, there is no reason to use archive mailboxes anymore. If you are having strict compliance regulations for archiving, you can use Exchange, Exchange Online, or of course third-party archiving solutions.
Exchange · Microsoft Ignite · Office 365

Microsoft Ignite 2017: BRK4029 – Inside Exchange Online

Matt Gossage and Ananth Sundararaj show how Exchange Online works. The engineering leaders who design and build the infrastructure reveal the secrets of deep neural networks, machine learning, substrate, shards, and much more. They also share how these mystical creatures actually impact IT pros and users of Exchange and Outlook.

Continue reading “Microsoft Ignite 2017: BRK4029 – Inside Exchange Online”

Azure · Office 365

Microsoft Enterprise Mobility Suite (EMS) – Identity + Access Management (IAM)

The growth of mobile devices such as smartphones and tablets changed the world rapidly. Most notably business users store important information on their devices such as emails, certificates, pictures, corporate apps and applications, etc. Maintaining control over their applications across corporate datacenters and public cloud platforms has become a significant challenge. IAM helps organizations to reduce helpdesk costs with self-service and single-sign-on experiences.

EMS – Enterprise Mobility Suite was introduced in the end of 2014. There is no specific product for EMS, it’s a collection of services you can choose.

Currently EMS contains the following services:

–          Cloud Identity + Access Management: gives users self-service capabilities and single sign-on for any corporate resource for easier identity management – for cloud-only and hybrid identities.

–          Mobile Device + Application Management: mobile device management, such as MDM in Office 365 and Intune to manage and protect corporate data and apps on almost any device.

–          Information Protection: information security management across on-premises environment and cloud applications while protecting corporate data inside and outside of the organization.

–          Desktop Virtualization: a scalable platform to deliver corporate applications simply and cost effectively – everywhere.

Continue reading “Microsoft Enterprise Mobility Suite (EMS) – Identity + Access Management (IAM)”