Sembee

Windows 11 Upgrade Failure – BitLocker, SAFE_OS / MIGRATE_DATA and 0x8007042B

Thu, 2 Apr 2026 13:25:00 +0100

I recently spent a lot longer than expected trying to get a Windows 11 23H2 installation to update. This was after many attempts at scripted updates which kept failing. Not my usual subject matter for this blog, but having spent a long time on it, I wanted to document it.

Scenario I was faced with:

- Needing to in place upgrade to Windows 11 25H2 due to a LOT of specialist software on the machine.
- Multiple attempts using various methods, such as Windows Update Assistant and an ISO install had failed
- Logs indicated a BitLocker issue
- Even after removing BitLocker, subsequent failures logged SAFE_OS / MIGRATE_DATA stage.

Step 1 - The BitLocker problem.

This is fairly common — once I had looked at the logs and confirmed BitLocker was involved, it was straightforward to fix.

BitLocker issues are usually one of:
- Active in a way setup doesn't like
- Mid-state - such as a suspension not applied correctly.
- Tied to something with the TPM chip or a change during the upgrade.

Check the status with

manage-bde -status

Protection was still active, so I suspended it:

manage-bde -protectors -disable C:

Confirmed of course:

manage-bde -status C:

Tried the upgrade again, still failed, so I decrypted the drive:

manage-bde -off C:

Still failed, but with a different error - so progress was made.

Step 2 - Persistent Upgrade Failures

With Bitlocker neutered, the upgrade still failed.
I tried a couple of techniques at this point:

- Fresh download, as the one I was using was from last year and there had been reports that the original ISO release had a problem.
- Copying the installation files locally.

Usually I would have wiped the machine by this point. However the client informed me that there is between 12 and 18 hours of setup time on this machine, plus the cost to get someone to do it, so I was being strongly discouraged from the wipe option.

At this point, I turned to AI, as it was a more complex problem than anything I had seen before. The logs went in to AI to see if it would make any suggestions.

AI immediately pointed to the migration phase (SAFE_OS / MIGRATE_DATA), and as there had been a problem with BitLocker earlier, that something with the TPM configuration was the most likely cause. AI was also able to confirm that a lot of the log was noise which I had been ignoring and was right to do so.

AI said that these four elements were the key indicators:

• 0x8007042B
• Result: 44
• V2VArbitrate
• "script" attribute is mandatory

Pointing to a migration engine failure. That kind of made sense.
There was a fair bit of back-and-forth while I got it to justify the conclusion - I needed to be confident before trying something this invasive and be aware of the consequences if it didn't work.

Satisfied that the solution it was suggesting was worth a try, I did the following:

1. Copied the Windows 11 ISO to the local machine.
2. Within the ISO files, I renamed the file tpmdriverwmi-replacement.man in the sources\replacementmanifests directory so it wasn't used during the setup.

Then as a further step, I cleaned up the previous upgrade attempts, as the logs had been showing some errors. This was a little more involved than expected.

rd /s /q C:\$WINDOWS.~BT
rd /s /q C:\$WINDOWS.~WS

Fairly obvious ones, not always present.
Sometimes a permission fix was required:

takeown /f C:\$WINDOWS.~BT /r /d y
icacls C:\$WINDOWS.~BT /grant Administrators:F /t /c

(I got very good at this by the end).

Then there is a registry key - learnt this one the hard way.

reg delete HKLM\SYSTEM\Setup\Upgrade /f

That often requires a ownership/permissions change to get it to all remove.

That still didn't do it.

The final fix was to actually remove the reference to that file from the OS:

Using an elevated command prompt:
notepad C:\Windows\WinSxS\migration.xml

Then searched for

microsoft-windows-tpm-driver-wmi

And removed the entry for it, including the <file> and </file> elements - of course a backup of the file was taken before editing!

After all that, running setup.exe without updates the installation went through.

I counted nine upgrade attempts, each taking two hours, so this went on for over two days. Fortunately the end user didn't need the device as it was used for sporadic specialist tasks, so I was able to keep plugging away.

Easily the most complex upgrade I have done, which just a few years ago I wouldn't have been able to resolve - I simply wouldn't have found that issue without AI.

When DKIM Exists But Still Fails: How Email Gets Broken in Transit

Mon, 30 Mar 2026 10:50:00 +0100

Following on from a recent issue with DMARC failures caused by missing SPF, I ran into a slightly different – and increasingly common – problem.

A client reported that emails from a supplier were being quarantined by their email filtering platform. The messages could be manually released, but even after adding the sender to the allowed senders list, the issue continued.

The quarantine reason pointed to DKIM failure.

What the headers showed

Reviewing the message headers revealed the following:

SPF: pass
DKIM: fail
Mail routed through a third-party relay service before delivery

At first glance, this might look like a simple DKIM configuration issue. However, the presence of a DKIM signature tells a different story.

DKIM was present – but invalid

In this case, the sending domain was signing outbound email correctly. The DKIM signature was present in the message headers.

However, by the time the message reached the recipient, the signature no longer validated.

This typically indicates that the message was modified after it was signed.

Why DKIM breaks

DKIM works by generating a cryptographic hash of selected message headers and the message body. That hash is then signed using the sender’s private key.

When the receiving system checks DKIM, it recalculates the hash and compares it to the original.

If anything in the signed portion of the message has changed, the hashes no longer match and DKIM fails.

Common causes of this include:

Disclaimer injection
Outbound spam filtering or relays
Message reformatting
Link rewriting or tracking additions

In this case, the message passed through multiple systems before delivery, making it highly likely that one of them altered the message after it had been signed.

Why whitelisting didn’t help

A common reaction in situations like this is to add the sender to an allowed list.

However, modern email filtering systems do not simply bypass authentication checks because a sender is trusted.

If DKIM fails, the system has to assume the message may have been altered or tampered with in transit.

As a result, the message is still treated as suspicious, regardless of allow list entries.

In other words, the system is behaving correctly.

The key takeaway

This was not a case of DKIM being missing or incorrectly configured at the source.

Instead, DKIM was broken in transit.

This is an important distinction, because it shifts the responsibility from DNS configuration to mail flow design.

If outbound email is being modified, then one of the following needs to happen:

The original message must be preserved so the DKIM signature remains valid
The message must be re-signed with DKIM after modification, using the correct domain alignment

Without this, email authentication will fail, and receiving systems will increasingly reject or quarantine those messages.

Final thought

Email authentication controls do not operate in isolation.

SPF, DKIM and DMARC all interact with each other, and with any systems that touch the message in transit.

Adding security layers without considering how they interact can easily result in legitimate email being blocked.

And as seen here, the problem is not always that authentication was never configured.

Sometimes, it was configured – but something else broke it along the way.

This is a different failure mode to missing SPF or DKIM entirely — here, authentication was present, but invalidated during transit.

DMARC Quarantine and Missing SPF: Why Legitimate Email Gets Blocked

Mon, 9 Mar 2026 10:55:00 +0100

At the end of last month I dealt with a mail delivery issue that is becoming increasingly common as organisations enable DMARC enforcement.

A user at a client reported that an important email was being quarantined by their spam filter. The messages could be manually released and the sender was even added to the allowed senders list, but it made no difference — the messages continued to be blocked.

The quarantine reason reported DMARC failure.

As organisations increasingly deploy DMARC to improve email security, configuration mistakes are becoming more visible. One of the most common is enabling a DMARC policy without properly configuring SPF or DKIM authentication. When that happens, legitimate email can be quarantined or rejected by receiving mail systems that are simply following the sender’s published policy.

Understanding What Happened

The sending organisation had enabled a DMARC policy of quarantine. This instructs receiving mail systems to treat messages as suspicious if they cannot be authenticated as coming from an authorised sending source.

In principle this is good practice. DMARC helps prevent spoofing and improves the overall trustworthiness of email.

However, the sender had not completed the configuration required to support the policy they had enabled.

They were using Google Workspace for email, but their domain did not have an SPF record published at all.

This meant that when a receiving mail system checked authentication it saw:

No SPF record
No validated sending source
DMARC policy instructing quarantine

From the receiving system’s perspective the message could not be authenticated, so the policy published by the sender’s own domain was applied.

The spam filter was simply following instructions.

Why Whitelisting Didn’t Help

A common response when email is blocked is to ask the recipient to whitelist the sender.

However, when DMARC enforcement is being honoured, authentication failure takes precedence. If a domain’s policy instructs receivers to quarantine unauthenticated mail, most modern mail security systems will comply with that policy.

In other words, this was not a spam filtering problem.

It was an authentication configuration problem.

The Fix

The technical fix was straightforward: publish the correct SPF record as recommended by the mail provider (in this case Google).

Once the SPF record was created, the sending source could be authenticated correctly. With authentication passing, the DMARC policy was satisfied and the messages were delivered normally.

How DMARC, SPF and DKIM Work Together

DMARC does not authenticate email on its own. Instead, it relies on SPF and DKIM to verify the sending source. If neither mechanism passes and aligns with the sending domain, the DMARC policy determines how the receiving system should handle the message.

Why These Issues Often Go Unnoticed

It is quite possible that the sending organisation had been experiencing similar delivery problems with other recipients without realising it.

Unless someone reports the issue, problems like this can persist quietly for months, causing friction between suppliers and customers.

In this case, because of the close relationship with my client and someone on the sender’s side keen to resolve the issue, the correct fix was implemented rather than relying on a temporary workaround.

A Reminder About DMARC Configuration

Major mail providers such as Google, Yahoo and Microsoft have tightened sender requirements in recent years. Proper authentication is increasingly expected for reliable email delivery.

If DMARC is enabled, the underlying authentication mechanisms must also be correctly configured. At a minimum this usually means:

A valid SPF record
DKIM signing enabled
Alignment between authentication and the sending domain

Enabling a DMARC quarantine or reject policy before those pieces are in place can easily lead to legitimate mail being blocked.

Sometimes the spam filter isn’t broken — it is simply doing exactly what the sender’s domain asked it to do.

Podcast - Holy Trinity for Email Delivery – DMARC, DKIM and SPF

Wed, 10 May 2023 14:05:00 +0100

This a blog post supporting one of the Cyber Anxiety podcast series, hosted by Inbay, alongside Daniel Welling of WellingMSP.

In this episode, Danial Welling, Luke Betteridge and I talk about the holy trinity for email delivery - DMARC, DKIM and SPF.

DKIM, DMARC and SPF have been described as the holy trinity for email delivery. Configured correctly they can improve the deliverability of your email and provide some useful reporting. Even if you are using a hosted email platform, you still need to be aware of them and ideally configure them for all services using your domain for email.

These are mainly DNS changes, with only DKIM requiring support from the email server to implement.

With the news that Microsoft Office365 now sends out DMARC reports, the value of setting up that system in particular has increased.

What they are NOT

These three standards are not something that will impact the amount of spam or other junk email that you receive. This has been a common reason why admins don’t take the time to implement them, because they are only interested in something which deals with the spam problem for their own users.

What they are

What they can do is improve the reliability of your email being delivered, and what remote sites do with emails that come from other sources, therefore enhancing the reputation of your email domain. On top of that, they can provide reporting on how those measures are doing and what recipient services did with the emails.

Furthermore, they can also help identity other services which are using your domain name, which could be harming your domain’s reputation, or simply discover that a service has been setup by end users without the involvement of their IT people.

All three are domain wide settings – it is not possible to exclude specific users.

However, you can use sub domains, and configure different records for each sub domain. That would allow you to have a separate sub domain for bulk or transactional emails, with its own set of records, allowing for more accurate reporting and reputational management.

SPF – Sender Policy Framework

SPF is the oldest of the three protocols, first discussed in the early 2000s.

As with the other two, it uses DNS extensively, but unlike DKIM, it does not require support of the sending server to work.

In an SPF record, you list what servers are allowed to send email for your domain. This could include your primary email server (Exchange, Office365 etc), then any marketing or sales tools. It could also include things like a payroll system, monitoring platform etc.

The drawback with this method is that because it simply lists IP addresses or hostnames, if a system gets compromised on the same network and starts sending emails from the same IP address, then it can pass the SPF test.

There are limits on the size of the SPF record, and there are various methods that can be used to reduce the size of the record. You will find online tools which can help flatten the SPF record to within the allowed sizes.

You can also only have one SPF record, so if a third party asks you to setup a SPF record, then you will need to incorporate their information in to your existing record (if you have one).

Be careful though – if you don’t have an SPF record then do not just implement the one from the third party. That can mean that you are telling the rest of the internet to only accept email from the third party and not from your primary source. For a first SPF record, follow the guidance from your primary email provider (Office365 or Google for example), or configure it with your servers if you are still hosting your own. You can find tools online which can help you create the first record. Then add in other providers to that record.

DKIM – Domain Keys Identified Mail

DKIM is not quite as old as SPF, but emerged in to the public view around the same time, and for a while was seen as a competitor to SPF as they both do the same thing – which is to tell the wider internet which servers are allowed to send email for that domain. However, as they do this in a different way, they are ideally suited to working together.

DKIM signs the message, with the public key held in DNS. It therefore proves the server sending the email is allowed to do so – even if the DNS or IP address changes.

DKIM uses a combination of DNS records and modifying the sent email, therefore requires support from the sending server. Exchange (on premises) doesn’t natively support DKIM, but Exchange Online from Office365 does, as do most third-party spam filtering services.

If you have multiple services that support DKIM in your outbound email flow, then it should be applied to the last server you control – i.e. the last hop before it goes out to the internet.

Each server or service which signs email using DKIM has its own set of DNS records – referred to as an identifier.

DMARC – Domain-Based Message Authentication, Report and Conformance

DMARC is the newest of the three, from around 2012, and emerged to cover gaps in SPF and DKIM – specifically telling receivers what to do with the email if it fails one of those tests, and also to provide reporting on what those receivers did.

As with the other two, it requires DNS records in each domain. As the sender, there is nothing to do on the server. As the recipient, you need to use a service that supports DMARC.

A DMARC record contains two parts – action and report.

The action element allows the sender to tell the recipient what to do with emails that fail the SPF and DKIM test. It has three stages – report (do nothing), quarantine (put in to the spam folder) and reject. On top of that, you can also set a percentage value – so the receiving server can be told to only quarantine or reject some of the failing email, allowing the email administrator to catch a potential configuration problem.

The end goal is to get to 100% reject, but that can take some time while the various services are found and setup correctly - either by moving to a subdomain, adding to the SPF record, and ideally having a DKIM record setup (if supported).

The reporting element is an email address, which is where the reports are sent to, and depending on the volume of email, you can receive multiple reports per day from the same recipient domain. The reports themselves are in XML format – they are not designed to be read by a human. The format is standard and they are designed to be sent to a service provider, who processes those reports and then presents them to their customer in a more structured format allowing for more sophisticated reports to be created. This allows patterns and trends to be noticed, and the configuration of the settings adjusted. As confidence grows, based on the reports, you can consider moving to quarantine and then reject, monitoring the reports to ensure that nothing is being rejected which shouldn’t be, due to incorrect or missing DNS records.

Not all providers support all functionality of DMARC. Google does for example, not only respecting the first part about what to do with the email if it fails a SPF or DKIM test, but also returning the reports back to the sending domain.

Microsoft Office365 have respected the DMARC policy settings, but were not sending out the reports, but that changed in March of 2023, with reports now being sent for most of their tenants where the MX records point directly to Microsoft, although at the time of writing it is still in preview. If you are using a third-party filtering service in front of Office365, then that service needs to send DMARC reports instead.

Their consumer targeted service (Hotmail/Outlook.com) has been sending DMARC reports for some time.

Full details on DMARC on the project web site at https://dmarc.org/

No Email Domains

If you have domains that do not send email at all, then configure an SPF and DMARC record to indicate that. Spammers will often use dormant domains as their spoofed from address. Therefore, by configuring SPF to say that there are no servers authorised and setting DMARC to 100% reject, you help out the rest of the internet community by not allowing your domain to be abused. You can omit the reporting email addresses from the record if you not want to receive reports for those domains.

Where to start?

All of this can seem daunting, particularly if you are an MSP. It can also take months to get to the point where you can confidently set the DMARC record to reject, so it is difficult to know where to begin.

The strategy I use is very simple.

On each client I will setup an email address for DMARC END User reports – these are the reports generated by the various DMARC reporting services. This could be something like dmarc-reports@example.com. The email address is on a group, and as the service provider, I will have an email address on my system that is a member of that group. You can also include any internal recipients at the client in that list.

Next, I will create another email address – dmarc@example.com which is used to receive the actual XML files. Again, a group, and could also include something like a public folder so that you can see who is actually sending reports.

I then start with the free Postmark weekly DMARC report - https://dmarc.postmarkapp.com/

That will give you the first DNS record for DMARC and ensure that you have configured it correctly. Set it up for each domain you control, using the dmarc-reports@example.com email address that you setup earlier.

Once it has been configured you need to test the DMARC setup. There are lots of test sites out there, but I find the most user friendly one is https://www.learndmarc.com/ .

If you decide to use additional services, simply take the email address that postmark provided to you and add it to the dmarc@example.com group as a member. Then change the email address in the DNS record to dmarc@example.com. As you sign up for other services, simply add the email address they provide to you to the group – that allows multiple services to get the same emails.

You can add multiple email addresses to the DNS record itself, but eventually you will hit a limit on the record size, and the group method is a good way to get round that.

For DKIM, if you are not using a third-party email filtering services for outbound email on Office365, then enable it in the Office365 console. Microsoft have instructions on how to do that here: https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/email-authentication-dkim-configure?view=o365-worldwide

It takes a few minutes to configure DMARC per domain.

For DKIM, if you are using a third-party service for outbound email, then look in their service guides on how to configure DKIM.

Similarly, if you know that the client is using other email services, such as SendGrid or Salesforce, then setup DKIM in their service as well. Remember each service has its own DKIM records, with a unique identifier.

For SPF, you may already have an SPF record as they are in the standard DNS records suggested for Office365, and most third-party spam providers will also ask you to configure one. If you don’t have one, then wait.

With that done, leave it a week or more, for the reports to build up. Review the reports to see what services are sending email as your domain. You can then investigate whether they are in the SPF record, whether they support DKIM etc. If you don’t have an SPF record then you can look up what the service provider suggests and use the various tools to create the SPF record.

DMARC Reporting Provider

When you start looking at DMARC reporting providers you will usually see that they charge per domain and also per email volume. The volume will be the number of emails that generate DMARC reports, not the total volume of email that you send. Use the trial periods to see what kind of volume emails are generating reports and then shop accordingly.

DNS record management

I see a lot of errors with DNS records, including duplication and multiple records not being supported, usually because service providers will provide a record presuming that it is only their service you are using, when you actually need to combine records. Therefore, to be clear on the DNS records:

SPF – single DNS record, but it can reference other records.
DMARC – single DNS record, but it can have multiple email addresses listed.
DKIM – multiple DNS records, one or more for each service that is sending email.

On Going Maintenance

It is important that the reports are reviewed regularly (which is why I like to start with the Postmark report as I get it once a week), to ensure that no new services have been introduced which need to be included in the various DNS records. As the confidence in what services are sending email grows, you can move from the default report setting, to a quarantine setting.

Summary

The holy trinity of SPF, DKIM and DMARC can be a very useful tool for IT service providers, whether internal or an MSP. It can not only help with email delivery, but it also identifies other email services that are being used and ensure that they are setup correctly in compliance with the best practises for email configuration. It can also show if the domain is being abused so that the client is aware. As the initial setup takes a few minutes and costs nothing, it can be something that all IT service providers can setup and configure for their clients.

DMARC Service Providers and useful links

The most comprehensive and independent list of DMARC service providers and other resources is available at https://dmarcvendors.com/

If you would like to listen to the rest of the series, then they can be found here:

Podcast - Conditional Access

Thu, 20 Apr 2023 15:30:00 +0100

This a blog post supporting one of the Cyber Anxiety series, hosted by Inbay, alongside Daniel Welling of WellingMSP.

The latest one is about Conditional Access in Office365

Conditional Access in Office365

Conditional Access is probably the most powerful tool available for security of your and your client’s Office365 tenants. When used in combination with Multi Factor authentication (MFA), it can keep most of the bad actors away, but can also protect your client’s data. It is such a powerful tool that I can only give you a brief introduction to the power it has.

First a warning – it is very easy to lock yourself out of the tenant with Conditional Access, so always have an emergency access account configured and confirmed as working.
This is documented on the Microsoft web site (https://docs.microsoft.com/en-us/azure/active-directory/roles/security-emergency-access) . If you have any rules already configured, then exclude this account from all of them.

What is Conditional Access?

Conditional Access is a feature of AzureAD which allows you to control who and what can access the Office365 tenant. It can enforce the use of a managed machine, MFA, location and approved apps, giving the administrator full control over the tenant access.
It can also be used to limit MFA prompts, which are one of the main barriers to adoption of MFA.

During initial adoption, the rules can be run in Report Only mode, therefore allowing you to catch anything that could be outside of the ruleset before it goes live. Enabling a small Azure subscription (usually less than £10/month) will allow more advanced reporting. This can be very useful to show to the client to demonstrate what conditional access could or is blocking. If you followed my advice at the start about configuring an Emergency Access Account, then the same log workspace can be used.

Two of the most common uses for Conditional Access are Country Restrictions and Securing Sign Ups for MFA.

Country Restrictions

One of the easiest to implement, but most effective uses of Conditional Access is to restrict what countries can access the tenant. If you or your client are all located in the UK, then restricting login to the UK will stop a lot of attacks, even if an account gets compromised.
Conditional Access works on the Block Everything, with exceptions rule, so you will need to build a list of countries that can access the tenant (typically I do UK, ROI, Jersey, Guernsey and Isle of Man).

The only drawback is if a user goes travelling and needs to access company resources. To combat that, create a group for the exceptions, then add and remove users from that group as required. Make sure that end users know to inform you that they are traveling.
It can be tempting to allow some users to be in the exception group all of the time. If this is a high value account (CEO, MD etc) then this should be discouraged, because they remain a target.
However, be creative – if an end user has a holiday home where they spend a lot of their time, then build an exception ruleset for them. If the staff member can get a static IP address, then even better as you can restrict it to that location only.

Securing MFA

Conditional Access is the preferred method to enforce the use of MFA, but you can also use Conditional Access to secure MFA. If you are using trusted location to allow office-based staff to bypass the need for MFA, then you can use Conditional Access to ensure that those users cannot have their account abused. A common attack would be for a bad actor to phish the user’s security details from them, then sign up for MFA using their own phone and are able to access the tenant from wherever.
Therefore, configure a trusted location and then restrict MFA sign up to that trusted location – so a user has to be in the office to sign up for MFA.

Spend some time with the conditional access documentation and see how you can secure both your own tenant and those of your clients. Just don’t lock yourself out!

Luke, Daniel and I discuss conditional access in the pod cast series Cyber Anxiety, the link to it can be found above.

If you would like to listen to the rest of the series, then they can be found here:

PodCast - Disaster Recovery & Business Continuity Planning

Mon, 27 Mar 2023 10:30:00 +0100

This a blog post supporting one of the Cyber Anxiety series, hosted by Inbay, alongside Daniel Welling of WellingMSP.
The latest one is about Disaster Recovery and Business Continuity Planning.

Is Disaster Recovery and Business Continuity an Outdated Concept, and if not, where to start?

Disaster Recovery or Business Continuity Planning is something that all MSPs should have not only for themselves, but also for their clients.

However, with the growth of cloud-based services, it could be argued that DR/BCP is now an outdated concept which most companies no longer need. Yet the company still needs to operate, so even if the plan is simply to work from home, it is still valuable to have a plan which can be communicated to staff.

Disaster Recovery – what is it and where to start?

Traditionally, what is a Disaster?

Simply put a disaster should be considered the loss of something within the business. All of the below would mean that a DR plan of some kind maybe activated.

Loss of the building.
Loss of access to the building.
Loss of data.
Loss of internet access.
Loss of electric supply.

The likelihood of the above and their impact will play a big part in the planning that needs to take place.

With the warnings in the Autumn of 2022 that we could have seen scheduled power blackouts, planning for that kind of scenario seems a good opportunity to open conversations with your clients regarding their plans and see what can be adapted or needs to be changed.

Where to Start with DR Planning?

A lot of companies and their IT support will look at DR and not know where to begin, or because some or all of their services are now in the cloud, feel they are at least partially covered.

Obviously the first thing to consider is what is still physically in the office and would therefore be affected by a local failure.

Then, after looking at the list above, it should be obvious that a lot of clients and their IT support will already have the beginnings of a plan, which can be adapted and expanded for other scenarios.

Start by asking the client a few simple questions:

What do you do when the power goes out?
What do you do when the internet goes down?
What do you do if everyone is snowed in and cannot get to the office?

For power loss, this could be a UPS that requires its own room and a generator the size of a small van in the car park, at the other end of the scale, something as simple as pen and paper or more sophisticated such as telling everyone to go home and work from there.

For loss of the internet, you might already have a 4g gateway available to take to a client at a moment’s notice.

If everyone is snowed in, do staff work from home, have soft phones for example, or is it just a day off?

For loss of data, you should already have a plan for dealing with ransomware.

All of those elements can be used and adapted for the more serious events. If the client has everything in the cloud already, then all you might have to add to a plan is how to get access to the data in a secure manner. That can be something as simple as renting dedicated servers and building a remote desktop services farm, or even just deploying Azure Virtual Desktop.

For on premise servers, if you have protection for encryption malware, then that protection may well be adaptable for dealing with other scenarios.

Disaster Recovery shouldn’t be seen as a massive thing that can overwhelm you or your clients. A basic plan can be put together which can be easily adapted for most clients IT needs. Even if the client then requires more advanced needs, this can be a good start and also a valuable source of revenue with the initial planning and then annual reviews to ensure the plan keeps up with the technology that they are now using.

Luke, Daniel and I discuss DR in the pod cast series Cyber Anxiety, the link to it can be found above.

If you would like to listen to the rest of the series, then they can be found here:

Version Error When you Install Exchange in Recovery Mode Post Jan 2023 SU

Wed, 25 Jan 2023 21:00:00 +0100

Quick post to say that if you are attempting to recover a server with the Jan 2023 security update installed, it will fail with a version mismatch error and ask you to use a later version of the installer. Microsoft have published a fix for the error.

https://learn.microsoft.com/en-us/exchange/troubleshoot/setup/version-error-in-recover-server-mode-install

Exchange 2016 and Exchange 2019 Certificate Management - Post April 2022

Fri, 29 Apr 2022 11:05:00 +0100

In April 2022 Microsoft released CU 23 for Exchange 2016 and CU 12 for Exchange 2019.
While these updates, which were much delayed, were very welcome, one of the changes which wasn't announced was the removal of the GUI management tools for SSL certificates.
With Exchange being heavily web based since Exchange 2010, SSL certificates play a key part in ensuring Exchange and its clients work correctly. The GUI controls have been in place since Exchange 2010 was released.
However the commands used for Exchange 2007 don't work, so a new set of commands is required.

Renewing an Existing Certificate

For most people, you will be renewing the certificate. If you don't have any changes to make to the current certificate settings, then a simple on-liner of PowerShell will issue a new renewal request for you to use with your preferred SSL provider to get a new certificate - just enter the thumbprint of the current certificate in the sample below:

$txtrequest = Get-ExchangeCertificate -Thumbprint 123456789012344567890 | New-ExchangeCertificate -GenerateRequest -PrivateKeyExportable:$true

[System.IO.File]::WriteAllBytes('C:\SSL\renewal.req', [System.Text.Encoding]::Unicode.GetBytes($txtrequest))

Use get-exchangecertificate on its own to list the certificates currently installed so that you can get the thumbprint.

New Certificate Request

However if this is a new server installation, or you need to change the current certificate configuration (for example to remove autodiscover.example.com as you are in hybrid), then you will need to create a new certificate request.
Another one-liner will do this, but does need to be constructed beforehand with the relevant information:

$txtrequest = New-ExchangeCertificate -GenerateRequest -PrivateKeyExportable:$true -SubjectName "c=GB,o=Test Company,cn=mail.example.com.com" -DomainName autodiscover.example.com,mail.example.net,autodiscover.example.net

[System.IO.File]::WriteAllBytes('\\Exchange01\ssl\example.req', [System.Text.Encoding]::Unicode.GetBytes($txtrequest))

Picking apart that request…

The first part is the ISO two letter designation for your country, followed by your company name.
cn= is the common name, which is usually the name used for OWA/ActiveSync etc as it will appear directly on the SSL certificate. In the old wizard, it would be set to the root of the domain (example.com) but most people would change it to mail.example.com or whatever URL they were using for OWA.
The -DomainName elements are the additional names on the certificate. If you are supporting multiple domains with Exchange and need Autodiscover to work directly, rather than one of the other methods, you need to include them here.
Finally is the location to place the certificate request. Unlike the first one-liner, this requires a file share, just like it did with the wizard. As with the old wizard, I would create a dedicated share for this process on the Exchange server, and give everyone full control. That will ensure that Exchange can write to it.
The -PrivateKeyExportable:$true allows the certificate to be exported for use in another server. The default is false.

Completing the Certificate Request

Whether a new or a renewal request, once you have the certificate issued by the SSL provider, you need to get it in to Exchange.

Use the same share as above…

Import-ExchangeCertificate -Server Exchange01 -FileData ([System.IO.File]::ReadAllBytes('\\Exchange01\ssl\response.cer'))

Where -server Exchange01 is the server where the certificate request was generated.

Exporting the Certificate for use on Other Servers

If you have multiple servers you will need to export the certificate to a PFX file, and then import back in again.

Once again, run get-exchangecertificate to find the thumbprint, then put it in the below one-liner. You will probably want to use a more secure password as well! Note that this command exports to a local folder, not a file share.

$bincert = Export-ExchangeCertificate -BinaryEncoded -Thumbprint 98765432109876543210 -Password (ConvertTo-SecureString -String 'Password123' -AsPlainText -Force)

[System.IO.File]::WriteAllBytes('C:\SSL\export.pfx', $bincert.FileData)

To import the file, use the following command, which goes back to using a file share for the source

Import-ExchangeCertificate -Server Exchange02 -FileData ([System.IO.File]::ReadAllBytes('\\Exchange01\ssl\export.pfx')) -Password (ConvertTo-SecureString -String 'Password123' -AsPlainText -Force)

Enabling the Certificate

With the certificate now installed, the final step is to enable it.

enable-exchangecertificate -ThumbPrint 98765432109876543210 -Services IIS

Services can be IIS, SMTP, IMAP and POP - any combination of them.
If you choose to include SMTP and get a prompt to replace the default certificate, then choose no.

Update May 2022

This post was originally written using a single test server, so exporting the certificate wasn't required. The commands supplied by Microsoft in their process are as used above, but the resulting certificate cannot be exported. I have updated the above commands to include the ability to export the certificate so it can be used on other servers.

Windows365 - The MSP Perspective

Thu, 15 Jul 2021 14:07:00 +0100

Microsoft have announced the much anticipated Windows 10 in the cloud - Windows365 (W365).
Having spent what seems like most of the past six months talking to various MSPs about Azure Virtual Desktop (AVD) (previously called Windows Virtual Desktop), the question is how does this new service fit in with what they have been working on, have in pilot stage etc.

It is becoming quite clear though that this is actually really good news for MSPs, particularly those that support the smaller companies of less than 100 seats.
While AVD is a really good product, and has a lot of good uses, for small companies it doesn't make much sense because of the complexity of setup, configuration and management.

What immediately came to mind when comparing AVD and W365, was the comparison between a regular Windows Desktop and Remote Desktop Services (RDS).

RDS has two main uses - providing a standard desktop to a large number of users, or providing a single application on dedicated servers. These functions I see being deployed on to AVD - it feels like the natural replacement. The classic example would be something like a call centre, where the staff are using a small number of apps, but intensely, probably not requiring the full Office suite and other applications.
However AVD has similar challenges of setup and management complexity to RDS, making it more of a challenge to get right, with the deployment requiring constant tweaking to get the balance between performance and cost just right - as AVD is priced on consumption.

For many companies though, a full Windows 10 desktop is the better option, because the staff member is using many different apps, needs the full Office suite and other applications. That makes W365 the better option, particularly being priced per user and not on consumption.
If the company is already invested in Office365, with email, OneDrive and SharePoint in extensive use, then having the desktop in the cloud and close to those data points will also bring performance gains.

For MSPs, it becomes even more clear cut.

A common challenge, particularly when it comes to taking on new clients is getting any kind of standardisation on the workstations. We all have the horror stories of the customer taking on a new member of staff and then going to the local computer shop, buying a "cheap" desktop then calling their MSP to get it to work (with Windows Home, and other garbage on it).
The more recent issues of staff working from home and general supply issues have made the end user workstation more of a challenge.

However if the end user workstation can be pretty much anything that runs an RDP client, then the problem almost goes away. With solutions to use a raspberry PI as thin client, the MSP can almost leave spares at each client for such events such as new member of staff or a system failing. If the Windows session exists in the cloud and their access device fails, they just move to something else and carry on. Power cut? Send them home. Self isolation? Work from home with everything you need available in the cloud PC. No computer at home, then a cheap chrome book or

I have a client who is getting close to replacing most of their field staff laptops with what would have been a AVD deployment and Samsung mobile phones (using their DEX feature), but that will probably be switched across to W365. That provides not only a consistent experience across all staff, but also provides some degree of data protection, not only from staff stealing content, but also by loss of the device. There are also considerable cost savings - the laptop and its maintenance for a start.

Yes there will be cases where a desktop is the better solution, but as those could be seen as niche cases the MSP will have a good opportunity to ensure those niche desktops are bought, built and managed in the way that best fits their technology stack.

From the MSPs commercial point of view, it will also allow the MSP to provide a single price per user which includes everything - Windows, office, AV and other security software, monitoring and support, with the only additional cost being a standard router in the office and whatever is on the desk. Supporting work from home and nomad users will become easier and more cost effective.

While the goal of a simple serverless environment for these small businesses has been possible for sometime (I did it many years ago for a small marketing company), there were trade-offs in performance and complexity. Windows365 takes away those two main issues, making that goal within reach of more companies (And their MSPs).

Therefore I simply hope that Microsoft get the pricing right...

Microsoft Announce Windows 365

Wed, 14 Jul 2021 16:18:00 +0100

Just as everyone was starting to look at Azure Virtual Desktop (formally Windows Virtual Desktop), Microsoft announce another new service - Windows 365.

https://www.microsoft.com/en-gb/windows-365

https://www.microsoft.com/en-us/microsoft-365/blog/2021/07/14/introducing-a-new-era-of-hybrid-personal-computing-the-windows-365-cloud-pc/

The big difference is that the main version that will appeal to small businesses supports AzureAD as the primary and sole domain - unlike Azure Virtual Desktop which still requires a hybrid domain with your on premise or a full server in Azure. For companies that want to go completely serverless, this is going to be the product they will go for.

For MSPs, particularly those that support the smaller companies, this could be a game changer with regards to support. Convert the current desktops in to thin clients, or even just switch to thin clients and have everyone running on the same virtual platform, whether they are at home or in the office. New member of staff joins, they can bring their own machine in, or just have something sat on the shelf waiting. You could even use a Raspberry PI for access!

Of course the main thing here is going to be pricing, which we need to wait a few more weeks for. As it is priced per user, rather than time based as with Azure Virtual Desktop, Microsoft get it priced right, it could be a winner.