Insights

Broad global retention policies are commonly used incorrectly, which can have a number of unintended consequences as well as posing various risks to your organisation.

In our day jobs as technical Microsoft 365 consultants, we work in hundreds of Microsoft 365 tenants. In many of these tenants, we come across very broadly scoped retention policies. A common example is “retain everything for seven years”, based on when items were last modified and applied to every mailbox, every SharePoint site, every OneDrive for Business account, and every Microsoft 365 Group mailbox and site. I wanted to take some time to outline what these policies do, what they don’t do, how they impact the management of your data, and how they can have unexpected consequences. I’ll try to cover both the technical and non-technical impact, so bear with me as by necessity we dive into some of the technical details.

What these retention policies do

Policies of this type prevent files and emails from being permanently deleted for a specified period. Depending on how you configure the policy, when each file or email reaches the end of its retention period, the policy either removes the protection (so the files or email continues to exist indefinitely or until they are deleted) or it deletes the file or email automatically.

If a user deletes files or emails within the retention period, the behaviour is as follows:

• If a user deletes an email within the retention period, the email is moved from the Inbox folder to the Deleted Items folder. If the email is deleted from the Deleted Items folder, it is moved to a hidden Recoverable Items folder within the user mailbox. It remains here for the duration of the retention policy. The user cannot access the email, but administrators can recover it and query the content using Microsoft Purview eDiscovery capabilities.
• In SharePoint Online or OneDrive for Business, every version of each file within scope of the retention policy is given a unique identifier and copied to a hidden library named Preservation Holds on each SharePoint or OneDrive site. If a user deletes a file, the file is removed from its storage location, but each copy in the Preservation Holds library remains while the file within its retention period. This enables administrators to locate and query the content using Microsoft Purview eDiscovery capabilities.

You can also create policies to protect Teams channel messages, Teams private channel messages, Teams chats, Copilot interactions, Viva Engage posts, and Viva Engage messages in a similar way. However, Microsoft forces you to be slightly more granular with these platforms, so we’ll leave them alone today.

What they don’t do

These types of retention policies are not a Business Continuity / Disaster Recovery (BCDR) solution and do not provide backup and restore capabilities. Attempting to piece together content from locations like the Preservation Holds library would be incredibly unwieldy and prohibitive in terms of effort. Retention policies are designed to enable you to preserve and gather content using eDiscovery tools to support specific investigations, typically for legal or contractual issues.

What are the issues with broad retention policies?

The use of broadly-scoped retention policies creates various issues. Let’s look at a few of them.

Data governance

Data lifecycle management is a key element of effective data governance. Across your organisation, you will have data with a range of lifecycle management requirements, including content that you are obliged to retain for specific periods and content that you are obliged to delete, either after specific periods or when you can no longer justify retaining it.

Broad retention policies make this kind of differentiated lifecycle management impossible, as data is retained even when a user actively deletes it. Retention policies also always take precedence over any policies that delete content. Any SharePoint sites in scope of a retention policy cannot be deleted.

Inadvertent retention

Global retention policies often result in content being retained inadvertently, particularly in OneDrive for Business. Most technically-mature organisations that use Microsoft 365 will link known folders (the Desktop, Documents, and Pictures folders on your Windows devices) to OneDrive for Business accounts – any files you save in these locations are synchronised to and stored in OneDrive for Business. This has many benefits: files remain accessible if a user loses access to their device, users can access saved files securely from anywhere, and so on. However, any files stored in these locations will be subject to the retention policy. For example, if you receive a CV and add it to your desktop, even for 10 minutes, that file will remain in a Preservation Holds library for the duration of your retention policy.

Storage consumption

Every time you edit a file that is in scope of a retention policy, SharePoint or OneDrive will write a separate copy of that file to a hidden Preservation Holds library, adding a unique identifier to the end of the filename on each occasion. This has a substantial impact on the amount of storage you consume, which will substantially increase the additional storage charges you pay to Microsoft.

In addition, the global retention policy prevents the deletion of any SharePoint site in scope of the policy, which makes cleaning up your tenant and removing redundant sites challenging.

Impact on user experience

The retention policy has a negative impact on the user experience within in-scope SharePoint sites and OneDrive for Business accounts. Most notably, it prevents users from deleting a folder without first deleting the contents of the folder. In the case of heavily nested folder structures, this quickly becomes unwieldy.

What are the alternatives?

Microsoft 365 supports various more effective ways of managing data lifecycle. Retention labels and policies, and group expiration policies, are two examples.

Retention labels and policies in Microsoft 365

Retention labels and policies enable you to manage retention on individual files and emails by applying labels, or by setting default labels on specific locations. For example, you might create a label named CV (or Resumé if you’re in the US) that automatically deletes files after six months. We can make that label available on SharePoint sites, OneDrive accounts, and mailboxes used by your HR team. We can create a document library named CVs and set the CV label as a default, so any files the HR team drops into that library are automatically labelled and deleted after six months. Depending on your licence level, we can use also pattern matching or machine learning approaches to apply labels automatically to content that meets your criteria.

Retention labels and policies are a highly flexible solution and can be made to work in a variety of ways to suit most scenarios. The advantages over broad-scoped retention policies include:

• Retention labels can be deployed in a targeted manner to meet differentiated lifecycle management requirements.
• Retention labels provide a visual indication of what policy applies to a specific file.
• Retention labels provide robust support for reporting, so you can provide insight on where labelled content is stored.
• Retention labels can be configured to prevent users from deleting files in the first place, rather than allowing files to be deleted and copying them to a hidden Preservation Holds library.

Group expiration policies

Dormant file data is a common problem, particularly in self-service-created workspaces such as Microsoft Teams. While some workspaces gain traction and continue to be actively used, many fall dormant and are quickly forgotten. Data stored in these workspaces becomes a risk – organisations are often unaware of what type of data is stored or its sensitivity, content owners may have changed roles or left the company, and so on. Expiration policies can automatically remove dormant workspaces after providing content owners with multiple opportunities to intervene and preserve their data.

Conclusion

Retention policies in Microsoft 365 have a role to play, providing they are appropriately targeted and providing that stakeholders understand exactly what they do and don’t do. In many cases, retention policies are “mis-sold” as providing BCDR capabilities when they’re just not intended to do that, and broad-scoped retention policies can make effective data governance next to impossible. Better solutions exist both for BCDR and effective lifecycle management of your data in Microsoft 365.