Did you know that some sign-ins are riskier than others? What’s more, you don’t have to go through all the Microsoft 365 audit logs or play detective on your own to tell them apart. Interested? Please feel free to read my post about Azure AD Identity Protection… Entra ID Protection, the service that is designed to detect and protect you from high-risk users and sign-ins.
What is Entra ID Protection?
Entra ID Protection, formerly known as Azure AD Identity Protection is a service designed to monitor, detect, and block suspicious and risky events (Risk detections). In short, Microsoft takes care of identifying and responding to any anomalies and potential attempts to exploit hijacked accounts with the magic of the cloud.
It is worth mentioning that the service runs completely in the cloud and we don’t need to install additional agents or proxies as in the case of Microsoft Entra Password Protection.
What data is ID Protection based on?
On data from your Entra ID tenant, as well as signals from other Active Directory services, Microsoft accounts, and even Xbox cloud gaming 🙂
This data is processed using machine learning, because Microsoft collects such a huge amount of events that it would be impossible to analyze them manually. This is a bit scary, but on the other hand, if Microsoft detects any threat or attack, we are automatically covered by the protection offered by Entra ID Protection.
What does Entra ID Protection cover?
Against a lot of stuff. The list is quite long, and I don’t want to copy Microsoft’s documentation point by point, so I’ll share this link with you.
However, I am not going to leave you with just a boring documentation. Below you’ll find a brief description of some of the risks that may appear in our tests.
Selected risks that Entra ID Protection protects against:
- Leaked credentials (Offline) - Entra ID checks for leaked passwords of our employees. If Microsoft finds a match to our user in their sources, we will be notified. Quite convenient alternative to the manual verification I mentioned in my previous post, but doesn’t necessarily replace it, as you’ll read later.
- Atypical travel (Offline) - Identifies sign-ins that would not be physically possible, such as logins from Poland and China 5 minutes apart.
- Anonymous IP address (Real-time) - Detects attempts to hide your true location, such as logging in from a VPN or TOR browser.
- Unfamiliar sign-in properties (real-time) - Entra ID analyzes our users’ login history for anomalies, such as a login from a new, unknown device or from an IP address an employee has never used before.
BTW: In the linked article, in the Detection type column, we can see which risks are verified in real-time (real-time - usually 5-10 minutes) and which require longer time for analysis by Microsoft (offline - up to 48 hours).
Entra ID Protection requirements
Remember to always check the service/product requirements before you start testing. I know from personal experience that some projects end right at this stage :winking_face:
Licenses
The case is quite simple. We need a Azure AD Premium P2 Microsoft Entra ID P2 license. While it is true that in the lower plans we can see something there, but it is unlikely that we will be satisfied with that.
Using this feature requires Microsoft Entra ID P2 licenses.
— Microsoft Docs - License requirements - What is Identity Protection?
If you have multiple applications integrated with Entra ID, you can buy Workload Identities Premium. This allows us to monitor if the credentials of those applications have accidentally ended up in a public repository on GitHub.
AD Connect aka Entra Connect
What does AD Connect Entra Connect have in common with Entra ID Protection? Passwords or more precisely, their hashes. Entra Connect may or may not synchronize password hashes with Entra ID (Password Hash Synchronization option). If it doesn’t, Entra ID Protection won’t be able to detect credential leaks for accounts in the local Active Directory, which is very important to us.
How do I check if I have PHS (Password Hash Synchronization) enabled?
Navigate to: Entra admin center -> Identity -> Hybrid management -> Azure AD Connect, then select your Entra Connect. In my case it is Connect Sync.
As you can see, the Password Hash Sync is enabled for my Active Directory domain.
Microsoft Defender for Cloud Apps (MDCA)
The Entra ID Protection documentation, more specifically the alert descriptions, mention Microsoft Defender for Cloud Apps service, but what is it?
What is Microsoft Defender for Cloud Apps?
Entra ID Protection alerts are mainly based on Sign-in logs. These include IP addresses from which sign-ins were made, application names, client/device information and authentication (MFA) / Conditional Access blocking details.
As you can see, you won’t find all the required data for handling alerts listed in the documentation. Let’s take the Mass access to sensitive files alert as an example. It is generated when a user accesses too many files in SharePoint or OneDrive. Does Entra ID store this information? No :winking_face:
Calculated offline. This detection is discovered using information provided by Microsoft Defender for Cloud Apps. This detection looks at your environment and triggers alerts when users access multiple files from Microsoft SharePoint or Microsoft OneDrive. An alert is triggered only if the number of accessed files is uncommon for the user and the files might contain sensitive information.
— Microsoft Docs - What are risk detections? - Mass access to sensitive files
In the Sign-in logs we have the events of the SharePoint or OneDrive sign-ins, but we won’t find any details of the user’s activity within the application. This is where Microsoft Defender for Cloud Apps comes in, a Cloud Access Security Broker (CASB) that stores such data, and Entra ID Protection simply uses it.
When deploying Entra ID Protection, it’s important to review your Microsoft Defender for Cloud Apps settings and, if you haven’t done so already, enable monitoring of Office 365 and Azure AD. This will give you access to user activity data within Microsoft 365 apps.
Conditional Access (CA)
At this point, I would like to redirect you to a separate post about Conditional Access, but I have not yet published it… So let’s start with a brief introduction to what Conditional Access is.
CA description in a nutshell
Conditional Access is kind of a must-have for any organization using Microsoft 365 services, so I won’t go into too much detail here. In a nutshell, it’s like a cloud firewall, but without ports and protocols, instead with applications integrated into Entra ID. In the CA rules, we specify which users and applications we want to protect, and what conditions must be met to access corporate resources.
Example
Let’s say your devices are managed by Microsoft Intune. Each of them has a compliant / non-compliant attribute that is controlled by your compliance policies. If you want to prevent non-compliant or unregistered devices in Intune from accessing Microsoft 365, you can create a conditional access rule to block them.
Risk-based Conditional Access
Back to Entra ID Protection. Conditional Access for our “identity defender” is a direct coercive measure that can either block potentially risky sing-ins or reduce the risk of high-risk users.
I don’t think blocking access needs any further explanation, but what exactly is the risk remediation/reduction option?
Not all risks that Entra ID Protection detects require an immediate raid by the IT department on an employee. Sometimes it’s enough to request additional MFA verification or a password change rather than blocking access to all applications. This is called a risk remediation policy.
Of course, it’s up to you to decide which risk levels qualify for remediation and which require access blocking and a meeting with IT.
Tips and tricks for Risk-based Conditional Access
What should you know before creating your first risk-based conditional access policy?
Remember to create exceptions for break glass accounts.
What can happen if the only Global Administrator account is blocked by Entra ID Protection? I don’t know, although I suspect it 🙂
Ensure that your employees have MFA configured.
If you create a policy that allows you to reduce risk by requiring additional MFA challange, but the employee has not previously configured MFA, you will block their access to resources.
Ensure that your employees have SSPR configured.
Like MFA, password reset relies on another service - Self-Service Password Reset (SSPR), which can be disabled or unconfigured. For more information on SSPR, see my previous post.
Entra ID Protection is risk-based….
…and it is determined by levels: High, Medium, Low and No risk (for Sign-in risk). As you may have guessed, not every sign-in will have the highest risk level, so use the Password Change / MFA options with caution.
Do not create policies in the Identity Protection blade.
Although it is still possible to create policies within the Identity Protection blade in the Entra admin center, it’s unclear when this will be discontinued. Microsoft recommends creating risk-based policies using Conditional Access Rules. This gives us more settings to use in our policies. So why make our lives harder?
Entra ID Protection risk levels
Since we can create Conditional Access policies that use certain levels of risk, surely they are written down and documented somewhere, right? Well, not really….
Microsoft doesn’t provide specific details about how risk is calculated. Each level of risk brings higher confidence that the user or sign-in is compromised. For example, something like one instance of unfamiliar sign-in properties for a user might not be as threatening as leaked credentials for another user.
However, nothing is lost on the internet, and in the depths of the web you can find old documentation which classifies the risks as follows:
| Risk Event | Severity |
|---|---|
| Leaked credentials | High |
| Impossible travel to atypical locations | Medium |
| Sign-ins from anonymous IP addresses | Medium |
| Sign-ins from IP addresses with suspicious activity | Medium |
| Sign-ins from unfamiliar locations | Medium |
| Sign-ins from infected devices | Low |
Are the above details up to date? Probably not, but it’s better than nothing 🙂
Entra ID Protection configuration
Identity Protection does not have a switch that activates the service in our tenant. If we have the appropriate licences in place, the service will simply start to assess the risk and provide us with insights in the Entra portal. What we can configure are the alerts and actions (conditional access) that are visible to our employees.
MFA and SSPR configuration
As I mentioned earlier, if we don’t configure MFA and SSPR, and require them in our CA policies, users will lose access to corporate resources, which is obviously something we don’t want. So how do we configure these two services?
MFA configuration
The MFA service doesn’t necessarily need to be configured, it’s more important to force employees to register authentication methods for their accounts. How do we achieve this? We can enforce MFA in any conditional access policy or use an option available under Identity Protection in the Microsoft Entra admin centre.
Navigate to: Entra admin center -> Protection -> Identity Protection -> Multifactor authentication registration policy, then configure the setting.
As you can see on the printscreen, I have enabled the MFA registration policy for the MFA-ADUsers-Enabled group, which contains my test users, while excluding my administrative accounts (break glass accounts).
The result? The user was asked to register an MFA when logging in. This can be skipped for the first 14 days, but we accept the risk.
SSPR configuration
The next thing we can use to reduce our users risk levels is to ask them to reset their passwords. In Microsoft 365, user-initiated password resets are handled by Self-Service Password Reset (SSPR). I’ve described how to configure SSPR before, so I encourage you to read that post.
Quick reminder on how to enforce SSPR registration
Navigate to: Entra admin center -> Protection ->Password reset -> Registration, and then tick Require users to register when signing in? to Yes.
SSPR Configuration - Important Note
Microsoft recently enhanced Entra ID Protection for hybrid accounts (synchronised with local AD). To enable password reset risk mitigation for such accounts, in addition to SSPR, you must also enable the appropriate option in the admin panel.
Navigate to: Entra admin center -> Protection -> Identity Protection -> Settings, and then tick Allow on-premises password change to reset user risk.
Email notifications
As we pay extra for email notifications (we cannot configure them in the free version), it is worth turning them on.
We have two types of notifications:
- Weekly digest - Weekly summary of risk events.
- Users at risk detected alerts - Alerts when a new user (or users) reaches a specified risk level.
Weekly digest
Navigate to: Entra admin center -> Protection -> Identity Protection -> Weekly digest, and… why we can’t add new members!
Only users with active Global Administrator, Security Administrator or Security Reader roles will be listed there and the first 20 people (per role) will receive notifications. If you don’t want notifications to be sent to a particular person, you can turn them off for that person. There is also an option to turn off all notifications at the bottom of the page (Send weekly digest emails)
What does such a message look like?
Users at risk detected alerts
Similar to the weekly digest, by default only users with active Global Administrator, Security Administrator or Security Reader roles will receive notifications. This time, we have the option to add a custom email address, for example, for our security department, which doesn’t necessarily deal with emails on dedicated accounts with higher privileges.
At the bottom of the page, we can also change the level of risk that triggers the alert.
What does such a message look like?
Conditional Access
Time to go to the Conditional Access settings and start creating our risk-based policies.
Known network locations
Even before setting up the first CA policy, it is worth setting up something that will automatically increase the credibility of our users, namely Named Locations.
Navigate to: Entra admin center -> Protection -> Conditional Access -> Named locations, and add your Public IP addresses with the Mark as trusted location option.
From now on, Entra ID Protection will treat all employee sign-ins from our company’s internal addresses as less risky.
Microsoft’s recommendations
I can’t tell you exactly what policies to put in place. Every company is different, and what may be onerous for one may not be for another. However, we can look together at what Microsoft recommends in their documentation :winking_face:
Microsoft recommends the below risk policy configurations to protect your organization:
User risk policy - Require a secure password change when user risk level is High. Microsoft Entra multifactor authentication is required before the user can create a new password with password writeback to remediate their risk.
Sign-in risk policy - Require Microsoft Entra multifactor authentication when sign-in risk level is Medium or High, allowing users to prove it’s them by using one of their registered authentication methods, remediating the sign-in risk.
Requiring access control when risk level is low will introduce more user interrupts. Choosing to block access rather than allowing self-remediation options, like secure password change and multifactor authentication, will impact your users and administrators. Weigh these choices when configuring your policies.
So we have recommended values for our settings. Time to create policies!
Sign-in risk policy
Before we get started, let’s take a look at what Sign-in risk is?
Sign-in risk is determined at sign-in and indicates the likelihood that the authentication attempt is not coming from our personnel.
Enforcing MFA sounds great to validate concerns about such an unauthorised sign-in attempt. Let’s create a policy to enforce it.
Navigate to: Entra admin center -> Protection -> Conditional Access -> Policies, and then click the + New policy button.
Our settings:
- Name: Test Sign-in risk policy
- Assignments: All users (Include), Break-glass accounts (Exclude)
- Cloud apps or actions: All cloud apps (Include)
- Conditions / Sign-in risk: Configure - Yes, Sign-in risk level - Medium, High
- Access controls / Grant: Require multifactor authentication
- Session: Sign-in frequency - Every time
- Enable policy: On
Done! Our policy should work within minutes.
User risk policy
Again, we start with the definition of our risk:
User risk applies to the user’s account and is verified offline (not in real time like sign-in risk), i.e. when Microsoft detects that something may be wrong with our account and it is likely to be compromised. For example, an employee’s password has leaked.
In this case, simply enforcing the MFA may not be enough, so we should also request a password change for the employee’s account after the MFA challenge.
Our settings:
- Name: Test User risk policy
- Assignments: All users (Include), Break-glass accounts (Exclude)
- Cloud apps or actions: All cloud apps (Include)
- Conditions / User risk: Configure - Yes, Sign-in risk level - High
- Access controls / Grant: Require multifactor authentication, Require password change
- Session: Sign-in frequency - Every time
- Enable policy: On
Done! Our policy should work within minutes.
Tests
Enough theory, it’s time for some practice 🙂 But before we start, a little explanation as to why you might feel cheated/deceived, although that was not my intention at all.
All the cool stuff I show on this blog runs and has been tested on my demo tenant. I don’t present here any information from tenants of the companies I work with. As you can probably guess, I don’t have a lot of data in my Entra ID. The Identity Protection real tests involve sign-ins and events that feed Microsoft’s algorithms, and unfortunately I don’t have users logging into their computers every day and creating a digital footprint in Entra ID. Therefore the tests are limited to what I can show without such records, but at the same time they show to some extent how the service works.
If you are able to work on your production tenant, Microsoft has prepared some test scenarios that you can use - Simulating risk detections in Identity Protection.
Leaked credentials
The leaked credentials test comes first. I have previously created a user with a very weak password which, according to haveibeenpwned.com, is compromised (leaked).
Unfortunately, I haven’t received any alerts from Entra ID Protection. I thought that Microsoft’s Leaked Credentials would actively monitor our passwords and return information about users with leaked passwords, but it doesn’t work as I originally thought. There is one important detail in the documentation that I missed:
Why am I not seeing any leaked credentials?
Leaked credentials are processed anytime Microsoft finds a new, publicly available batch. Because of the sensitive nature, the leaked credentials are deleted shortly after processing. Only new leaked credentials found after you enable password hash synchronization (PHS) are processed against your tenant. Verifying against previously found credential pairs isn’t done.
It is likely that my password was not detected because: 1) it was not found in a recent data leak detected by Microsoft and/or 2) the service needs to find both the username (login) and password in a match/leak (credential pair).
Well, a slight disappointment right at the start of the tests. It seems that Entra ID Protection is not a replacement for Entra Password Protection and manual passwords verifications that I wrote about earlier.
Conditional Access - Sign-in risk policy
Okay, how about we at least test the conditional access policies we created earlier and generate some nasty notifications in the admin panel. Let’s start with the Sign-in risk policy.
What our test scenario looks like:
Log in to your test account (to any service) to see if it is still working and has MFA and SSPR registered.
Open a virtual machine (you have one, right?) and download the Tor browser.
I don’t think I need to remind you to avoid doing this on company-owned equipment. The Tor browser itself is not dangerous, but the source you download it from may be. Also, downloading and running the Tor browser will most likely generate an alert in the AV/EDR console, so it’s better not to give your administrators a hard time.
Open the Tor browser and connect to the Tor network. Next go to any M365 service, such as https://myapps.microsoft.com
Confirm that you have received an MFA challenge.
These actions should be classified by Entra ID Protection as Anonymous IP address (Real-time).
Result?
The policy worked and I was prompted to confirm the login in the Microsoft Authenticator application. Technically, the risky sign-in was “fixed” by passing the MFA challenge, which we can see in the admin panel under Risky sign-ins.
Did such a sign-in trigger an alert (email)? No, because we mitigated the employee’s risk by passing the MFA challenge, and the user’s risk was not raised to Medium/High in Risky Users.
Interesting fact!
What will an employee see if he or she doesn’t have an MFA registered and our policies require it?
An error that doesn’t necessarily say that we don’t have an MFA 🙂
Conditional Access - User risk policy
Time for the second Conditional Access - User risk policy. It is triggered when user risk increases, for example, when an employee’s password is detected as compromised (leaked).
Unfortunately, this is where the test tenant problem comes in - how do you raise an employee’s risk level to high if you don’t have the employee’s activity in Entra ID?
The answer is: MFA Fraud Alert, or more specifically the ‘Report Suspicious Activity’ setting.
MFA - Report Suspicious Activity
What is ‘Report Suspicious Activity’?
Report Suspicious Activity is an enhanced version of the Fraud Alert. This option, as the name suggests enables users users to report suspicious login requests 🙂
After such a report, the user will receive a High User Risk status and will be subject to our Conditional Access / Identity Protection policies.
How to turn it on?
Navigate to: Entra admin center -> Protection -> Authentication Methods -> Settings.
Next, change the State setting to Enabled and click Save to apply the changes. Done!
The ability to report suspicious MFA attempts is now enabled. Time to test the User Risk Policy.
User risk policy testing
What our test scenario looks like:
Pretty similar to the Sign-in risk policy. We must initiate a sign-in that asks us for an MFA challenge, then deny and report it in Microsoft Authenticator.
Once a suspicious MFA request is reported, access to the account is not immediately blocked because our risk must first be re-evaluated. During this time, the user will see a screen saying that MFA has been declined. However, if we try to log in again and confirm the MFA, we will gain access to the company resources.
After a few minutes, we will notice our user in the Risky users section with Risk level High status, and an alert from Entra ID Protection (User at risk detected) will arrive in our inbox.
The only thing left to do is to fix the user risk by changing the password! Let’s log back in and change the user’s password.
We did it! The user received a notification to change his / her password, and the password was changed successfully. Additionally, this account was synchronized with the local AD, which confirming that Microsoft’s latest preview feature is working 🙂
One final check in the Entra admin center panel confirms that we have successfully eliminated the risk, which concludes our tests.
Conclusion
I hope you now understand how to detect and deal with risky logins and users in Microsoft 365 / Entra ID.
If you need more assistance or have unanswered questions, please visit the links listed below. Until next time 🙂