Hunting a Rogue App in Microsoft 365: Detecting Abused Application Credentials
In a test Microsoft 365 tenant, simulate an app that signs in as itself with its own secret, then find it in the audit and sign-in logs, and lock it down. Inspired by the ConnectWise Cloud Backup incident.
Authorized environments only. Every lab runs in an isolated environment I own. Never test systems you don't have written permission to test.
Scenario
A backup vendor's app has been granted access to your Microsoft 365 tenant. Its credential is stolen, and the attacker signs in as the app itself. There is no user, no password prompt and no MFA, because app-only sign-ins don't use them. This is the pattern reported in the ConnectWise Cloud Backup incident (still developing at the time of writing).
In this lab you reproduce that pattern in a test tenant you own, find the evidence it leaves behind, and restrict what the app can reach.
Objectives
- Explain why app-only (client credentials) sign-ins bypass MFA and Conditional Access for users.
- Simulate an app with broad mail permissions and a client secret.
- Detect the key events: consent, new credentials and app sign-ins.
- Inventory every app and its high-risk permissions.
- Contain the app and limit its mailbox scope.
Environment
- A Microsoft 365 test tenant with two or three fake user mailboxes and a Global Administrator account you control.
- Microsoft Graph PowerShell module and Exchange Online PowerShell module on a workstation.
- Audit logging enabled. Mailbox-level
MailItemsAccessedevents need the right licensing (Audit Premium). Check what your test tenant has and note it in your findings. - Optional: Entra logs sent to a Log Analytics workspace, for the KQL step.
Steps
1. Create the "vendor" app (simulated attacker foothold)
- In Entra admin center, go to App registrations > New registration. Name it
LabBackupVendor. - Under API permissions, add Microsoft Graph Application permission
Mail.Readand grant admin consent. - Under Certificates & secrets, create a client secret. Record the time you did it, as you will look for it in the logs.
2. Use the app with no user involved
Request a token with the client credentials flow, then read one fake mailbox through Microsoft Graph (/users/{id}/messages). Note that nothing asked for MFA at any point. Record the time of the calls.
3. Find the evidence in the audit log
In Purview audit search, filter by the time window and look for these operations:
Consent to applicationAdd service principal credentials/Update application – Certificates and secrets managementMailItemsAccessed(if licensed)
In Entra, open Sign-in logs > Service principal sign-ins and find LabBackupVendor. Note the IP address, resource and time.
4. Detect it with KQL (optional)
If service principal sign-ins go to Log Analytics, start with:
AADServicePrincipalSignInLogs
| where TimeGenerated > ago(1d)
| summarize signIns = count(), ips = make_set(IPAddress) by ServicePrincipalName, AppId
| order by signIns descThen adapt it: alert when a known app signs in from an IP outside its normal range.
5. Inventory risky apps
With Microsoft Graph PowerShell, connect with read-only scopes (Application.Read.All, Directory.Read.All) and list service principals and their app role assignments. Flag any with mail, files or directory write permissions (for example Mail.Read, Mail.ReadWrite, Files.ReadWrite.All, Directory.ReadWrite.All).
For each flagged app, record: owner, who consented, when its secrets were created, and whether you still need it.
6. Contain and limit
- Disable the service principal (Enterprise applications > Properties > Enabled for users to sign-in: No) and remove its secret.
- Restrict the app's mailbox reach in Exchange Online, so it can only see the mailboxes it needs. Check current Microsoft guidance on RBAC for Applications, which is replacing Application Access Policies.
- Repeat step 2 and confirm the app is now refused on mailboxes outside its scope.
Findings
To be completed after you run the lab. Record your own results here:
- Which events appeared in the audit log, and how soon after each action?
- Did
MailItemsAccessedappear in your tenant? What does that say about your licensing? - What did the service principal sign-in entry show (IP, resource, status)?
- How many apps in your tenant hold high-risk permissions?
- Did the mailbox restriction block access as expected?
Lessons learned
- App credentials are as powerful as a password with no MFA. Treat secrets and certificates as high-value.
- A vendor shutting down its service doesn't remove what an attacker already created inside your tenant: extra secrets, consents or tokens. Review these yourself.
- Least privilege matters most for apps that read mail or files. Scope them to the mailboxes they need.
- You can only detect what you log. Turn on audit logging and send service principal sign-ins somewhere you can query.
- Review app consents and credentials on a schedule, not only after an incident.
References
- The Hacker News, ConnectWise suspends Cloud Backup after attack (feed summary only; I did not read the full page)
- Microsoft Learn: Entra service principal sign-in logs
- Microsoft Learn: Microsoft Purview audit and
MailItemsAccessed - Microsoft Learn: Limiting application permissions to specific Exchange Online mailboxes
- MITRE ATT&CK: T1078.004, T1098.001, T1528, T1114.002