Monitoring and Alerting: What Should Page Someone at 2 AM

Every IT environment generates alerts: disks filling up, backups failing, devices going offline, internet circuits flapping. The danger is not too little monitoring. It is monitoring that treats every alert the same. When everything is urgent, people start ignoring all of it, and the one alert that mattered gets lost among the noise.

The answer is to decide in advance what is worth waking someone up for, what can wait until morning, and what is just information. That decision is a business decision as much as a technical one, and owners and office managers should be part of it.

Start With Business Impact, Not Technology

The question to ask about any alert is simple: if this goes unattended for the next eight hours, what happens to the organization?

For a healthcare or senior-living operator, that might mean:

Can staff still reach the electronic health record to document care or review medication information?

Do nurse call, door access and phones still work?

Can the building still reach the internet for eFax, pharmacy communication and email?

Are backups still running so data can be recovered?

An alert that threatens any of the first three should be treated very differently from a notice that a workstation has been offline since Friday.

Build Three Simple Tiers

Three tiers are enough for most small and mid-size organizations. More than that and nobody remembers the rules.

Tier 1: Page Immediately, Day or Night

These are conditions that stop care or operations, or signal an active attack.

A site-wide internet or network outage.

Core systems unreachable, such as the EHR connection, the main server, or the phone system.

Signs of active ransomware, such as mass file changes or security tools reporting malicious activity.

Failure of a critical safety-related system the facility depends on.

Tier 1 alerts should reach a real person by phone or text, with an escalation to a second person if there is no acknowledgment in a set number of minutes.

Tier 2: Handle Next Business Morning

These matter, but a short delay does no harm.

A backup job that failed once.

A disk at a high-but-not-critical level.

A single server or switch showing errors while a redundant path still works.

A repeated failed-login pattern for one account.

Send these to a ticket queue or email summary that someone reviews at the start of each workday.

Tier 3: Informational

Patch status, normal restarts and routine reports belong in a dashboard or weekly report. They should never interrupt anyone.

Set On-Call Expectations in Writing

Alert tiers only work if people know what is expected of them.

Who is on call and when. Publish a schedule, with a named backup.

Response time. For example, acknowledge within 15 minutes and begin work within an hour. Pick numbers you can really deliver.

Escalation path. Who is called if the first person does not answer, and who at the organization is told when a Tier 1 event lasts beyond a set time.

Authority. What the on-call person may do without asking, such as restarting a service, failing over a circuit or calling a vendor.

Fairness. On-call duty is real work. Rotate it, compensate it, and respect rest after a rough night.

Tune Relentlessly

The first version of any alert policy will be wrong. After each month, review the alerts that fired.

Alerts that fired often and required no action should be downgraded or removed.

Alerts that fired late, after users had already called, should be tightened.

Every night-time page should get a short review: was it really worth waking someone up? If not, change the rule.

Aim for a state in which a Tier 1 page is rare and always means something.

Keep Staff Reporting Paths Clear

Monitoring will never catch everything. Tell nurses and administrators exactly how to reach someone after hours, and make that number part of the downtime procedures posted at nursing stations. A person who calls with a problem should reach a human, not a voicemail box that is checked on Monday.

Working With a Partner

UnityCare IT helps healthcare and senior-living organizations define alert tiers, build after-hours escalation and keep the noise down so that real problems get fast attention. If your team is tired of false alarms, or worried that real ones are hidden among them, we are glad to review your current setup.

Related service

An outsourced IT department with proactive maintenance and one number to call.

Related articles

Keep reading

Contact UnityCare Technologies

Call or text: 405-285-3845

New customers: start@unitycareit.com

Existing customers: support@unitycareit.com

Address: UnityCare Technologies, 2524 N Broadway Ste 554, PMB 947974, Edmond, Oklahoma 73034-4172