Tool-belt series · Microsoft Fabric · Streaming & KQL
Real-Time Intelligence is the half of Fabric that Power BI people skip — which is exactly why it's worth your weekend. It's a different query language, a different storage engine, and a genuinely different way of thinking: instead of a dashboard someone remembers to open, a rule that messages them. KQL is also, unexpectedly, the most enjoyable query language you'll learn this year.
Nearly everyone in a finance or analytics group knows SQL and DAX. Almost nobody knows KQL, and almost nobody has built an alert that fires in seconds. That gap is not because it's hard — KQL is easier than SQL — it's because nobody in that world has had a reason to look at it yet.
You don't need IoT sensors to justify it. Failed payment runs, integration errors, approval queues, refresh failures, threshold breaches on a live balance — all of it is event data, and all of it is currently discovered by someone noticing on a Thursday.
Batch analytics answers "what happened last month". Real-time answers "something is happening now, and here's who needs to know". Different questions, different machinery.
Real-Time Hub sits across all of it as the catalogue of what's streaming in your tenant.
Same trick as the rest of Fabric: a handful of items, each an engine pointed at data. Learn what each one is for and the menu stops being intimidating.
A no-code pipeline for events. Connect a source, optionally filter/aggregate/route in flight, and send it to a destination — usually an Eventhouse, but it can also land in a lakehouse.
Think of it as: Data Factory for things that never stop arriving.
A columnar store descended from Azure Data Explorer, built for append-heavy, time-stamped data. Automatic indexing on time, retention policies, and it stays fast at volumes that would make a warehouse sweat.
Think of it as: the right home for anything with a timestamp and a lot of rows.
Native KQL-backed dashboards that auto-refresh on a live tile cadence, built for operational watching rather than monthly reporting. Point one straight at an Eventhouse table — no extra pipeline needed.
Think of it as: a wall screen, not a board pack.
Rules over streams, KQL results or Power BI visuals. When a condition holds, it fires — Teams message, email, or a Power Automate flow that does something.
Think of it as: the part that makes all the rest worth building.
One idea carries the whole language: start with a table, then pipe it through steps that each change it a bit. Top to bottom, in the order things happen — unlike SQL, where you write SELECT first and it executes almost last.
FailedPayments
| where Timestamp > ago(1h) // filter first — always
| where Amount > 10000
| summarize Failures = count() by Vendor // group
| order by Failures desc
| take 10Read it as a sentence: take FailedPayments, keep the last hour, keep the big ones, count by vendor, sort, show ten. Each line takes a table in and hands a table out. That's the entire mental model — and it's why KQL is easier to compose than SQL: you build it up one line at a time and can run it after every line to see what you've got.
where on the timestamp should be your first line in essentially every query. These tables hold billions of rows; the time filter is what keeps it instant — and it's the single biggest driver of both speed and cost.| Operator | What it does | SQL equivalent |
|---|---|---|
where | Keep matching rows | WHERE |
project | Pick / rename columns | SELECT |
extend | Add a calculated column | SELECT ..., x AS y |
summarize | Aggregate by something | GROUP BY |
order by | Sort | ORDER BY |
take / top | Limit rows | TOP / LIMIT |
join | Combine tables | JOIN |
render | Draw a chart, instantly | no equivalent — and it's lovely |
bin() rounds timestamps into buckets, which is how every time-series chart gets built. summarize count() by bin(Timestamp, 5m) gives you a count per five minutes. Add | render timechart and you have a chart, from a query, in one line. There's genuinely nothing this pleasant in SQL.// 1. what's happened in the last hour, by type
Events
| where Timestamp > ago(1h)
| summarize count() by EventType
| order by count_ desc
// 2. a chart of activity per 5 minutes
Events
| where Timestamp > ago(6h)
| summarize count() by bin(Timestamp, 5m)
| render timechart
// 3. is today unusual? compare to the same hour yesterday
Events
| where Timestamp > ago(25h)
| extend Period = iff(Timestamp > ago(1h), "now", "yesterday")
| summarize count() by Period, EventType
// 4. find the spikes — anomaly detection, built in
Events
| where Timestamp > ago(7d)
| summarize Cnt = count() by bin(Timestamp, 1h)
| extend (anomalies, score, baseline) = series_decompose_anomalies(Cnt)
// 5. latest status per entity — the dedup pattern, KQL-style
JobRuns
| summarize arg_max(Timestamp, *) by JobNamearg_max is the one to remember"Give me the whole row where this column is highest, per group." It's pattern 01 from the SQL page — ROW_NUMBER, PARTITION BY, filter to 1 — in a single readable line. Once you've used it you'll resent SQL slightly.Everything above is plumbing. This is the bit people actually notice, and it's the smallest amount of work on the page.
Critically, Activator can watch per entity, not just in aggregate — alert on each vendor that breaches, rather than on the total. That distinction is what separates a useful alert from a noisy one.
Most people alert on bad values. The more valuable alert is on absence — the feed that went quiet, the nightly job that didn't report, the queue that stopped moving. Silence is the failure mode that reaches a board pack, because nothing looks wrong until someone asks why a number hasn't moved.
It's also a nice piece of KQL: count the events in the recent window and alert when the count is zero, rather than when a value is high.
// no heartbeat in 30 minutes = something is wrong
Heartbeats
| where Timestamp > ago(30m)
| summarize LastSeen = max(Timestamp) by SourceSystem
| where isnull(LastSeen) or LastSeen < ago(30m)You don't have a factory floor. You do have plenty of event data hiding in plain sight — and the first one should take an afternoon, not a quarter.
The safest possible starting point: nobody else is affected if you get it wrong, and you're the customer. Stream your pipeline run outcomes into an Eventhouse, then alert on failures — and on the sneakier one, a success with suspiciously few rows.
A balance, a queue length, an exposure, an exception count. Somebody opens a report each morning to look at it — replace that with a message that only arrives when it matters. Instantly popular.
Pick the feed whose absence would be most expensive and alert on it going quiet. This is the one that eventually saves someone's month-end, and it's the one nobody else thought to build.
Don't study it separately. Every one of the above needs a query; write them, break them, run them line by line. A week of that and you'll be the KQL person — which, in a finance organisation, means the only one.