top of page

Data Observability

Ensuring Data Health, Quality & Trust at Scale

Alerts1.png

Overview

Data Observability is the ability to continuously monitor the health, quality, and reliability of data across pipelines, systems, and applications.


It helps teams detect when something goes wrong with their data — what broke, why it broke, and how to fix it — before it impacts dashboards, reports, or business decisions.​

​

Machine learning continuously monitors data patterns and promptly alerts users to anomalies.

The Problem

Data quality checks were manual and time-consuming

Pipelines broke silently - leading to inaccurate dashboards

No way to detect sudden drops in Data Volumn or Freshness

Lack of transparency into root cause

High congnitive load on Data Engineers

Profiling was reactive, not proactive

Goal

To address this, we set out to design a cloud-based Data Observability system that empowers users to:

​

  • Observe their critical data assets in real time.

  • Receive actionable alerts across Freshness, Volume, Schema Drift, and Data Drift.

  • Drill down into anomalies with clarity to resolve issues faster.

  • Evolve beyond profiling into a scalable, cloud-first observability platform.

My Role

  • Led the Data Observability MVP alongside a junior designer, driving end-to-end UX.

  • Planned and conducted stakeholder research 

  • Competitive analysis of Monte Carlo, Informatica/Collibra, and Metaplane

  • Defined Information Architecture, Interaction Design, Visual Design, and UX Writing

  • Supported product strategy across multiple releases (~1.5 years)​

Unserstanding the User

 Data Fixer - Frank

Role
Hands-on fixer who needs to investigate anomalies, identify root cause, and apply quick fixes.
Needs
Fast drill-downs, clear traces, quick run/debug of rules, and concise context for anomalies.
DI Suite_Personas.png

 Data Steward - Steve

Role
Hands-on fixer who needs to investigate anomalies, identify root cause, and apply quick fixes.
Needs
Fast drill-downs, clear traces, quick run/debug of rules, and concise context for anomalies.
Steve persona.jpg

Competitive Landscape

To understand the broader landscape of Data Observability solutions, I analyzed three leading platforms—Monte Carlo, BigEye and Acceldata.

 

This helped benchmark industry standards, identify UX gaps, and inform opportunities for a more intuitive, scalable, and user-centric observability experience.

CA 1.png
CA2.png
Observability List.png

Information Architecture

IA.png

Customer Journey Map

A step-by-step view of how users detect, investigate, and resolve data issues.

 

Journey map_edited.png

User Journey Map

Created a detailed user journey of the observer creation process that provides a comprehensive overview of the entire observer creation flow. This process focuses on a step-by-step process that users will go through, giving a clear understanding of how to create an observer within the platform.

UJ2.png

Observer Creation Flow

  • A Miller Column approach for asset selection aligns with users’ mental model for scoping data assets.​

  • Built-in search, visible asset counts, and disabled states prevent duplicate monitoring

Create Observer 1_edited.jpg

  • Smart default rules for freshness, volume, data drift and schema drift

  • Side-panel for advanced configurations​​

Create Observer 2_edited.jpg
Edit - Observer Rules.png

  • Mandatory scheduling (hourly/daily/weekly/monthly)

  • Notification recipient management with validation

Create Observer 3_edited.png

The Solutions

Observer List Page

​

  • Compact card/list hybrid layout

  • Scan-first design showing:

    • Rule count

    • Schema changes

    • Profiling status

    • Last run & next run

  • Bulk actions

  • Status badges (green/amber/red) for clarity

  • ​Users can quickly assess the health of hundreds of observers without drilling into each one.

Alerts & Anomaly Detection

​

  • KPI rings summarizing alerts by severity and reducing time-to-triage 

  • The layout (top KPIs → alert list → alert detail) lets users go from overview to detailed diagnosis without switching screens.

  • Hover interactions for exact values

  • Contextual tags (Freshness, Volume, Schema, Distribution)

  • “Investigate deeper” drill-downs for root cause analysis

  • Expandable graphs for side-by-side comparison

alerts list .png

Data Visualization

  • Each alert shows a small time-chart in the details panel so users can see trends and exact values quickly.

  • Simple interactions (hover for values, pick a time range, and quick action links) make it easy to investigate and act on alerts.

Freshness alert.png
volumn alert.png

Expand Chart & Modify Selection

​

​​

Data Visualization expanded.png

Graph visualization across multiple time ranges

Graph visualizations.png

Seamless Onboarding Experience

With Alerts and Observers introduced, the goal was to ensure users could learn and adapt effortlessly.

Business Impact

Strengthens the positioning of the Precisely Data Integrity Suite

Enables enterprise-scale proactive data monitoring

Supports governance, auditability, and compliance

bottom of page