Plotly.js Embedded Analytics in a FastAPI Portal

The analytics portal needed subscriber-level dashboards — churn risk scores, signal quality, pipeline health. Grafana was the obvious choice. I didn’t use it.

Grafana makes sense for ops teams: time-series telemetry, alerting, shared visibility. For a user-facing product portal it means a separate service to run, authenticate, and style. Plotly.js lives inside the same FastAPI app, renders charts client-side from a JSON endpoint, and lets me write the HTML directly. The tradeoff is losing Grafana’s query tooling, but the portal doesn’t need it — the data comes pre-aggregated from the DuckDB Gold layer.

Architecture

Two FastAPI endpoints feed the dashboard: /api/v1/analytics/summary for Gold layer KPIs and /api/v1/analytics/quality for pipeline health. Both query DuckDB in milliseconds against Parquet files. NaN and Infinity values get stripped server-side before JSON serialization — DuckDB returns them from AVG over empty sets and Python’s json.dumps() fails on them.

The template is static Jinja2 HTML with inline Plotly and custom CSS. No JavaScript framework. Chart divs start empty; a fetch() on load populates them client-side. All table rows are built with createElement() and textContent — never innerHTML.

KPI Cards

KPI cards needed visual hierarchy without extra markup. I used CSS pseudo-elements for the top accent bars:

.kpi-card {
    border-radius: 10px; padding: 20px;
    display: flex; flex-direction: column; gap: 4px;
    position: relative;
}
.kpi-card::before {
    content: ''; position: absolute; top: 0; left: 0; right: 0; height: 3px;
    border-radius: 10px 10px 0 0;
}
.kpi-total      { background: #132d1f; }
.kpi-total::before { background: #37872D; }
.kpi-atrisk     { background: #2d1319; }
.kpi-atrisk::before { background: #C4162A; }

Each card variant gets a semantic background and a matching accent bar. The six cards grid at 6 columns on desktop, 3 on tablet, 2 on mobile, 1 on phones — the grid-template-columns rule cascades across breakpoints so each only overrides what changes.

position: absolute; top: 0; left: 0; right: 0; height: 3px lets the bar bleed edge-to-edge under the rounded corner without a visible div.

Info Tooltips and the Overflow Clipping Trap

Context tooltips appear on hover over i icons in card labels. Each tooltip explains what the KPI measures:

<div class="kpi-label">
  Total Subscribers
  <i class="info-icon">i
    <span class="tooltip">
      Unique subscriber profiles in the Gold feature layer. Each profile is built by aggregating all BSS, OSS, and IoT events for a single subscriber...
    </span>
  </i>
</div>

.info-icon .tooltip {
    visibility: hidden; opacity: 0;
    position: absolute; top: calc(100% + 10px); left: 50%;
    transform: translateX(-50%); width: 260px;
    background: #272b3d; border: 1px solid #3d4466;
    border-radius: 8px; padding: 12px 14px;
    box-shadow: 0 8px 24px rgba(0,0,0,0.5);
    z-index: 1000; pointer-events: none;
}
.info-icon:hover .tooltip { visibility: visible; opacity: 1; }

The trap: any ancestor with overflow: hidden clips the tooltip. I hit this on mobile when I added overflow: hidden to .kpi-card to tighten the boundaries. The fix is keeping overflow: visible on everything above the tooltip in the DOM — in this layout that meant leaving .kpi-card with visible overflow, which was fine since the only overflow: hidden is at the table level below the cards.

Funnel and Donut Charts

The medallion pipeline data flows Bronze → Silver → Gold. I render this as a horizontal bar chart (Plotly calls it “funnel-like”):

Plotly.newPlot('chart-funnel', [{
    type: 'bar',
    x: [pipeline.bronze_records, pipeline.silver_records, pipeline.gold_subscribers],
    y: ['Bronze (Raw)', 'Silver (Cleaned)', 'Gold (Features)'],
    orientation: 'h',
    marker: { color: ['#56A64B', '#3274D9', '#FF9830'], line: { width: 0 } },
    text: [...].map(v => v.toLocaleString()),
    textposition: 'inside',
    textfont: { color: '#fff', size: 14, family: 'system-ui' },
    hovertemplate: '%{y}: %{x:,}<extra></extra>'
}], {
    paper_bgcolor: 'transparent',
    plot_bgcolor: 'transparent',
    font: { color: '#a0aec0', family: 'system-ui' },
    margin: { l: 120, r: 30, t: 10, b: 30 },
    xaxis: { showgrid: true, gridcolor: '#1e2235', zeroline: false },
    yaxis: { autorange: 'reversed' },
    bargap: 0.35
}, { displayModeBar: false, responsive: true });

Event source distribution (BSS, OSS, IoT) uses a donut:

Plotly.newPlot('chart-donut', [{
    type: 'pie',
    labels: labels.map(l => labelMap[l] || l),
    values: values,
    hole: 0.55,
    marker: { colors: ['#FF6D00', '#3274D9', '#8AB8FF'], 
              line: { color: '#1a1d2e', width: 2 } },
    textinfo: 'label+percent',
    textposition: 'outside',
    hovertemplate: '%{label}<br>%{value:,} events<br>%{percent}<extra></extra>',
    pull: [0.02, 0.02, 0.02]
}], { ... });

displayModeBar: false removes the Plotly toolbar. responsive: true lets charts resize with the window. Transparent backgrounds blend with the dark theme. Grid lines at #1e2235 match the sidebar color.

DOM-Based Table Rendering

The subscriber table carries risk scores and signal quality from the API. Every cell is built with createElement() and textContent:

function createRow(r) {
    const tr = document.createElement('tr');

    const tdId = document.createElement('td');
    tdId.className = 'sub-id';
    tdId.textContent = r.subscriber_id;  // textContent, not innerHTML
    tr.appendChild(tdId);

    const tdEvents = document.createElement('td');
    tdEvents.textContent = r.total_events;
    tr.appendChild(tdEvents);

    const tdSignal = document.createElement('td');
    tdSignal.className = signalClass(r.avg_signal_dbm);
    tdSignal.textContent = r.avg_signal_dbm != null ? r.avg_signal_dbm.toFixed(1) : '--';
    tr.appendChild(tdSignal);

    tr.appendChild(createLatencyCell(r.avg_latency_ms));
    return tr;
}

Latency gets a mini inline bar:

function createLatencyCell(ms) {
    const td = document.createElement('td');
    if (ms == null) { td.textContent = '--'; return td; }

    const wrapper = document.createElement('div');
    wrapper.className = 'bar-cell';

    const num = document.createElement('span');
    num.textContent = Math.round(ms);
    wrapper.appendChild(num);

    const track = document.createElement('div');
    track.className = 'bar-track';
    const fill = document.createElement('div');
    fill.className = 'bar-fill';
    const pct = Math.min(ms / 200 * 100, 100);
    fill.style.width = pct + '%';
    fill.style.backgroundColor = ms < 100 ? '#3fb950' : ms < 150 ? '#e3b341' : '#f85149';
    track.appendChild(fill);
    wrapper.appendChild(track);

    td.appendChild(wrapper);
    return td;
}

There’s no way to inject HTML through subscriber data. Signal, latency, and billing columns all use plain text or pre-calculated style colors.

15-Second Refresh Loop

const REFRESH_INTERVAL = 15000;
let countdown = 15;

function updateRefreshText(ts) {
    const el = document.getElementById('refresh-info');
    while (el.childNodes.length > 1) el.removeChild(el.lastChild);
    const text = document.createTextNode('LIVE · ' + ts + ' · next in ' + countdown + 's');
    el.appendChild(text);
}

refresh();
setInterval(refresh, REFRESH_INTERVAL);
setInterval(function() {
    if (countdown > 0) countdown--;
    const el = document.getElementById('refresh-info');
    const lastText = el.lastChild;
    if (lastText && lastText.nodeType === 3) {
        lastText.textContent = lastText.textContent.replace(/next in \d+s/, 'next in ' + countdown + 's');
    }
}, 1000);

The countdown ticks per second and resets on each refresh. The live-dot pulse is CSS @keyframes pulse on a small indicator in the topbar.

NaN and Infinity Handling

DuckDB returns NaN from AVG over an empty set and Infinity from division by zero. Python’s json.dumps() fails on both. I strip them server-side after fillna():

def list_subscribers(limit: int = 20):
    con = get_db()
    rows = con.execute(q).fetchdf()
    rows = rows.fillna(0)
    records = rows.to_dict(orient="records")
    for rec in records:
        for k, v in rec.items():
            if isinstance(v, float) and (np.isnan(v) or np.isinf(v)):
                rec[k] = 0.0
    return records

Fixing it server-side is less surprising than relying on Plotly’s handling.

Data Quality Page

A second dashboard shows pipeline health. The quality endpoint queries the Bronze layer for completeness:

@app.get("/api/v1/analytics/quality")
def data_quality_report():
    con = get_db()
    quality_dist = con.execute("""
        SELECT
            COUNT(*) FILTER (WHERE quality_score >= 80) AS excellent,
            COUNT(*) FILTER (WHERE quality_score >= 60 AND quality_score < 80) AS good,
            COUNT(*) FILTER (WHERE quality_score >= 45 AND quality_score < 60) AS fair,
            COUNT(*) FILTER (WHERE quality_score < 45) AS poor
        FROM bronze_raw_events
    """).fetchone()
    
    by_type = con.execute("""
        SELECT event_type,
               COUNT(*) AS total,
               ROUND(AVG(quality_score), 1) AS avg_quality
        FROM bronze_raw_events
        GROUP BY event_type
    """).fetchall()
    
    return {
        "pipeline": {...},
        "quality_scores": {
            "excellent_gte80": quality_dist[0],
            "good_60_79": quality_dist[1],
            "fair_45_59": quality_dist[2],
            "poor_lt45": quality_dist[3],
        },
        "by_event_type": [
            {"type": r[0], "count": r[1], "avg_quality": r[2]} 
            for r in by_type
        ],
        "completeness": {
            "subscriber_id": null_checks[0],
            "event_timestamp": null_checks[1],
            "raw_payload": null_checks[2],
        },
    }

The template renders a bar chart for quality score distribution and a source-wise breakdown. A field completeness checklist below that uses color coding: green for > 99%, yellow for 95–99%, red for < 95%. Events scoring below 45 are quarantined at the Silver layer — this page makes that visible.

Responsive Breakpoints

@media (max-width: 1024px) {
    .kpi-row { grid-template-columns: repeat(3, 1fr); }
    .chart-row { grid-template-columns: 1fr; }
}
@media (max-width: 768px) {
    .topbar { padding: 12px 16px; flex-wrap: wrap; gap: 8px; }
    .kpi-row { grid-template-columns: repeat(2, 1fr); gap: 10px; }
}
@media (max-width: 480px) {
    .kpi-row { grid-template-columns: 1fr; }
}

Four breakpoints: > 1024px (6-column KPIs, 2-column charts), 768–1024px (3-column KPIs, 1-column charts), 480–768px (2-column KPIs, wrapping topbar), < 480px (single column). Each breakpoint only overrides what changes — the grid-template-columns rule cascades from the previous.

What I’d Change

Plotly.js is 3.5MB. It should be preloaded earlier or served locally — it’s the biggest bottleneck on first paint. The subscriber table tops out at 50 rows; at 1000+ the DOM build slows and needs pagination. The chart config for the dashboard and quality pages is duplicated and should be pulled into a shared config object.

The Grafana call held up. Not because Grafana is bad — it’s the right tool for ops. For a product portal where the data is pre-aggregated and the UI needs to match the application, embedding Plotly.js in Jinja2 was faster to build, costs nothing to run, and is the only approach that gives full control over the HTML.