HyperDX walkthrough

HyperDX is the built-in observability UI for ClickStack — bundled with every ClickHouse Cloud service. This guide walks you through the four things you need to do in HyperDX for the migration lab:

1. Launch ClickStack from the Cloud console.
2. Create three data sources — one each for traces, logs, and OTEL metrics — pointing at the otel database you populated in Step 2.
3. Search live logs in the Search view to confirm data is flowing.
4. Build a chart with the AI assistant to demonstrate ad-hoc visualization without writing SQL.

Prerequisites: Step 4 of README.md passed cleanly — otel.otel_logs_v2, otel.otel_traces, and otel.otel_metrics_* are all receiving data, and you can confirm with bash scripts/validate_migration.sh.

---

Step A — Launch ClickStack

1. Sign in to clickhouse.cloud and open the service you provisioned in Step 1 of the README.
2. In the left sidebar of the SQL console, scroll to the bottom. Click the launch icon (↗) next to ClickStack (Beta).
3. ClickStack opens in a new tab and authenticates you via Cloud SSO — no separate login.

What you should see in the SQL console first: the otel database in the database picker (top centre) listing the 11 tables, 3 materialized views, and 2 dictionaries you created in Step 2 — otel_logs, otel_logs_v2, otel_traces, otel_metrics_* (5 tables), geoip_data + geoip_country/geoip_city (1 table + 2 dictionaries), alert_error_rate*, logs_summary_1min*. If any are missing, re-run clickhouse/dictionaries.sql, clickhouse/schema.sql, and clickhouse/alert-tables.sql before continuing.

---

Step B — Create the three data sources

HyperDX queries ClickHouse via named "sources" that bind a UI tab (Search, Service Map, Chart Explorer, ...) to a specific table and a column-mapping convention. You'll create one source per signal type.

Open Team Settings in the bottom-left sidebar, then Data → Sources. Click Add source to create each of the three below.

B.1 — Traces source

Click Save Source. The Trace, Default, otel.otel_traces summary should appear at the top of the Sources list.

B.2 — Logs source

Click Add source again and configure:

Why TimestampTime and not Timestamp? Timestamp is DateTime64(9) (nanoseconds). HyperDX's time-range picker and histogram bucketing default to DateTime precision; pointing it at the materialized TimestampTime column avoids implicit casts in every dashboard query.

B.3 — OTEL Metrics source

Click Add source again and configure:

The OTEL Metrics source spans all five metric tables because the OTel data model encodes different metric types into different rows/shapes. HyperDX dispatches to the right table based on the metric type you query. Setting Correlated Log Source = log lets HyperDX jump from a metric chart to the matching logs in one click.

After all three sources are saved, the Sources list should show Traces, log, and otel_metrics — same shape as the screenshot above (one collapsed entry per source).

---

Step C — Search live logs

Click Search in the left sidebar. At the top, select the log source you just created.

You should see:

• A histogram of event counts over time (use the time-range picker to set "Last 15 minutes" or "Last 1 hour")
• A table of log rows with columns Timestamp, service, level, body
• A facets sidebar on the left listing high-cardinality fields like ServiceName (with per-service counts) and SeverityText

Things to try:

• Filter to one service: click inventory-service (or any service) under the ServiceName facet — the table reloads filtered to that service in a fraction of a second.
• Search by string: type error in the top search bar. ClickHouse's text indexes (the text(tokenizer='sparseGrams') skip index on Body in schema.sql) accelerate this dramatically vs. a full scan.
• Inspect a row: click any log row to expand the structured view — every LogAttributes map key becomes a clickable filter.

Note on time-range: the start_at: end setting in the file-based collector means historical lines that existed before Step 3b are not in otel_logs_v2. If "Last 24 hours" looks sparse in the early hours, that's expected — the data starts when the collector did.

---

Step D — Build a chart with the AI assistant

HyperDX's AI Assistant (currently labelled Experimental) translates natural-language descriptions into chart configurations.

1. Click Chart Explorer in the left sidebar.

2. Click the AI Assistant [A] toggle at the top of the chart pane.

3. With the log source selected, type a natural-language prompt into the input box, for example:

   Error count by services for past 2 hours

4. Hit Enter. The assistant fills in the chart config below: chart type (Line/Bar), data source (log), aggregation (Count of Events), and a Where clause (SeverityText = 'ERROR').

5. The chart renders immediately. Adjust the time range or chart type with the tabs at the top (Line/Bar, Table, Number, Pie, Search, Markdown).

Save useful charts via the Chart Name field and the save icon — they appear under Saved Searches / Dashboards in the sidebar for later reuse.

Prompt library — recreate every Kibana panel from Part 1

The six Kibana dashboards from Part 1 contain 30 panels in total. The tables below give you a one-prompt-per-panel recipe so you can rebuild each chart in HyperDX. For each entry:

• Source — set this in the AI Assistant before sending the prompt (log for everything from otel_logs_v2, Traces for everything from otel_traces, otel_metrics for metrics charts).
• Chart — the HyperDX chart-type tab to select (or let the assistant pick) before saving. See the legend below.
• Filter — many Kibana panels were implicitly scoped by data stream (e.g., the Web Traffic dashboard only queried logs-web_access-lab). In ClickHouse you'd express that with a Where clause; the prompts below either include the filter inline or rely on HyperDX's source-level filter. If the AI Assistant doesn't pick up the filter, switch the chart to SQL mode and add the suggested Where expression.
• Field-name fallback — the AI Assistant maps natural-language terms to schema columns. If a prompt produces an empty chart, edit the generated Where / aggregation field manually using the column hints in the Field hint column.

HyperDX chart types — reference legend

The Chart Explorer pane has six tabs along the top (Line/Bar, Table, Number, Pie, Search, Markdown). The mapping between HyperDX chart types and the Kibana visualizations they replace:

Where's "Histogram"? Kibana has an explicit "Histogram" visualization (counts bucketed by a numeric or date field). HyperDX folds this into the Line/Bar tab — pick a date or numeric groupBy, choose bar as the shape, and you have a histogram. The standalone histogram strip you saw at the top of the Search view in Step C is rendered automatically and isn't a chart you build manually.

Key column-name differences from Kibana ECS: request_path → RequestPath (raw) or RequestPage (path-only, recommended for grouping); request_type → RequestType (HTTP method); status → StatusCode; geo.country_name → GeoCountry; user_agent_parsed.name → BrowserFamily; service / service.name → ServiceName; level → LogLevel or SeverityText; event.severity → SeverityText; hostname → HostName; event.outcome → derived from StatusCode (1xx–3xx success / 4xx–5xx failure on traces); transaction.duration.us (μs) → Duration (nanoseconds — divide by 1000 for μs).

Web Traffic Overview — 8 panels

Filter all of these to web access logs only: in the AI Assistant or the chart's Where field, add RequestType != ''.

Application Health — 5 panels

Application logs are the rows where the level is set but RequestType is empty (web access has both). Filter: LogLevel != '' AND RequestType = ''.

Infrastructure Overview — 4 panels

Infrastructure (syslog) rows have ServiceName starting with k8s- (the regex_parser promotes the syslog hostname to service.name). Filter: ServiceName LIKE 'k8s-%'.

OTel Demo — APM Traces — 6 panels

Traces flow into otel.otel_traces. Switch the AI Assistant data source to Traces before running these prompts.

OTel Demo — Latency — 4 panels

OTel Demo — Logs — 3 panels

OTel Demo log records land in otel.otel_logs_v2 from the OTel Demo services (frontend-proxy, cart, checkout, etc.). They have neither RequestType nor LogLevel materialized — distinguish them with ServiceName NOT LIKE 'k8s-%' AND RequestType = ''.

Tips when the AI Assistant gets it wrong

1. Switch to SQL mode in the Where selector. The AI Assistant generates a partially-filled chart; you can edit any field manually. The schema picker (Schema button next to the data source) shows every column with type info.
2. Pin the time range. The Assistant respects "for the past N hours / today / yesterday" but defaults to the chart's time-range picker if you omit it. Set the range explicitly in the picker before saving.
3. Map-typed columns (LogAttributes, ResourceAttributes, SpanAttributes) are common stumbling blocks for the assistant. If it generates WHERE process = 'kernel' instead of WHERE LogAttributes['process'] = 'kernel', fix it in SQL mode.
4. Save the working version with Chart Name filled in, then drop charts onto a HyperDX dashboard so you have a single-page view that mirrors the original Kibana dashboard.

---

What's next

• Service Map (sidebar, beta) — visualizes the OTel Demo's microservice call graph from otel_traces. No configuration needed beyond the Traces source.
• Alerts — define alerts on any saved search or chart. This is what you'll use in Step 8 of the main README; see README.md § Step 8 Option A for the two recommended alerts (web-5xx-errors, service heartbeat).
• Notebooks (preview) — combine charts, queries, and Markdown commentary into shareable investigations.

When you're done exploring, return to Step 6 of README.md to verify TTL is configured.

---

Troubleshooting

Sources list is empty after saving. HyperDX caches source definitions per browser session. Hard-reload the tab (Cmd-Shift-R / Ctrl-Shift-R). If still empty, check Team Settings → Data → Server Connection — it should be Default and resolve to the same CH Cloud service.

Search shows no rows but validate_migration.sh reports thousands.

• Check the time-range picker — default is "Last 15 minutes". Bump to "Last 24 hours" if your collector started recently.
• Confirm the Default Select for the source uses TimestampTime (not Timestamp). Symptom: the histogram appears but the table shows "no rows".
• Confirm the source's Database is otel (not default).

AI Assistant returns "I couldn't translate that." The assistant is constrained to the schema of the selected source. Common fixes:

• Switch data source: trace-shaped questions ("p95 latency") need the Traces source, not log.
• Be explicit about the field: count by ServiceName works better than count by service.