Backfilling history

Backfilling gives new customer timelines useful history from the start. Import existing profiles and trustworthy historical events before relying on the live event stream. A safe backfill needs accurate timestamps, rate-limit pacing, a test run, and a clear restart strategy.

Decide what’s worth importing

Apply the same capture rules used for live traffic:

Profiles for current customers, including email. Consider excluding long-churned customers you will not contact; they count toward the plan’s profile limit, and messages for new customers beyond that limit are skipped.
Lifecycle history from your billing system: trials, subscriptions, upgrades, and cancellations. Billing exports work well because their timestamps are usually reliable.
Skip synthetic reconstructions. If the original timestamp is unavailable, do not guess.

Use real timestamps

Every method accepts a timestamp so events land at their true time:

await glyph.track(row.userId, "Invoice paid", { amount: row.amount }, {
  timestamp: new Date(row.paidAt),
});

from datetime import timezone

glyph.track(
    row.user_id,
    "Invoice paid",
    {"amount": row.amount},
    timestamp=row.paid_at.replace(tzinfo=timezone.utc),
)

Check these timestamp details before running the import:

Numbers are epoch milliseconds, not seconds. Epoch seconds parse as January 1970 without erroring. When in doubt, pass a native Date/datetime or an ISO 8601 string.
Python datetimes must be timezone-aware. A naive datetime is interpreted as the machine’s local time — construct with tzinfo set to the source’s real zone.

Send each customer’s profile before their events so the timeline has traits and email from the start. Events for a new userId still create the customer, and a later profile fills in the traits.

Work within the rate limit

Each write key accepts 600 messages per 60-second sliding window by default; some plans have a higher limit. The SDKs retry occasional 429 responses, but a sustained import should stay below the limit. A pace of 500 messages per minute leaves headroom: At that rate, 1,000 messages take about 2 minutes, 10,000 take about 20 minutes, and 100,000 take about 3.5 hours.

import { GlyphNode } from "@glyphhq/node";

const writeKey = process.env.GLYPH_WRITE_KEY;
if (!writeKey) {
  throw new Error("GLYPH_WRITE_KEY is not set");
}

const glyph = new GlyphNode({
  writeKey,
  onError: (error, messages, { status, retryable }) => {
    console.error(`delivery failed: status=${status} retryable=${retryable}`);
  },
});

const PACE = 500; // messages per minute, under the 600 budget
let sent = 0;

for (const row of rows) {
  await glyph.track(row.userId, row.event, row.properties, {
    timestamp: new Date(row.occurredAt),
  });

  sent += 1;
  if (sent % PACE === 0) {
    await glyph.flush();
    console.log(`${sent}/${rows.length}`);
    await new Promise((resolve) => setTimeout(resolve, 60_000));
  }
}

await glyph.close();

import os
import time

from glyphhq import Glyph

PACE = 500  # messages per minute, under the 600 budget


def on_error(error, messages, context):
    print(f"delivery failed: status={context.status} retryable={context.retryable}")


with Glyph(write_key=os.environ["GLYPH_WRITE_KEY"], on_error=on_error) as glyph:
    for sent, row in enumerate(rows, start=1):
        glyph.track(
            row.user_id,
            row.event,
            row.properties,
            timestamp=row.occurred_at,  # timezone-aware!
        )
        if sent % PACE == 0:
            glyph.flush()
            print(f"{sent}/{len(rows)}")
            time.sleep(60)
# close() runs on exit and flushes the tail

Always configure onError for an import. A non-retryable failure drops the batch, and the callback identifies the rows that need to be sent again.

Dry-run against the test environment

Run a representative sample with a glyph_test_... key before the live import. Do not send the full dataset because the test environment stops accepting requests after it reaches 5,000 stored events. Check timestamps, time zones, event names, and profile traits in the resulting timelines.

Make it re-runnable (or don’t run it twice)

The SDKs generate a fresh messageId per call, so re-running an import script duplicates every event. Pick one of:

Run it exactly once. Fine for a one-off import — keep the script’s output log as your record.
Track your progress. Record the last imported row (or mark rows as sent) so a crashed run resumes instead of restarting.
Use the HTTP API with deterministic message IDs. Derive messageId from your source data — import-invoice-1042 — and the server’s idempotency makes the whole import safely re-runnable:

curl https://in.glyphhq.io/api/v1/ingest \
  -H "Authorization: Bearer $GLYPH_WRITE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "track",
    "userId": "user_123",
    "messageId": "import-invoice-1042",
    "event": "Invoice paid",
    "properties": { "amount": 4900 },
    "timestamp": "2025-11-03T14:00:00.000Z"
  }'

Verify

After the import, check several known customers from different parts of the dataset. Backfilled events sort by timestamp, so older history appears below recent activity. Resolve gaps, incorrect ordering, or 1970 timestamps before importing another source. See Troubleshooting.

Getting started

SDKs

Guides

Reference

Backfilling history

Decide what’s worth importing

Use real timestamps

Work within the rate limit

Dry-run against the test environment

Make it re-runnable (or don’t run it twice)

Verify

​Decide what’s worth importing

​Use real timestamps

​Work within the rate limit

​Dry-run against the test environment

​Make it re-runnable (or don’t run it twice)

​Verify

Decide what’s worth importing

Use real timestamps

Work within the rate limit

Dry-run against the test environment

Make it re-runnable (or don’t run it twice)

Verify