Logging Is a First-Class Citizen: Building Good Logging Hygiene for Better Alerting and Observability

In this latest blog from the developers on our Product Traction team, they share the importance of building good logging hygiene for better alerting and observability.

As developers, we’ve all been there: production’s on fire, PagerDuty is screaming, and you’re wading through a sea of unstructured logs like “something went wrong” or worse—nothing at all. Good logging hygiene isn’t just about clean output. It’s the difference between a quick diagnosis and hours of digging. Done right, it lays the foundation for reliable alerting, effective debugging, and scalable system observability.

We’re putting this into practice across two separate backends—one in TypeScript using Pino for structured logging, and another in Go, where context propagation is key for distributed tracing. Both are being instrumented for full visibility in Grafana.

This post outlines how we’re architecting our logs for clarity, consistency, and operational sanity.

Logging Best Practices

Use the Right Log Levels

Log levels are your communication layer with observability tools. Get them right, and your alerts will be right too.

  • Trace: Request tracing across services and functions. Captures timing, payloads, and control flow. Always enabled in production.

  • Debug: For development or temporarily verbose logs. Use to capture internal states, branching decisions, retries, and performance benchmarks. Usually disabled in production unless debugging a specific issue.

  • Info: Standard operations. Log events like user signups, background job completions, and external API interactions.

  • Warn: Handled edge cases. An S3 file is missing but fallback logic kicks in? That’s a warning.

  • Error: Only for unexpected, critical failures. These require developer attention and should not be common.

Don’t Abuse Error Logs

Common mistakes include logging expected behavior as errors. Avoid this.

Examples of what should not be an error:

  • A 404 Not Found because a user asked for a missing record.

  • An “email already exists” check during registration.

  • A business rule like “user must accept terms before continuing.”

Examples of real errors:

  • Database insert failure due to a constraint that shouldn't have triggered.

  • External service timeout when no retry/fallback exists.

  • Unreachable code paths.

Log Message Architecture

If you're looking for one high-impact improvement to your logging strategy, this is it:

Every log message should be unique across the codebase.

Why?

Because at 3 AM when your logs say “failed to update resource”, and you find five different places with that message, you’re burning time you don’t have.

✅ Good Example

logger.error({ userId, orgId, err }, 'failed to update user status to verified after KYC completion')

This gives:

  • A searchable message that’s unique to one code path.

  • Structured metadata (userId, orgId).

  • A direct pointer to the failing logic.

Anti-patterns to avoid:

  • Logging errors without any identifying context.

  • Putting variable data in the log message string.

  • Reusing the same vague message in multiple places.

Instead, keep variable data in structured fields and your message text fixed and specific.

Error Handling Patterns

Database Errors

In both Go and TS backends, your database driver will throw a range of error types—learn them, and handle the known cases explicitly.

Go Example:

if errors.Is(err, sql.ErrNoRows) {
    return ErrNotFound
}

Everything else? That’s a 500, and it should be logged:

log.Error().Err(err).Msg("failed to fetch user by ID")

TypeScript Example:

if (error instanceof NotFoundError) {
  return res.status(404).json({ error: 'user not found' })
}

logger.error({ err: error }, 'unexpected error while fetching user')
return res.status(500).json({ error: 'internal error' })

Error Propagation

  • Log errors at the point of handling, not where they’re thrown.

  • Wrap errors with additional context:


    • In Go: fmt.Errorf("failed to start job: %w", err)

    • In TS: throw new Error('failed to start job: ' + err.message)

Constant Error Strings

When you use human-readable error strings (for UI messages or logs), treat them like code—don’t duplicate.

export const ERR_EMAIL_TAKEN = 'email is already in use'

Only use that constant in one place per code path. If another flow needs the same phrasing, make a second constant. This guarantees message uniqueness and helps trace issues faster.

Implementation Notes

TypeScript Backend

  • Logger: Pino, chosen for its performance and JSON-friendly output.

  • Log Levels:


    • error: unexpected failures

    • warn: unusual but handled behavior

    • info: standard operations

    • debug: internal decisions and verbose outputs, typically disabled in production

    • trace: request tracing for distributed observability

  • Traceability: All logs include request ID, user ID, and organization ID where applicable.

  • Observability: Logs are streamed to Grafana Loki and correlated with request traces via request IDs.

Go Backend

  • Context-aware logging: Every HTTP request generates a context.Context that carries metadata.

  • Each log includes:

    • request_id
    • user_id
    • org_id

  • Logs are structured and correlate to Prometheus/Grafana dashboards.

  • No logging in utility/helper functions—only at controller/service layer, to avoid duplicated or misleading logs.

  • Tracing is enabled for production to follow requests across services, components, and time boundaries.

Good logs do more than tell you what went wrong. They help you find where, why, and how it failed—without diving into the debugger.

By enforcing clear levels, structured metadata, and unique log messages, you’ll:

  • Accelerate debugging

  • Cut down on false alarms

  • Empower new devs to trace unfamiliar flows

And perhaps most importantly—you’ll sleep better when you’re on call.


The Thin Air Labs Product Traction team provides strategic product, design and development services for companies of all sizes, with a specific focus on team extensions where they seamlessly integrate into an existing team. Whether they are deployed as a team extension or as an independent unit, they always work with a Founder-First Mindset to ensure their clients receive the support they need.

To learn more about our Product Traction service, go here.

Build what's next with us