Error Handling

Overview

Errors can occur at two points in the system: during upstream sync and during client requests. Each is handled differently.

Upstream Sync Errors

Upstream API Failures

If any of the three upstream API calls fail (non-2xx status code), HttpRequestException is thrown and the entire sync aborts.

What happens:

• No version increment occurs.
• No data is written to MongoDB.
• The exception propagates to ASP.NET Core's default exception handler, which returns 500 Internal Server Error. In Development mode, the response includes a developer exception page (HTML). In Production mode, the response body is empty.
• Existing data in MongoDB remains untouched — clients continue to receive the last successfully synced data.

Recovery: The Kubernetes CronJob has backoffLimit: 3, so it will retry automatically. If retries are exhausted, the job is marked as failed and can be investigated via kubectl get jobs.

Partial Sync Failures

If upstream fetches succeed but a MongoDB write fails mid-sync:

• The version counter has already been incremented.
• Some raw FHIR or output records may have been upserted with the new version number.
• Soft-deletes may not have been applied.
• The SyncHistoryEntry for this run will not have been written, so the run is missing from SyncHistory.

Impact: Clients using ?sinceVersion=N may see some but not all changes from this sync run.

Recovery: Trigger a new sync. The next sync is idempotent — content-hash change detection means already-correct records are left alone, while inconsistent records are reconciled. No manual intervention is needed.

Timeout

The sync endpoint has no explicit timeout configured. The Kubernetes CronJob should set an activeDeadlineSeconds to prevent runaway syncs:

jobTemplate:
  spec:
    activeDeadlineSeconds: 300  # Kill after 5 minutes

Client Request Errors

Authentication Errors

The error message is intentionally identical for both cases to avoid leaking information about valid keys.

Invalid sinceVersion

MongoDB Connection Failures

If MongoDB is unreachable when a client makes a request:

• The controller throws an unhandled MongoConnectionException.
• ASP.NET Core returns 500 Internal Server Error.
• The error is logged at Error level.

Monitoring: Watch for 500 responses on the output endpoints. MongoDB connection issues typically resolve once the database is reachable again — no data loss occurs.

Logging

The sync service logs at Information level. Unhandled exceptions (upstream failures, MongoDB errors) are logged by the ASP.NET Core infrastructure.

Key Log Messages

# Sync lifecycle (SyncService)
"Starting sync"
"Sync completed in {Duration}ms: {Added} added, {Updated} updated, {Unchanged} unchanged, {Deleted} deleted (version {Version})"

Note: Items without identifiers are silently filtered out during conversion (no log message is emitted). Consider adding explicit logging for skipped items if observability is important.

For per-collection detail, query the SyncHistory collection — every sync run writes one entry there with full statistics and business keys for added/updated/deleted records.

Structured Logging

All log messages use structured logging with named parameters, making them queryable in log aggregation tools (e.g., Seq, Elasticsearch, Application Insights).

Monitoring Recommendations

Failure Modes Summary