Skip to content
T-Chat Docs

Optimize Send Message Latency

Context

The send-message endpoint (functions/v1/api/v1/messages) is slow. Tracing the handler shows the dominant cost is synchronous moderation against OpenAI, which blocks the message INSERT and the 201 response:

Critical path today (supabase/functions/api/index.ts:2350-2451):

  1. Promise.all([isConversationMember, loadModerationConfig]) — ~1 DB roundtrip (config cached 60s).
  2. moderateText — blocks on omni-moderation-latest (~200-500ms) and, for suspicious text, a gpt-4o-mini contact-evasion call (~500ms-1.5s). moderation.ts:148-258, 285-350.
  3. resolveSendExtras — validateMentions then validateReplyTarget run sequentially (up to 3 DB roundtrips that don’t depend on each other). messages.ts:356-407, 238-297.
  4. INSERT message.
  5. endUserActorLabel — a DB roundtrip that blocks the response even though it only feeds the fire-and-forget audit log (index.ts:2439).

Push and audit are already deferred (void / EdgeRuntime.waitUntil).

Decision (from the user): move moderation off the response path (optimistic insert + async moderate), plus apply the safe structural overlaps, across the text path, attachment path, and the PATCH edit handler.

Key architectural fact that shapes the design: recipient delivery is broadcast-driven, not RLS/postgres_changes-driven. The sender’s client broadcasts MESSAGE_NEW_EVENT after it receives the 201 (useMessages.ts:749), and recipients render from that broadcast. History/reconnect uses an RLS-filtered SELECT. So to insert optimistically without leaking unmoderated content to recipients we must (a) gate the broadcast on moderation passing, and (b) hide non-approved messages from non-senders in the RLS SELECT policy (history path). The server already has a broadcast helper (broadcastMessageNew, system.ts:101) used for system messages — reuse it.

Intended outcome: the 201 returns in ~tens of ms (membership + cheap validation + INSERT only); moderation, broadcast, and audit happen in the background; recipients still never see unmoderated content; the sender sees their own message immediately as “sending” and it either confirms or is retracted with a reason.

Approach

1. Migration — moderation status + RLS (new supabase/migrations/<ts>_messages_moderation_status.sql)

  • Add to public.messages:
    • moderation_status text not null default 'approved' check (moderation_status in ('pending','approved','rejected'))
    • moderation_reason text, moderation_detail text
    • Default 'approved' keeps all existing rows and non-moderated sends visible with no backfill.
  • Replace the SELECT policy messages_select_same_tenant (20260608130000_phase2_messaging.sql:145-150) so non-senders only see approved messages: 
    using (
      tenant_id = ((auth.jwt() -> 'app_metadata' ->> 'tenant_id')::uuid)
      and deleted_at is null
      and (moderation_status = 'approved' or sender_id = auth.uid())
    )
  • No realtime-publication or replica identity change needed (delivery is via broadcast, and the Edge Function updates status with the service-role client, which bypasses RLS).
  • Then run pnpm db:types to regenerate packages/contracts/src/database.types.ts.

2. Contracts (packages/contracts/src/schemas.ts)

  • Add messageModerationStatusSchema = z.enum(["pending","approved","rejected"]) (export inferred MessageModerationStatus).
  • Add moderationStatus: messageModerationStatusSchema to messageResponseSchema (line 242).
  • Add a messageRejectedEventSchema ({ messageId, localId, reason?, detail? }) for the new rejection broadcast payload. Export from src/index.ts, then pnpm --filter @tesark/contracts build.

3. Edge Function (supabase/functions/api)

index.ts — formatting & shared helper

  • MESSAGE_SELECT (line 689): add moderation_status, moderation_reason, moderation_detail.
  • formatMessage (line 692): include moderationStatus: row.moderation_status.
  • Reuse a background scheduler: export the existing scheduleAuditWrite pattern from audit.ts:143 as a generic scheduleBackground(promise) (rename/duplicate-minimally) so the send handler can defer moderation+broadcast+audit via EdgeRuntime.waitUntil.

messages.ts — parallelize validation

  • In resolveSendExtras (line 356), run validateMentions and validateReplyTarget with Promise.all instead of sequentially (they’re independent). Preserve current error precedence (mention error before reply error) when both fail.

index.ts — POST /v1/messages text path (lines 2350-2451) and attachment path (2196-2348)

  • Compute needsModeration = hasText && (config.harm_enabled || config.profanity_enabled || config.contact_id_enabled).
  • Keep synchronous: membership check, resolveMentionSlicesForModeration, resolveSendExtras (these are cheap DB/CPU and produce real 400s the sender should get inline). Remove the inline await moderateText.
  • INSERT with moderation_status: needsModeration ? 'pending' : 'approved'. Return 201 immediately with the formatted message (now carrying moderationStatus).
  • Schedule background work via scheduleBackground:
    • If needsModeration: run moderateText.
      • Passed → UPDATE messages SET moderation_status='approved'; then broadcastMessageNew(supabase, conversationId, formatted-with-approved).
      • Failed → UPDATE messages SET moderation_status='rejected', moderation_reason, moderation_detail, deleted_at=now(); broadcast a new MESSAGE_REJECTED_EVENT (system.ts) with { messageId, localId, reason, detail }.
    • If not needsModeration: broadcastMessageNew immediately (still in background).
    • In both cases, do the audit write here, looking up endUserActorLabel inside the background task (removes it from the response path).
  • Apply the same structure to the attachment path: caption present → pending + async moderate; no caption → approved + broadcast.

system.ts

  • Add MESSAGE_REJECTED_EVENT = "message:rejected" and a broadcastMessageRejected(...) mirroring broadcastMessageNew (line 101). Reuse broadcastMessageNew for the approved case.

4. SDK (packages/chat-react)

message-delivery.ts — add a "moderating" state to MessageDeliveryState ({ state: "moderating" }) so the UI can show “sending…” while a message is pending; keep pending/failed as-is.

message-utils.ts — add export const MESSAGE_REJECTED_EVENT = "message:rejected"; (beside line 192).

useMessages.ts — sendMessage (lines 699-785)

  • On 201: upsert the returned message; set delivery to "moderating" if message.moderationStatus === "pending", else clearDelivery.
  • Stop the client-side broadcast(MESSAGE_NEW_EVENT, message) for text send (line 749) — the server now broadcasts on approval. (Same removal in the attachment send path, line 982.)
  • MESSAGE_NEW_EVENT handler (line 640): when incoming.senderId === userId, also clearDelivery(incoming.localId) so the sender’s “moderating” echo confirms on approval.
  • Add a MESSAGE_REJECTED_EVENT listener: match by localId → set delivery "failed" with reason/detail and removeMessage the optimistic row (mirrors current 422 handling at lines 752-770).

Critical files

  • supabase/migrations/<ts>_messages_moderation_status.sql (new)
  • packages/contracts/src/schemas.ts, src/index.ts
  • supabase/functions/api/index.ts (MESSAGE_SELECT, formatMessage, POST text + attachment paths)
  • supabase/functions/api/messages.ts (resolveSendExtras parallelization)
  • supabase/functions/api/system.ts (rejection broadcast)
  • supabase/functions/api/audit.ts (expose generic background scheduler)
  • packages/chat-react/src/useMessages.ts, message-delivery.ts, message-utils.ts

Edit (PATCH /v1/messages/:messageId) — secondary

The edit path (index.ts:2454-2600) already-approved content is visible, so async re-moderation must snapshot the pre-edit content and revert (restore content + re-broadcast MESSAGE_UPDATED_EVENT) on rejection rather than soft-delete. Apply the structural wins here regardless (parallelize mention validation, defer endUserActorLabel into the background audit). Recommend: keep edit moderation synchronous initially OR ship the async+revert variant only if the added revert complexity is wanted — flag at implementation.

Verification

  1. Build: pnpm build (contracts → chat-react → dashboard) and pnpm --filter @tesark/contracts build.
  2. Apply migration to remote (sknjeespbkyxzgddgfcn) via Supabase MCP apply_migration, then pnpm db:types and confirm moderation_status appears in database.types.ts.
  3. Local Edge Function: pnpm exec supabase functions serve api --no-verify-jwt.
    • Send a clean message → assert 201 returns in tens of ms (no OpenAI roundtrip on the response; time it with curl -w "%{time_total}"), moderationStatus: "pending".
    • Confirm a second (recipient) realtime client receives MESSAGE_NEW_EVENT only after moderation approves, and that a recipient history SELECT does not return a still-pending message (RLS check).
    • Send content that trips harm/contact moderation → message inserts then is retracted; sender receives MESSAGE_REJECTED_EVENT with reason/detail; recipients never receive a broadcast; the row ends moderation_status='rejected', deleted_at set.
  4. SDK: in the dashboard (pnpm dev), send a message and confirm the bubble shows “sending…” (moderating) then confirms; send a blocked message and confirm it flips to failed with the moderation reason and disappears for the recipient.
  5. pnpm test (chat-react vitest — add/adjust tests for the new moderating delivery state and the MESSAGE_REJECTED_EVENT handler in useMessages/message-delivery) and pnpm lint.