Request Lifecycle
→ Client submits form with input
→ Validate input against Zod schema
→ Hash input to generate cache key
→ Check in-memory cache
✓ HIT: Return cached result
✗ MISS: Continue to AI call
→ Generate prompt from template
→ Call Orchestrator.callAI()
→ Estimate tokens & cost
→ Attempt API call with retry
→ Parse and validate response
→ Cache result (24h TTL)
→ Log event with metrics
→ Return response + metadata
→ Client displays result
Architecture Layers
Presentation Layer
React components for form input and result display
components/
ExperimentForm.tsx - Form generation & validation
ResultDisplay.tsx -Result visualization
ExperimentPage.tsx -Page orchestration
API & Route Handlers
Next.js 16 route handlers with server-side processing
app/api/experiments/[slug]/route.ts
→ Validate input
→ Check cache
→ Call orchestrator
→ Log metrics
Core Services
Production-grade business logic and integrations
AI Integration Layer(lib/ai/orchestrator.ts)
• Multi-provider API abstraction
• Automatic retry with exponential backoff
• Token estimation & cost calculation
Cache Manager(lib/cache/manager.ts)
• SHA-256 input hashing
• TTL-based expiration
Logger(lib/observability/logger.ts)
• Structured event logging
• Experiment metrics tracking
Demo System
Modular demo definitions with versioning
server/experiments/
schema.ts - Zod input/output validation
prompt.ts - Versioned prompt templates
handler.ts - Response parsing
index.ts - Module export
Data Flow Diagram
Presentation Layer
Client (Next.js React)
ExperimentForm → Submit → ExperimentPage State
API Layer
POST /api/experiments/[slug]
• Zod Validation• Input Hashing
Cache Hit
Return Cached Result
Cache Miss
Proceed to AI Call
Intelligence Layer
AI Orchestrator & Experiment Handler
• Dynamic Prompt Generation
• Model Selection & API Call
• Schema-bound Output Parsing
• Cost & Token Estimation
Persistence Layer
Neon PostgreSQL
• Analytics• Feedback• Toggles• Iterations
Unified API Response
Result + Full Metadata + Persistence Confirmation
Type Safety & Validation
End-to-end type safety using TypeScript and Zod schemas
Input Validation
Each demo defines input schema that validates:
- • Required fields presence
- • Type correctness (string, enum, etc.)
- • Length constraints (min/max)
- • Custom validation rules
Output Validation
AI responses are validated against output schema to ensure:
- • Correct JSON structure
- • Valid enum values
- • Array and object shapes
- • Type coercion where appropriate
Adding New Demos
Creating new experiments requires minimal code and no core changes:
1. Create schema.ts with Zod definitions
Define inputSchema and outputSchema
2. Create prompt.ts with template functions
Support version control
3. Create handler.ts to parse responses
JSON parsing and schema validation
4. Export module from index.ts
Combine all pieces with metadata
5. Register in config/experiments.ts
Add to EXPERIMENTS_REGISTRY
Performance & Optimization
Caching
- • 24-hour default TTL
- • SHA-256 cache keys
- • Automatic expiration
- • In-memory (Redis-ready)
Resilience
- • Exponential backoff retry
- • 30s request timeout
- • Graceful error handling
- • Structured logging