Skip to main content

Advanced Usage

This guide covers advanced usage patterns and features of Eliza, including working with services, custom implementations, and advanced configuration options.

Video and Media Processing​

Video Service​

Eliza provides robust video processing capabilities through the VideoService class. Key features include:

  • Downloading and processing videos from multiple sources (YouTube, Vimeo, direct MP4 links)
  • Automatic transcription of video content
  • Caching mechanisms for efficient processing
  • Support for both manual and automatic captions
import { VideoService } from './services/video';

// Initialize the service
const videoService = VideoService.getInstance(runtime);

// Process a video URL
const media = await videoService.processVideo(videoUrl);

Image Processing​

The ImageService provides advanced image analysis capabilities:

  • Local and cloud-based image recognition
  • Support for GIF processing (first frame extraction)
  • Integration with multiple AI models for image analysis
  • Caching and batch processing capabilities

Memory Management and Embeddings​

Advanced Memory Operations​

The system supports sophisticated memory operations through various database adapters:

// Search memories with embedding similarity
const similarMemories = await db.searchMemoriesByEmbedding(embedding, {
  match_threshold: 0.95,
  count: 5,
  roomId: currentRoom,
  tableName: "long_term_memory"
});

// Create unique memories with deduplication
await db.createMemory(memory, "episodic_memory", true);

Custom Database Adapters​

You can implement custom database adapters by extending the DatabaseAdapter class:

class CustomDatabaseAdapter extends DatabaseAdapter {
  async searchMemories(params: {
    tableName: string;
    roomId: UUID;
    embedding: number[];
    match_threshold: number;
    match_count: number;
    unique: boolean;
  }): Promise<Memory[]> {
    // Custom implementation
  }
}

Speech and Transcription​

Speech Service​

The system includes a comprehensive speech service with support for:

  • Text-to-speech conversion with multiple providers
  • Voice customization options
  • Streaming audio support
  • PCM and WAV format handling
const speechService = new SpeechService();
const audioStream = await speechService.generate(runtime, text);

Advanced Transcription​

The TranscriptionService provides:

  • Local and cloud-based transcription options
  • CUDA acceleration support
  • Audio format conversion and normalization
  • Debug logging and error handling

Trust Score System​

The system includes a sophisticated trust score management system:

interface RecommenderMetrics {
    trustScore: number;
    totalRecommendations: number;
    successfulRecs: number;
    avgTokenPerformance: number;
    riskScore: number;
    consistencyScore: number;
    virtualConfidence: number;
}

Key features include:

  • Historical metrics tracking
  • Performance analysis
  • Risk assessment
  • Consistency evaluation

Browser Automation​

The BrowserService provides advanced web interaction capabilities:

  • CAPTCHA handling
  • Ad blocking
  • Content extraction
  • Proxy support
  • Cache management
const browserService = BrowserService.getInstance(runtime);
const content = await browserService.getPageContent(url);

Best Practices​

Memory Management​

  • Implement proper memory cleanup and garbage collection
  • Use the caching system effectively
  • Monitor memory usage in long-running processes

Error Handling​

  • Implement comprehensive error handling
  • Use the logging system effectively
  • Monitor system performance

Performance Optimization​

  • Use batch processing when possible
  • Implement proper caching strategies
  • Monitor and optimize database queries

Configuration Options​

Environment Variables​

Key configuration options include:

CUDA_PATH=/usr/local/cuda  # For GPU acceleration
OPENAI_API_KEY=sk-...      # For OpenAI integration
ELEVENLABS_API_KEY=...     # For voice synthesis

Runtime Configuration​

The runtime can be configured with various options:

const runtime = {
  character: {
    settings: {
      model: "gpt-4",
      temperature: 0.7,
      maxTokens: 2048
    }
  },
  // Additional configuration options
};

Advanced Features​

Custom Actions​

Implement custom actions for specialized behavior:

class CustomAction extends BaseAction {
  async execute(context: ActionContext): Promise<ActionResult> {
    // Custom implementation
    return {
      success: true,
      data: {}
    };
  }
}

Custom Evaluators​

Create specialized evaluators for specific use cases:

class CustomEvaluator extends BaseEvaluator {
  async evaluate(context: EvaluatorContext): Promise<EvaluationResult> {
    // Custom evaluation logic
    return {
      score: 0.95,
      confidence: 0.8
    };
  }
}

Security Considerations​

  • Implement proper input validation
  • Use secure token management
  • Monitor system access
  • Implement rate limiting
  • Use proper encryption for sensitive data

Troubleshooting​

Common issues and solutions:

  1. Memory leaks

    • Monitor memory usage
    • Implement proper cleanup
    • Use garbage collection

  2. Performance issues

    • Optimize database queries
    • Implement proper caching
    • Use batch processing

  3. Integration issues

    • Check API keys and permissions
    • Verify network connectivity
    • Monitor API rate limits