Skip to main content

run_llm_with_timeout

Function run_llm_with_timeout 

Source
fn run_llm_with_timeout(
    provider: Option<Arc<dyn LlmProviderTrait>>,
    prompt: String,
    timeout: Duration,
) -> Option<SearchIntent>
Expand description

Runs LLM classification with a timeout (CHAOS-H3).

ยงThread Lifecycle

Spawns a background thread for LLM classification. If the timeout is exceeded:

  • The result is discarded (receiver times out)
  • The thread continues to completion naturally (Rust threads cannot be killed)
  • A metric is recorded for monitoring orphaned operations
  • The thread will complete its HTTP request and exit cleanly

This is a deliberate design choice because:

  1. Rust has no safe way to forcefully terminate threads
  2. Interrupting HTTP requests mid-flight can cause resource leaks
  3. The thread will complete quickly once the LLM responds

For production, consider using async with timeout + cancellation tokens.