LLMDebug: Prompt-Engineered Large Language Models for Automated Root Cause Analysis in Microservices Architectures

Abstract

Debugging failures in modern microservices architectures presents significant challenges due to distributed execution, complex inter-service dependencies, and massive log volumes. Traditional rule-based approaches struggle with novel failure patterns and require extensive manual expertise. This paper introduces LLMDebug, a framework leveraging prompt-engineered Large Language Models (LLMs) for automated root cause analysis in distributed systems. We propose a multi-stage prompting strategy that combines chain-of-thought reasoning, retrieval-augmented generation (RAG), and multi-agent collaboration to diagnose failures across microservices. The framework achieves 89.3% accuracy (95% CI: 87.9–90.7%) in identifying root causes across 12 common failure scenarios, reducing mean time to resolution (MTTR) by 68% compared with traditional monitoring tools. Evaluation on a microservices testbed with eight services demonstrates that LLMDebug outperforms baselines including DeepLog, LogRobust, and MicroRCA by 12.9–31 percentage points. Ablation studies examine the contribution of each prompting strategy, and the implementation, datasets, and evaluation framework support reproducibility.