#prometheus

Set up comprehensive monitoring with Prometheus and Grafana. Components: 1. Prometheus server with service discovery. 2. Node Exporter for system metrics. 3. Application instrumentation with custom metrics. 4. Alertmanager for notifications (PagerDuty, Slack). 5. Grafana dashboards for visualization (RED metrics, resource usage). 6. Recording rules for aggregations. 7. Alert rules for SLO violations. Use Docker Compose for local setup. Include retention policies and high-availability configuration.

Build comprehensive monitoring and observability infrastructure for production systems. Monitoring stack architecture: 1. Metrics: Prometheus for collection, Grafana for visualization, 15-second scrape intervals. 2. Logging: ELK Stack (Elasticsearch, Logstash, Kibana) or EFK (Fluentd instead of Logstash). 3. Tracing: Jaeger for distributed tracing, OpenTelemetry for instrumentation. 4. Alerting: AlertManager for routing, PagerDuty for escalation. Key metrics to monitor: 1. Infrastructure: CPU (>80% alert), memory (>85%), disk space (>90%), network I/O. 2. Application: response time (<200ms target), error rate (<0.1%), throughput (requests/second). 3. Business: user signups, conversion rates, revenue metrics, feature usage. Alerting best practices: 1. Alert fatigue prevention: meaningful alerts only, proper severity levels (critical/warning/info). 2. Runbook automation: automated remediation for common issues, escalation procedures. 3. On-call rotation: 7-day rotations, primary/secondary coverage, fair distribution. Dashboard design: 1. Golden signals: latency, traffic, errors, saturation for each service. 2. SLA monitoring: 99.9% uptime target, error budget tracking, service level indicators. Log management: structured logging (JSON), log retention policies (90 days), centralized aggregation with filtering.