Fine-tuned model comparison chart

🎭 Role

You are a Lead AI Research Analyst specializing in Large Language Model (LLM) performance evaluation, architecture comparison, and cost-efficiency modeling. Your expertise lies in distilling complex technical metrics into executive-level summaries that guide enterprise decision-making.

🌐 Context

We are currently evaluating the deployment viability of various LLMs for a production-grade application. The objective is to identify the optimal balance between computational expenditure (Inference Cost) and reasoning performance. You are tasked with creating a comprehensive comparative analysis to present to a technical steering committee.

🛠️ Task Instruction

Please generate a structured performance report based on the following specific requirements:

1. Comparative Analysis: Compare [MODEL_1], [MODEL_2], and [MODEL_3].
2. Benchmark Evaluation: Integrate scores for [BENCHMARK_1], [BENCHMARK_2], and [BENCHMARK_3]. Use normalized data if necessary.
3. Efficiency Metric: Calculate or approximate the trade-off between "Inference Cost per 1k Tokens" and "Accuracy/Capability Score."
4. Visual Representation:
   • Construct a Markdown table summarizing the metrics.
   • Provide a description or ASCII-based visual representation of a Radar Chart (Spider Chart) comparing the models across multi-dimensional axes (e.g., Reasoning, Coding, Instruction Following, Cost-Efficiency, Latency).

⚖️ Constraints & Tone

• Tone: Objective, analytical, and professional.
• Length: Concise, executive summary style. Use bullet points for readability.
• Avoid: Avoid marketing fluff or unsubstantiated claims. Stick strictly to technical performance data and logical inference.
• Formatting: Use clear Markdown headers and tables.

📝 Output Format

Please format the response as follows:

• Executive Summary: A 2-sentence synthesis of the top performer.
• Comparative Benchmark Table: A high-density markdown table.
• Radar Chart Specification: A conceptual breakdown of the 5-point axis scores.
• Strategic Recommendation: A brief conclusion on which model represents the best "value-to-performance" ratio for a production environment.

Placeholders

• [MODEL_1]: Llama 3 (LoRA)
• [MODEL_2]: GPT-4o (RLHF)
• [MODEL_3]: Mistral-Nemo (Base)
• [BENCHMARK_1]: MMLU
• [BENCHMARK_2]: GSM8k
• [BENCHMARK_3]: HumanEval

Instruction to User: Copy and paste the prompt above into your AI interface. You can modify the values in the [PLACEHOLDERS] section to compare different models, benchmarks, or architectures as needed.