mheval
Submit results

Quality benchmark

CBT-Bench

Whether a model can read a patient's thinking the way a CBT therapist does, by identifying cognitive distortions and core beliefs.

“While LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios requiring deep analysis of patients' cognitive structures.”
Zhang et al., 2024
Metric
Mean weighted F1
Scale
0–1
Source
Paper ↗

Headline score

  1. Claude Opus 5.5Anthropic · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.632
  2. Gemini 2.5 ProGoogle · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.630
  3. GPT-5OpenAI · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.617
  4. Qwen3.5-122B-A10BQwen · Open weightsRun settings
    Temperature
    0
    Max tokens
    2048
    0.609
  5. GPT-6 AstraOpenAI · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.606
  6. GPT-6 LunaOpenAI · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.603
  7. GPT-4oOpenAI · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.602
  8. Claude Sonnet 4.5Anthropic · Public APIRun settings
    Temperature
    0
    Max tokens
    2048
    0.600
  9. Qwen3.5-27BQwen · Open weightsRun settings
    Temperature
    0
    Max tokens
    2048
    0.592
Bars span the metric's full 0–1 scale; higher is better. Hover the info icon for settings a model ran with.

Task

Model
Gemini 2.5 ProGoogle · Public API0.5050.8170.567
Qwen3.5-122B-A10BQwen · Open weights0.4840.7870.556
GPT-5OpenAI · Public API0.4830.7780.589
Claude Sonnet 4.5Anthropic · Public API0.4770.7370.585
Claude Opus 5.5Anthropic · Public API0.4700.8080.617
GPT-6 AstraOpenAI · Public API0.4670.7670.584
GPT-6 LunaOpenAI · Public API0.4630.7730.573
Qwen3.5-27BQwen · Open weights0.4540.7880.532
GPT-4oOpenAI · Public API0.4410.8090.557

Cognitive distortion (F1 per label)

Model
Gemini 2.5 ProGoogle · Public API0.6340.2710.1210.6670.4130.6240.3530.3280.6900.544
Qwen3.5-122B-A10BQwen · Open weights0.6070.3620.0690.6750.4270.5350.2750.3730.6190.523
Qwen3.5-27BQwen · Open weights0.5790.3620.0000.7110.4510.3940.1820.3240.5840.571
GPT-6 LunaOpenAI · Public API0.4900.3820.0000.7250.4330.5140.3160.2500.6590.555
Claude Opus 5.5Anthropic · Public API0.4790.4000.0000.7250.3800.5280.3400.2860.6970.559
GPT-6 AstraOpenAI · Public API0.4550.3680.0000.7040.4470.5280.3080.3820.6600.536
Claude Sonnet 4.5Anthropic · Public API0.4530.3750.0670.6670.4370.5070.3000.3940.7230.567
GPT-5OpenAI · Public API0.4190.4000.0740.7650.4350.5290.3270.4180.7050.548
GPT-4oOpenAI · Public API0.1670.3780.1290.7080.4150.5590.3100.4840.6520.589

Primary core belief (F1 per label)

Model
Gemini 2.5 ProGoogle · Public API0.8760.8450.664
GPT-4oOpenAI · Public API0.8660.8400.656
Claude Opus 5.5Anthropic · Public API0.8470.8390.690
Qwen3.5-122B-A10BQwen · Open weights0.8320.8000.682
Qwen3.5-27BQwen · Open weights0.8260.8210.673
GPT-5OpenAI · Public API0.8220.7800.689
GPT-6 LunaOpenAI · Public API0.8180.7780.678
GPT-6 AstraOpenAI · Public API0.8080.7460.710
Claude Sonnet 4.5Anthropic · Public API0.7270.8210.652