CLIP-CC-Bench Leaderboard

Seventeen video–language models ranked on paragraph-level descriptions of 90-second movie clips. Borda is the Borda count across the five embedding judges; Mean HM-CF is the average harmonic mean of coarse- and fine-grained similarity across those judges.

Final ranking of the 17 VLMs on CLIP-CC-Bench.
Rank	VLM	Borda	Mean HM-CF
1	VideoLLaMA3	80	0.67
2	mPLUG-Owl3	75	0.66
3	LLaVA-OneVision	67	0.64
4	ViLAMP	67	0.64
5	LongVU	61	0.63
6	Qwen2.5-72B	55	0.62
7	Qwen2.5-32B	48	0.61
8	VideoChat-Flash	42	0.60
9	MiniCPM-V	42	0.60
10	Video-XL	36	0.59
11	ShareGPT4Video	29	0.58
12	InternVL2	27	0.58
13	TimeChat	20	0.56
14	LLaVA-NeXT-Video	16	0.55
15	TS-LLaVA	9	0.53
16	Oryx	6	0.52
17	LongVA	0	0.48

Coarse beats fine, almost everywhere. Paragraph-level (coarse-grained) matching scores higher than sentence-level (fine-grained) matching for essentially every model — the top-ranked model reaches 0.73 coarse against 0.63 fine. Models capture the overall gist of a clip far better than they get the individual events right.

The best mean HM-CF on the benchmark is 0.67: long-form video description is a long way from solved.

Metrics are computed on 199 of the 200 clips — clip 126 is absent from all 17 model outputs, so it is excluded throughout.

Citation

@misc{ali2026clipccbench,
  title         = {{CLIP-CC-Bench}: Evaluating Paragraph-Level Video Descriptions in Video--Language Models},
  author        = {Ali, Mukhtiar and Dubey, Harsh and Mishra, Sugam and Pack, Chulwoo},
  year          = {2026},
  eprint        = {2608.04302},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  note          = {Presented at the 2nd Workshop on Evaluation for Multimodal Generation (EvalMG), ACM SIGIR 2026},
  url           = {https://arxiv.org/abs/2608.04302}
}

CLIP-CC-Bench · Leaderboard

Links

Citation