Prosody Diversity

Measuring Prosody Diversity in Zero-Shot TTS: A New Metric, Benchmark, and Exploration

Contents

DS-WED Metric
Comparison of DS-WED and Acoustic Metrics with Human Ratings
A Group of Samples from Prosody Diversity Benchmark
Effect of Duration Perturbation on Prosody Diversity
Effect of DPO on Prosody Diversity

DS-WED Metric

We reinterpret the weighted edit distance as the minimum perceptually prosodic modifications required to transform one speech into another at the discrete level for audio samples synthesized by the same system, using the same text and the same speaker.

Comparison of DS-WED and Acoustic Metrics with Human Ratings

PMOS exhibits the strongest correlation with DS-WED, followed by MCD. The correlation with log F0 RMSE is weaker, suggesting that pitch deviations contribute to perceived differences but account for only part of the variability in pitch, neglecting rhythm, intensity, and duration.

Pairwise comparisons within group. For each group, pairs with the minimum and maximum PMOS score are shown.
	log F0 RMSE	MCD	DS-WED	PMOS
Group 1	0.330	6.433	101	3
Group 1	0.220	6.157	263	5
Group 2	0.141	3.259	123	1
Group 2	0.138	4.047	168	3
Group 3	0.352	4.052	145	2
Group 3	0.261	4.591	204	5

A Group of Samples from Prosody Diversity Benchmark

English Text

after early nightfall the yellow lamps would light up here and there the squalid quarter of the brothels

Speaker Prompt

Average pairwise diversity scores for a group of 5 generated samples per system.

System

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Avg. DS-WED

XTTS-v2

115.9

CosyVoice

120.9

CosyVoice 2

104.6

E2 TTS

72.3

F5-TTS

65.0

ZipVoice

105.6

MaskGCT

161.4

Pairwise comparisons within group. For each system, pairs with the minimum and maximum DS-WED score are shown.

System

Sample A

Sample B

DS-WED

XTTS-v2

139

CosyVoice

143

CosyVoice 2

136

E2 TTS

F5-TTS

ZipVoice

132

MaskGCT

130

195

Effect of Duration Perturbation

Duration Perturbation increases prosody diversity, indicating that duration variation during inference is a key factor shaping prosody.

English Text

so no tales got out to the neighbors besides it was a lonely place and by good luck no one came that way

Speaker Prompt

Average pairwise diversity scores for a group of 5 generated samples with or without duration perturbation.

System

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Avg. DS-WED

F5-TTS w/o Duration Perturbation

59.4

F5-TTS w/ Duration Perturbation

84.6

Effect of DPO

Applying DPO reduces prosodic diversity, reflecting the general tendency of RL to prune variation while amplifying reward-aligned behaviors.

English Text

i will endeavour in my statement to avoid such terms as would serve to limit the events to any particular place or give a clue as to the people concerned