"They're happy now," says Combeferre. "And I trust they will remain so. He's a fine man, as you say. But--he is a fallible man, and husbands have great power. Much as he loves her, he may still hurt her accidentally, and she'll need to go to you, to be comforted and defended. A father's love is more selfless than a husband's, monsieur."

If he's being brutal, he's also speaking the truth, and surely it would be still more brutal if Valjean never heard this?