This sadness is noted in the story of the young rich man:
... The young man saith to him: All these I have kept from my youth, what is yet wanting to me? Jesus saith to him: If thou wilt be perfect, go sell what thou hast, and give to the poor, and thou shalt have treasure in heaven: and come follow me. And when the young man had heard this word, he went away sad... (Matthew 19:16-22)
The Handbook of Moral Theology by Prummer lists some of the evil results of sloth: tepidity to the divine precepts, wandering towards what is forbidden, faint heartedness, and despair of salvation. The remedies given are: consideration of the evil results of this vice, meditation on man's eternal reward.