After working temp for a credit card company, I've tried to assuage my moral scruples by learning exactly what usury is. No such luck.
Hillaire Belloc held that usury is taking interest on a non-productive loan. Belloc's example of a usurious relation is if a farmer's field is let out, only to be rendered useless by river which has changed its course. While still paying interest on the property, the tenant would have no means of making a return on his investment. This has a great deal to be said for it. If I loan somebody $100 for food and charge them 5% interst, they're out $105 with no chance to make it back. But if I loan somebody $100 at 5% interest, and he puts it to productive use, then we're both ahead, and there's no transfer of wealth from the poorer to the richer. Yet there are a few difficulties: for instance, what if the loan is potentially productive, yet the loanee botches the use of his capital?
The author of the usury article in the Catholic Encyclopedia mentions those who share Belloc's theory, while neither approving nor condemning it.
John Paul II on Usury, from the Italian Jesuits.