Ye understand the bank situation perfectly - Banks have been built, and are well established. Lots of banks, not enough borrowers, too much gold coin floating in the game to create loan demand.
Lots of swag spent on em. There's been requests to be able to tear them down, but no go.
As an old banker who was one of the first, I keep at it and will. Why? Because it's there.
There's much more to the game than just banks - they are simply a means to move swag around Avonmora.