UK banking sector believes Brexit would have negative impact on their business