I'm 6'3" and the only problem I have with the armrest is that it hurts my ears.
My wife is constantly complaining and won't shut up about the lack of a passenger armrest.

To be honest, my wife really does like the xB also. Even though she preferred another vehicle while we were shopping, she now admits that I made the right choice getting the xB. There are a few items we feel are missing in the car, like the front armrests, However, for the price, we both believe the xB is a great value.