Most definately can contribute that's for sure but I still wouldn't expect them to rub even with a fair bit of wear.
Wheel bearings okay?? A badly failing wheel bearing could cause that problem for sure but then again you'd sure know about it too.
Mmmm...