I definitely think that would annoy me. I can see why you want to move it down. Will that be a quick fix?
It would be a very quick fix (10 minutes max) if I ever got around to it. Perhaps Christmas break. My coat has a hole in the armpit too, which I haven't gotten around to fixing. Sometimes it's hard to motivate to mend things.
Funny, it took me a minute to get my bearings in this picture. I rather like the perspective though!