Things that are slightly annoying but not very:

Holding the lift for someone, when it turns out that they only want to go up one floor.

Look, if you take the lift to go up (or worse, down) one floor, then you are incredibly lazy. Unless you are on your feet the whole day, roaming from floor to floor in order to keep society functioning as we know it, then you don't need to use the lift to go one floor. You could use the stairs (the one over there, right next to the lift), it would be quicker and you'd even get a bit of exercise.

I personally don't think you should use the lift for going down fewer than a couple of floors, but I think that's more of a personal preference thing.

See also: driving to the park so that you can go for a walk; running machines and exercise bikes; drive-thru fast food restaurants; the United States of America.