That happens to me too. It's like they expose their ass so that they can cover it everytime they see somebody look at it. It makes no sense to me, but hey.
I tug my shirt down sometimes but it is because I don't like to show off my body at all and sometimes shirts ride up while walking and need to be pulled back down. I also agree with Meliai in that I do feel feel self conscious sometimes if I sense a guy looking at my ass but I don't really think anything about the guy because as she said, it is the nature of men. I just hate people looking at me.
I feel men do the same thing if a lady is walking behind them. But its just like oh someone is behind me do I look okay? Do a quick clothes check, straighten your shirt out, pull your pants up, okay everythings good.
I usually notice a marked increase in hip movement when they realize I'm behind them. And I don't think it's because they're interested, just showing off to show off. So, either you're really creepy, or they're showing off to me BECAUSE of that lack of interest, and shy with you because you're so amazingly sexy. Hell if I know.
Like Antithesis said, sometimes shirts ride up when you walk. I sometimes check my shirt when I'm walking, and I pull it down if it's too high up. But I don't pull it down to cover my ass. My shirts don't cover my ass, my jeans cover my ass.