Yeah. Thats a very young child. When he gets older track might be good too. Track has many things that are team oriented. Martial arts are more about control and self esteem from what I gathered talking to parents that have their children enrolled in them.
My nephew will be 3 in December and my brother put him in tumbling/gymnastics. He loves it. He actually bounces around and flips around LESS than he did pre-tumbling...because now he has an outlet... Give it a shot, IronGoth. You don't know unless you try it.
Rock climbing. It's not really competitive in the traditional sense; more along the lines of gymnastics or weight lifting. You compete against yourself and gravity. It's not violent, as is reasonably safe (one death, ever, inside an indoor gym). There's also DDR, which is pretty athletic at high levels (friend of mine who does crew spit up after pushing himself too far). Sailing? Maybe a bit young for sailing at four. Roller hocky -- i started at around 6 and did it until I was eight. We just scrimmaged other people in the group (i.e. it was a 20 or so person group, we just did 10v10 or whatever.) Very laid back, but I did give myself a few (read: lot) scratches.
What about softball? Yeah, it is competitive, but it's not really a contact sport. You don't have to steal the ball or anything. They tap people with the ball, but if he goes out for pitcher or outfielder, he doesn't have to do that. Maybe that's the place to start, because from there he can learn that competition can be friendly if you practice good sportsmanship.