Probably 95% of Xbox Live players will never face the banhammer and for them the enforcement team is working in the background and they'll never see them. They play fair, they communicate properly, they have inoffensive gamertags and profiles. The other 5% are generally trolls, "l337" hackers, bullies, thieves, pirates, and jerks. The Official Xbox Live Code of Conduct spells things that are against the rules out fairly clearly. There is also a great article from Microsoft all about how the Xbox Live Policy and Enforcement Team actually works you can see here - Xbox Live Enforcement "Unicorn Ninja" Keeps Service Safe and Fun for Everyone.
The important thing to note is that when you are reported for a violation by someone on Xbox Live (yes, reporting people on Xbox Live actually does accomplish something), a real person actually checks every single report and makes a punishment decision. There isn't some automated process where people are undeservingly banned accidentally by a computer. Real people are reading every report, checking every reported gamertag, and doing their due diligence to see if you really should be banned or not. If you get banned from Xbox Live, you more than likely deserved it barring the occasional name misunderstanding or misinterpretation that usually get cleared up quickly. Punishment can range from suspensions of 24 hours up to two weeks, on up to full permanent bans.
What Behavior Are We Talking About?
We've covered Xbox Live Etiquette and general behavior as well as an Xbox Live Survival Guide for Girls already. If you harass people, yell and scream into your microphone, curse a lot, obviously cheat in a game, etc. you are going to get banned. That is all pretty obvious. There are some less obvious, but just as stupid and easily avoidable, things that some people do to get banned as well.
Offensive Gamertags - Don't put something racist, sexist, gross, violent, etc. into your gamertag. People will report you, and you will get in trouble. Usually Microsoft will give you a free opportunity to change the gamertag to something else, and if you comply you won't get banned. Just make sure you actually change it to something inoffensive, or you'll end up banned anyway.
Offensive Motto - Again, racism, sexual content, violence, etc. in your Xbox Live motto will get you banned. Don't do it.
Offensive Profile - For whatever reason, people like to brag about dumb stuff in their profile. We've seen people brag about how they can hijack accounts, or sell mods, or help people cheat in games, or try to sell stolen MS Points, and a bunch of other stupid illegal stuff like pretending to actually be on the Xbox Live Enforcement Team in their profiles. You will be banned.
Note that these are just gamertag bans and not system bans. You are free to create a new gamertag (and pay another $40-60 for a year of Xbox Live) if you want, by why risk getting banned in the first place?
An important thing to remember is that Xbox Live players are really the police, while the moderators and enforcement team are the judge, jury, and executioners. Other people read your profile, play games with you, etc. and when you do bad things they will report you. So if you don't do bad things, you won't ever get in trouble. Seems obvious, but as the blog we linked to at the top indicates, it isn't quite so clear for some people.
These are all really obvious things not to do, but people get banned for them constantly. And it is kind of mind boggling that people would want to engage in any risky activity on Xbox Live at all since it is a paid service. When you get banned, your money is basically flushed down the toilet by your own stupidity. Why throw away a $60 subscription (or, really, less than that if you pay attention) just for the short term thrill of saying something naughty on your profile? We're not trying to be preachy, we just don't understand.