I came across this article about youtube ditching its fivestar rating basically because users treat it as a binary thing, either they love it (5 stars) or hate it (0 stars) or simply dont bother voting, there are some edge cases where users DO bother voting however.
more here... http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html
So, one of my sites that has 4210 votes in votingapi_vote, I was able to see if my users are like what is being reported elsewhere - do they really bother to click the right 'fivestar' or are they simply using it like a "love/hate" button.
mysql> select value, count(*), 100/4210*count(*) as percentage from votingapi_vote group by value order by value;
+-------+----------+------------+
| value | count(*) | percentage |
+-------+----------+------------+
| 1 | 40 | 0.9501 |
| 2 | 40 | 0.9501 |
| 3 | 39 | 0.9264 |
| 4 | 39 | 0.9264 |
| 5 | 39 | 0.9264 |
| 20 | 565 | 13.4204 |
| 40 | 124 | 2.9454 |
| 60 | 230 | 5.4632 |
| 80 | 404 | 9.5962 |
| 100 | 2690 | 63.8955 |
+-------+----------+------------+
10 rows in set (0.01 sec)
if i lay my results (normalised) over the top of what youtube are also reporting, the following graph illustrates whats going on, the blue line is youtube, and the redline is the ratings from http://tshirtslayer.com

You can see that except for a small percentage (5%?) the results of the tshirtslayer behaviour is pretty much the same as to be expected, run that query on your sites and see what you get?
I think this paragraph from the youtube post sums it up
Seems like when it comes to ratings it's pretty much all or nothing. Great videos prompt action; anything less prompts indifference. Thus, the ratings system is primarily being used as a seal of approval, not as an editorial indicator of what the community thinks about a video. Rating a video joins favoriting and sharing as a way to tell the world that this is something you love.
Again, fivestar could be another case of "Hey that's cool! let's do that!" but is it REALLY what the users care for..
Perhaps it's in the language? maybe the options arent clear enough Poor, Okay, Good, Great, Awesome, maybe 5-stars is 2 or 3 stars too many?