Preliminary math
Initial formula for calculating the rating of a post. n is the post number, r is the number of posts (which I'd like to work in somehow, but can't immediately think of a good way.) En is the expertise value of the reviewer, Rn is the rating of the review, and Cn is the reviewer's credibility. Valid values are 0-3 for E, -2, -1, 1, or 2 for R, and -1 to 1 for C. So what ends up happening with these values? If C is negative, which means the reviewer is not credible, (C+1)2 is less than 1, so the overall influence of the review is lowered. If C is positive, the reviewer is credible, so their rating counts more. R is obvious. 2E scales the value of the rating exponetially with the reviewer's expertise value. On the bottom of the equation, you have the same formula as the top, only with an ideal rating. The result of this formula is going to be -1 to 1, with 1 meaning that everybody who has reviewed the post has concurred, and -1 meaning that everybody who reviewed the post thought it was stupid. People who are experts have exponentially more influence on the result than people who aren't. People who have no credibility have no impact on the final value, and people who are perfectly credible have 4x the influence as a poster with neutral credibility. And the positive/negative ratings are reflected in the final positive/negative value.Still want to work in the number of times a post has been rated, but I'm not sure how...for that matter, I'm not entirely sure that metric is meaningful in this context, but I have the feeling it should be.
posted by John @ 11:25 PM
2 comments
2 Comments:
Hi John,
I think this formula is a great start. It looks fine from a distance, but the real test is what happens when you put in sample people and situations.
So my first suggestion for your next post is to post some graphs to show how the overall rank changes as you modify one variable while you leave the others constant (make sure the axes are linear rather than logorithmic).
My second suggestion is to pick some sample scenarios and evaluate the metric for them. Eg, someone who has low expertise but high credibility (Lawrence Lessig weighing in on chili recipes), or high expertise but low credibility (Rush Limbaugh commenting on Talk Radio).
Finally, some additional questions:
* Why can R go up to 2 instead of just -1 to 1?
* What formulae determine the value of E and C?
Anyway, great start--keep it going.
jon
I guess there's no real reason that R goes to 2 and -2...I did want a tiny degree of shading beyond just good=1 and bad=-1, but I don't think it really matters in the formula whether the options then become -2, -1, 1, and 2 or -1, -.5, .5, 1.
E is just the claimed expertise of the poster...it isn't calculated, it's input. C is the credibility calculation I just describe in the next post, 'Instant Karma (or not)'.
When I was figuring out the formula, I did plug in some different situations...I didn't graph it though, since there were so many independent variables involved. But I'll isolate some of them and see what it looks like later.
Post a Comment
<< Home