Friday, October 4, 2013

- 'Low Information' Comments


I found a few really old comments waiting to be published this morning.  Blogger is supposed to email me when a comment is withheld by their spam filtering tool but it failed to do so, and I don't think to check very often.  All in it's hard to complain about a tool that's priced at 'free'.

And besides, while the comments themselves weren't SPAM, they might as well have been.  There were quite a few typed in all caps that explained as rationally as someone in that state could, that George Zimmerman was a racist murderer and that Blythe Masters should go to jail for ... something.  Obviously no actual data to support either view was offered - just the fact free opinion.

Since the piece I wrote about Blythe was directly pertaining to the way people seem to hate her for no good reason (when there are other perfectly good reasons to dislike her if you actually know her)  I approved one of the nonsensical comments and responded to it.  The George Zimmerman complaints I just dumped in the trash.  George has enough people hating him for no good reason and his illiterate detractors need no further help.

My daughter, in 8th grade now, is learning about the difference between 'fact' and opinion in school.  I know... I laughed when I heard it too. But unlike her peers from more liberal homes, she's actually learning the difference.  In fact, she mostly knew it already.  Making fun of her teacher's political opinions is great sport at our house.  And remember, this is the girl who wanted to give her anti-gun teacher a free membership in the NRA.

But it seems to me to be a real breakdown in the education system.  This is the world we live in now, and I would think the education establishment would try very hard to respond to that.  We teach people to read and write (apparently in all caps) but still never tell them how to include any real information in their writing.  How does anyone ever expect to persuade with such illiterate yammering? It's the literary equivalent of passing gas. Surely not everyone can be so narcissistic that they believe their opinion alone, totally unsupported, should be enough to win an argument.

I guess the most interesting thing though was that in these two particular cases, the Spam filter seems to be considerably smarter than the people writing the comments.  If I'm not mistaken, it has a natural language parsing engine that examines the 'information content' of the comment and if it's below a certain threshold, it holds back the post.  This may sound super sophisticated but I've been working with NLP for over a decade and in English it's easier than it sounds.  The only question is whether the threshold is set properly, and with the sample size that blogger has, I'm sure they're pretty close.

So a simple computer program - probably written as no more than a few hundred lines of python with a pretty good dictionary behind it - can figure out if your opinion contains something relevant to a discussion.  But our public schools don't seem to be capable of teaching people to do the same.  If my daughter's teachers are any indication, it's because they don't know how to do it themselves.  That's not exactly the rise of 'skynet' yet, but if you're one of those commenters it probably seems like it is.

 Anyway, I do what I can with the comment section, and I will continue to.  Apologies to those of you who may have been slighted unjustly by the imprecision of NLP.  If it makes you feel any better, I'm sure it's cost me a lot more over the last 7 years than it has you.  






No comments: