Sponge

I've mentioned this before but it has never been more obvious than this week during the Xenophon shit storm.
Basically when you search on scientology and sort by date (as you'd want to do to get the latest news of course) you wind up getting pages of totally irrelvant articles from ninemsn.com.au (9 News, Australia).

Ray Hill's scientology news gathering tool shows the true scale of the problem.
Scientology-related news from all over the Web (just look/click at the stats at the top graphs and the pig pink column on the right listing all the actual irrelevant articles).
2 days ago it was 72 relvant vs. 120 irrelavnt, with ninemsn.com.au being the main cuprit by far. The cult doesn't need to publish scientology PR bullshit to push down the critical articles when it apparently has 9 News website doing it for them!

What I need is someone who can figure out why google news is picking this mass irrelvancy up.
I've clicked on some of these 9 News articles and in most cases there NO mention of scientology anywhere on the page. I had thought that they might have had some sort of badly coded "top stories" thingummyjig on all their news articles which confused google news but it doesn't seem to be the case and the article's I've just clicked from google news has no instance of the word scientology anywhere on the page and rarely any click-through relevancy.

I want to get in touch with the webmaster of th 9 News website and put them on the spot. I also might contact the competing network 7 News and see if they can make something out of it. However, I'm not prepared to put my tinifoil hat on and say that 9 News is totally doing it on purpose, and without any back-up evidence, but I've just tried other subjects, like the bushfires, cricket and allsorts of other things yet there is no irrevancy faggotry from 9 news on those. You can even search for "xenophon" and there's no faggotry. It's just "scientology" and it is fucking weird.

tl;dr
Are there any web page coding experts here who can look at these totally irrelvant article pages on ninemsn.com.au and see, from a technical standoint, why they are being picked up via google news on a date sorted search for scientology (and the likelyhood of intentional or human error).
I want to get this nailed.


You don't see this on a default relevancy search on googlenews and it has to be said that there are one or two other albeit less noticeable culprits on the date sorted scientology search faggotry (the "Gaea Times" being another example - who I'd never heard of before).

In the meantime, the best place to get the latest scientology critical news from several gathered sources, without most of the junk, is Ray Hill's epic news gathering robot filter at: Scientology-related news from all over the Web

David Miscavige's worst nightmare: www.truthrundown.org

Operation Clambake on Twitter: http://twitter.com/oper_clambake
whyaretheydead.net replacement site now live at: http://www.whyaretheydead.info

Anon123456

yeah i noticed this too.

functionunknown

it's picking it up in the HTML.
Example: in the source of this page:

Sponge

That's a good exmple. Totally no mention of scientology, visibly, on the webpage at all.
Now, the questions is, how and why?

David Miscavige's worst nightmare: www.truthrundown.org

Operation Clambake on Twitter: http://twitter.com/oper_clambake
whyaretheydead.net replacement site now live at: http://www.whyaretheydead.info

Anon123456

im not a code guy but why is it in the code in the first place if its not even on the page? is it possible they are doing it on purpose?

RosyGlass

is this video html:
<!--HL_GROUP72556BEGIN-->
<div id="cat_hl_87501" class="orderedItemsIs6">
<span class="odd" id="item_4" ><span class="image"><a href="http://ninemsn.video.msn.com/v/en-au/v.htm?f=39&g=039E9F23-2F5C-4FEE-AD88-949B25E1E27E&p=aunews_auaca&t=m164&mediaid=277075" ><img src="http://news.9msn.com.au/img/2009/candy_box/1911_hubbard_cb.jpg" alt="ACA" title="ACA" border="0" align="middle"></a></span><span class="content"><span class="title"><a href="http://ninemsn.video.msn.com/v/en-au/v.htm?f=39&g=039E9F23-2F5C-4FEE-AD88-949B25E1E27E&p=aunews_auaca&t=m164&mediaid=277075" >Church slammed</a></span><span class="text">Aussie senator brands Scientology a 'criminal organisation'.</span></span></span>
</div>
<!--HL_GROUP72556END-->

1911 is year Hubbard born

_You_

I'm not a CSS OR HTML guy, but I guess it comes from the selection of articles that you can find near the bottom of the page, below their top 10 list. When I checked, there was an article named "Church Slammed", with "Hubbard" and "Scientology" in it. Why does Google pick up this unrelated stuff?

_You_

The code is on the page, but it is hidden from display using Cascading Style Sheets. So you don't see it, but a bot/spider reading the page does, since bots don't look at css..just xhtml/html etc.

The content in question is in a box called #article_feat. This is the part where you see the headlines and a thumbnails down at the bottom. The rule that hides it is:


That rule tells this section of code in the html that it is not to show up on the page:


In doing this, they've essentially "loaded" their search results, because this shows up on every story. The only thing really curious about it is that two of featured articles in this section are still loaded, but they aren't displayed...so why are they there? There could be legit answers for this(laziness is the most likely), or they could just be loading their search results with some extra items because it earns them a couple of extra hits.

This is more a problem with Google than it is this site. Bots can't distinguish between what story is actually on the page, and what's in some irrelevant section. They just know that this particular keyword appears on this particular page.

_You_

Bots can discriminate up to a point. It appears Yahoo and Bing can do it. One easy quick trick is to look for a block element (<div>, etc.) with highest plain text word count, which is likely to be the main article. I would have expected Google to do a better job at filtering.

fitch2000

So is someone KSW or is this a coincidence?

I don't know what ya'll are saying :(

Sponge

Google can do it too however it seems that they lower the bar considerably on a date sorted search though. The default relevancy search doesn't appear to be too much of a problem. What ninemsn.com.au is doing, whether intentional or not, is taking advantage of this.
Yahoo and bing are much less confused but overall they seem pick up much less articles about any specific keyword string than googlenews does anyway. It's a very unusual problem and the sheer number of these irrelvant articles is stunning. Over the last few days there have been more irrelvant artcles picked up by googlenews on scientology than relevant ones, most from ninemsn.com.au.
I've never seen this on such a scale before from one news outlet. I've tried keywords from other peviously ran and off the page articles from 9 News but it isn't having the same affect as the scientology search.

David Miscavige's worst nightmare: www.truthrundown.org

Operation Clambake on Twitter: http://twitter.com/oper_clambake
whyaretheydead.net replacement site now live at: http://www.whyaretheydead.info

hartley patterson

So it's not coding aimed at secretly boosting Scientology in Google News, since any news item could be at #4-5. It is annoying to readers, who are being led to pages with no visible link to what they are seeking...

Google does IIRC downgrade pages it thinks are 'linkfarms', ie full of links but with no content. Is this a way of trying to circumvent this, if they think the linkfarm check is only looking at the visible page and not the coding? Or is some bright young coder at ninemsn feeding them a load of bovine effluent?
Hm... if 4-5 were there, would some non compliant browser cause them to break the layout, and the coder has used CSS to comment them out?

The Cult of Eternal Litigation

VMs in Haiti 2010

fitch2000

Not from down under so I googled ninemsn.com.au
"fastest growing" just made me lol

_You_

My guess is either they changed their layout, are in the process of changing their layout, or made an aesthetic decision on that section, and instead of changing the output of whatever piece of code is spewing out items 1-5, they took the lazy root and turned 'em off with css.

No matter what tho', that's some sloppy coding and shitty user experience.

In an age of wysiwyg editors, this can fail hard so I could see arguments for indexing the whole page. Simple fact of the matter is that most automagick output suffers from divitis and many of them are retarded enough to employ div's where none are needed or where they should've used a <p>. (Not that I see any problem with SEO penalties for shit code, mind you. ) You run the risk of returning a lot of irrelevant results--say someone's site disclaimer or footer getting returned on anything vaguely legal because that was the largest div without a bunch of other nested div's in it..as well as missing pertinent content because the actual story was brief or consisted primarily of some type of other non-textual media like img's or vid with only some brief descriptive or introductory text.

Sponge, have a look at these results: drinking age australia - Google News
It isn't just scientology. They're fagging up other search results as well.

Sponge

^I see now (about other search results). I think I maybe just didn't manage to hit the right ones to show any consistency.

The last bad episode for the scientology date-sorted search results was during the French fraud trail judgement media shit storm. Thre was a lot of irrelvancies then but the sheer amount of international media on that story was far higher than this time round.

What I need now is a clever bot/script to email 9 News, for every article irrelvant to a given googlenews search, with a suitable message. ;)
Ray, are you there? :)

David Miscavige's worst nightmare: www.truthrundown.org

Operation Clambake on Twitter: http://twitter.com/oper_clambake
whyaretheydead.net replacement site now live at: http://www.whyaretheydead.info

tazor

Wait till tomorrow and a lot of anons protest videos will be up on nine and 7. Those are two of the stations that want them for news reports.

Scientology was just a joke written by hack science fiction writer L. Ron Hubbard. Xenu is the punch line.

Please! Don't get the fuck me out! At least let me be in the dome! ~Lily the Troll~

Sponge

That's just it though. 9News are fagging up their own googlenews results for a given search subject.

David Miscavige's worst nightmare: www.truthrundown.org

Operation Clambake on Twitter: http://twitter.com/oper_clambake
whyaretheydead.net replacement site now live at: http://www.whyaretheydead.info

_You_

Most other news sources also have a section for "Other news etc." I didn't look into this deeply, but it seems to me that ninemsn aside other news links at the bottom of the page (like all others do), it also include summaries which can't be seen -- one wonder why these summaries would be included if they can't be seen. Google news might be looking for a number of plain text words above a specific trigger, and the summaries causes it to assess it as related to whatever matches the search query. It sucks.

I currently have four categories for any news item showing up on the feed:

* Pure crap => Goes in the incinerator (currently only feeds.malaysianews.net is there)
* Likely irrelevant/self-serving (blacklist of web sites which typically provide irrelevant/self-serving stuff)
* Likely relevant (default)
* Awesome (St. Pete's Time and The Age are in there)

Obviously I categorized ninemsn as likely irrelevant, and I would move it to the incinerator if it wasn't for the three pieces of news that were deemed relevant in the past:

Scientology library: "ninemsn"

In any case, regardless of how I categorize them, these ninemsn irrelevant items I think could cause relevant news items off the rss feed before they are seen, which is very bad. I don't understand why Google News didn't fix that as of now, as clearly the problem is widespread (not just Scientology keywords) and has been going on for quite a while now, in effect defeating the purpose of their feed.

FurFagsWorldWide

So google should be contacted, with examples of both searches on "scientology" but also other keywords, to show how the search is generally being messed with.

Also, ninemsn should be contacted, to highlight this issue for them, so they can avoid getting downgraded by google, should they chose to try and fix this error.

Action > Talk

_You_

I just did:
Contact Us - Google News (publishers) Help

but I suppose the more complaints the more likely the blip will show up on their radar