How to search websites

...and actually find what you are looking for

"Search websites? Really? Are you kidding?" Searching for information on the Web became a no-brainer. Just type your query into the search engine of your choice! Well, the choice might actually be your browser's, but even then, hardly anybody questions the procedure nowadays. Or thinks about it. Strangely enough, not many seem to question the results they get either. We are pretty sure that these results are expertly fetched and ranked, with best matches on top of the list. Sort of, in Google we trust. Amen?

Google search page

How engines search websites and rank them

When we perform a web search, we rarely think about how the result list gets compiled. This knowledge, however, can help us understand how search engines tick and how we can use them to find what we are searching.

To be included in search engine's database, a web page needs to be indexed by a program usually called bot, short for robot. The program scans the content of the page just as it is found on the Web, and tries to make out what it is about.

Since bots are not human, they can only rely on what's on the surface when analyzing a page. They look for words and word combinations, count their appearances and evaluate other factors. For example, words in headlines, set in bold font or emphasized otherwise, are considered more important. The same goes for special on-page areas like title and description.

As the result of this scan, the page gets included in the search engine's index for some word combinations from its content. If someone makes a query to search websites for exact same words, the engine may consider to include the page in its result list.

The position at which the page is presented in this list is a mystery. Pages from larger and older websites with many inbound links tend to rank higher, but there is no guarantee. The pursuit of a higher ranking with search engines gave birth to a whole industry known as search engine "optimization".

"Manipulation" would have been a more proper term, even if with some negative connotations. For my part, I don't consider reasonable human reaction to an arbitrary piece of software unethical.

Google Search Console doesn't even list queries my pages get impressions for, deeming them too obscure and personal. Last month I got clicks from search pages for "best time's to travel to antarctica" and "snæfellsnes peninsula day trip reykavik gray line 18000 isk". Repeating them here may give engines a good reason to include this very page next time somebody tries to search websites for the same.

As Google keeps track of both queries you make and pages you visit afterwards, your search results might very well be personalized according to your browsing preferences.

5 tips to search websites

When you search websites, at least some of the results will be in the list form. From 7 Deadly Sins to Fifty Shades of Grey, countable sells. Here's my take on it.

1. Ask the right question

Not all queries are the same.

“In order to ask a question you must already know most of the answer.”

Robert Sheckley

Queries for "brexit leave" and "brexit remain" will probably reveal different results about, not of, the recent British referendum.

2. Skip the Top 10

Lists are a huge phenomenon of our time. Just have a quick look at the Internet, including this page.

I find it fascinating how modern search engines are able to compose a top 10 list out of what seems like 75,000,000 or so hits an average query returns, all in a mere fraction of a second.

Occasionally, I am really surprised how spot-on the very top link on the first result page is. For the rest of the time, I am amused and/or mildly irritated by how irrelevant all results from the first page look to me.

An online article I read recently makes a similar point by stating that we are happy with the Top 10 presented to us "largely because we're unaware of the better results that are often buried out of sight."

“...lists are a form of cultural hysteria...”

Don DeLillo

Google developers often stress that the first page of results is composed to reflect as many search intents as possible. It can mean that only 2 out of 10 presented links fit yours.

Don't limit yourself – scroll beyond the first page.

3. Question the answers

You can challenge search results found by one search engine asking several others the same question. The more often a specific page comes up in the result list, the more relevant it may proove for your search intent.

4. Be patient

If you are unhappy with found websites the first time around, modify your query and try anew. Repeat until succeeded.

5. Consult a human

If in despair, ask a real person working in the field you are researching. Chances are, a kindred spirit has a better insight and understanding of your problem than an one-size-fits-all algorithm.

“Google can bring you back 100,000 answers. A librarian can bring you back the right one.”

Neil Gaiman

An afterthought

I hope I'll live long enough to see someone push Google out of relevance. It's not as foolish an idea as it might seem. It happened before, it happens every day (Internet Explorer, anyone?)

"Someone" can be something, however, I would prefer "it" to be a person rather than the European Commission (for example).

How about a public rating system for webpages? Visitors would rate them for content keywords, awarding thumbs-up or -down, or everything in-between.

Use the votes to search websites by keywords. Pages with the most ups should rank higher – simple.

This can be easily implemented with a browser add-on. Something similar to WOT (Web of Trust), maybe even the same platform?

Just one thing, keep the bots out – add a captcha. And don't forget you read it here first!

An "after sought"

This is a short introduction to what shall become my guide to some remarkable websites I consider worth visiting. Most of these websites are hard to search for and even harder to find. They are my private "search websites" results.

Consider the links below as the beginning.

Further publications in this section will be first made available to those who subscribe to newsletter. (Hint: The subscription form is on top of the screen.)

Tags: #humansarebetter #webwatch #caughtontheweb

In this series

Custom site search with DuckDuckGo
Build your own custom search widget for your website or blog! Use the easy to follow steps to generate its code and test it on the fly—within minutes.

Previous newsletter issues

Access the current issue when you subscribe!

07.17 – The numbers issue
As numbers tend to dominate our everyday lives and dictate decisions we make, let's take a look on how they come about and what they really mean.
06.17 – The Photoshop issue
Join me as I scan my images and edit them in Photoshop, and learn some funny facts about the application while at it.
05.17 – The Norway issue
Follow me along the coast of the happiest country in the world, all the way around the North Cape to the Russian border—and back.
04.17 – The retro issue
This issue is about the past, the longing for it, and the ways to satisfy this longing.
03.17 – The marketing issue
This issue is about creating your brand and marketing your work, told by those who've been there and done exactly that.
02.17 – The travel issue
The issue's topic is finding your way on the Web, and in one, quite literally.

Links

Ask a Foolish Question, by Robert Sheckley
An early science fiction short story about the limitations of artificial intelligence in general, and search engines by induction. Available online and in several e-book formats for download.
Still? How to Beat Google With This One Simple Trick, by Ken Evoy
A modern variation on the same topic: our search tools may be not without fail, as they try to convince us. It is a good, thought-provoking read, and well worth 10 minutes of your time.
WOT (Web of Trust) browser extension
Check websites, or install WOT in your browser. See the reputation of a site along the link to it in search results, social media platforms, and webmail before you make that click.
Robert Sheckley author pages at Amazon
Browse at Amazon.com
Browse at Amazon.ca
Browse at Amazon.co.uk

Web watch

Back to Top

Tell me what you think!

Is it useful 👍? Awful 👎? Leave a message! Your comments help making this site better (and give me a kick—one way or another).

Popular articles

  1. "Might as well have the best"

    Aiming for better images? Think better lenses! This is your most important piece of gear, so you better get it sorted out.—Need some advice?

    Read on

  2. A duck for a dog

    If you got your own place on the Internet, helping your visitors find what they are looking for is a great way to engage them and keep them staying a bit longer. A custom site search can achieve just …

    Read on

  3. Perfectly legal

    Going on a trip and bringing a camera along? — Great! However, be sure to know what you can photograph at your destination without breaking the law. The short answer to what is legal, is: it very much…

    Read on

Back to Top

What's new

  1. Outnumbered

    Aug 31, 17 04:48 AM

    We live by numbers. We make decisions based on them. We study ratings, statistics, and trends. We compare prices, count stars, and draw charts — first, then conclusions. Everything we do in the end …

    Read on

  2. Photoshopping

    Jul 21, 17 05:55 AM

    With almost 30 years of market presence under its belt, Photoshop is one of the most popular and powerful image editing software packages. From the beginning, it has become the tool of choice for many…

    Read on

Featured article