6 Ideas For Better Search Results

Searching on the web is deceptively easy. Pop a word or two into the search bar and like magic you get a list of results. In fact, you get millions or billions of results. What do these results mean to you? Are you really going to look at millions of results to find what you need?

Be honest, you as in an individual, are not going to look at even a fraction of those results. Maybe you will look at the entire first page or keep scrolling to what appears to be a second page. So how do you figure out how to find which sites and pages are worth reviewing?

You can assume that the most relevant results are on the first page and go no further. That may once have been a reasonable approach. Unfortunately, as time goes on it is less reasonable. Increasingly, the results you get and the order of appearance are a reflection of a number of factors some of which are in your control and many that are not.

The quality of your search input matters but so does the “pay to play” policies of the search engines, the search engine optimization techniques used by sites to boost their results and the technology employed by the search engine including algorithms and artificial intelligence. What can you do to improve the chances of getting more relevant results?

Basically, there are two areas you can control, at least somewhat:

• Input – learn to craft and refine your inputs into the search engine to improve your output; and
• Output – use techniques, offered by the search engines, to filter results to more easily find those that are relevant for your diligence project.

Today I focus on 6 ways that you can improve input or output.

This is an input idea.

Each industry, field or topic has a specialized vocabulary associated with it. These words are referred to as “terms of art”. The definition of a particular term of art changes based on various factors including the industry, topic even geographic location.   When I research a topic, I am not familiar with I do a preliminary search to find the terms of art.

I call this a “pre-search”.  The pre-search is a quick way to gather official and widely used terms, acronyms, the names of organizations associated with the topic and laws and regulations that are are relevant to the topic.  The pre-search isn’t about finding the best sources, or even carefully reading the source material (I skim it). It is about developing a list of vocabulary (terms of art) associated with the topic.

My goal is end up with a list of words, phrases and acronyms that are associated with the topic. Once I have the list I can do a more precise and comprehensive search using these words.  

For example, if when I did a “pre-search on the word bribery I came up with a:

• list of words that includes, anti-bribery, corruption, anti-corruption, kleptocracy
• a US law that is officially named the Foreign Corrupt Practices Act (FCPA)
• organizations such as Transparency International and the OECD.

I can then use these words to craft a more precise and targeted set of searches, for example, “enforcement of the FCPA” or “Is there kleptocracy in the US”.

Another input idea.

We are used to informality in our speaking and writing. This means that while we may be familiar with an abbreviation, a media or public relations nickname, a brand name or even a “doing business as” (DBA’s) we may not know the formal legal name of the entity behind the informal name. Also, if we are dealing with the translation of a foreign name we may have no idea how far off the name is from the original language name.

This presents a problem for our research. The legal name of an entity is important because a search that omits the formal name may miss:  

• foreign language sites (even if translated) which use the entities non-English legal name
• government sources, business reports, news and other official or formal resources that use the full legal name of an entity
• legal, regulatory and compliance actions, lists and warnings that use the legal name of the entity.  

So what do you do if you are not sure if you have the correct name. Again, the answer is to do a pre-search. A quick search to uncover the legal name of the entity that you need to research. If it is a foreign entity be sure to find the name in the original language.

While you are doing this pre-search make a list of all other forms of the name including other nicknames, trade name, abbreviations, brand names etc. so that you have a list of name to use for your research.

This works for input.

While the development of Boolean search logic for research predates the invention of the Internet, Boolean operators (words) and modifiers (symbols) can be used on search engines such as Google and Bing. However, how they can be used and the effectiveness will vary. Check the search instructions for your search engine of choice.

When you are doing a Boolean search you are including certain words or symbols that tell the search engine the logic it should use in returning your results.

Here are two common Boolean operators:

• If you use quote marks around your search terms the search engine knows to return search results in which all the words in the quotes are together.  This is really useful when you are searching a person by name and want the first, last and middle initial to be present in the results.

• Another useful operator is the word and. When you place the word and between two words the search engine returns results that include both words that you submitted for the search. The words don’t have to be next to each other (use quotes as a modifier to get those results) but both words must be in each search result.

There are additional operators and modifiers so for a refresh, or to learn more, take a look at the University of Toronto, University of Alaska, University of Illinois, New York Public Library. Here is a Boolean search tip sheet.

If you interested in adding Boolean logic to your search methods there are many online resources including videos.

Filtering is a way to limit the results that you get back so it is an output idea.

Google – When you search using Google and receive results, below the results is a menu:

• Images
• News
• Maps
• Videos
• More
• Tools

This menu allows you to filter the results by type. After all, why look at news if you need images or at maps if you need videos. Other search engines also have filtering functionalities so take a look at your search engine instructions.

Bing – When you search Bing and receive results there is a menu bar:

• Images
• Videos
• Maps
• News
• Shopping
• More
• Tools

This menu allows you to filter the results by type. There is also a “Copilot” choice. Copilot is Microsoft’s AI chatbot which you can use to ask natural language questions relating to the search that you have conducted. Including helping you narrow down the results.

A way to filter the output.

Your project may require sources that are limited to a certain time period. For example, you are looking for current news articles or reports only issued within the last year. Alternatively, you may only be interested in material from past years. 

If you don’t want to wade through results that don’t fit your time frames you can, at least on Google and Bing, filter the results of a search based on time.

Google – go to the search results menu that appears under the search box, the same one that you use for filtering types of content. Choose “Tools”. Then choose “Any Time” and in the drop down menu pick your time frame for results.

Bing – it is much the same. After you perform a search you will see a menu below the search bar. Choose “Tools” and then you will see “Any Time”. Again, choose this option and a drop down menu will appear that will allow you to choose a time period.

Finally, if I want to do a quick search, and I have flexibility on the accuracy of the results I add the year to the search term. This works as a filter. For example, if I need recent articles about bribery, I search “Bribery 2024”. The search results are displayed with those dated or referencing 2024 at the top.

I have had many frustrating experiences with internal search functions. I use the search function and get no results but I am sure the material I want is on the site. This frustration led to experiment with different search strategies. I discovered that if I used an external search engine such as Google or Bing and did a search that included the name of the website and my search terms, I get results from within the website.

This works so well that I now search externally to find information internally and it turns out that there is an official way to conduct this type of search.  

How?

Google – has a search input page Advanced Search. This input page includes the option to enter a domain address. This limits search term results to those found only on that site.

It is also possible to do this by putting, the word site before the URL address and adding search terms. For example, if you want to see what the OECD has on its website for the topic bribery but restricted to the year 2022 the search string:

 site:oecd.org 2022 bribery

will result in Google search results returned with OECD website pages from 2022 that have the term bribery included at the top of the results

Bing – is similar in that you can also use the “site” instruction to limit searches to a particular URL. Microsoft has instructions about how to do that on its How-to guides.  

More

• You can combine these ideas, so for example you can use terms of art for the search and filter the results by date range.
• If you are not familiar with these techniques practice them, see how they impact your searches
• I used the two major search engines as examples but if you use a different search engine, look for the instructions that apply to the search engine that you prefer.
• These are only 6 tips, there are many more. You usually find these types of tips and instructions in the search engine’s help section, advanced search instructions and guides.
• Get curious, think about what might be useful for you and see what is available. If you want a certain feature it is possible that millions of others did as well and it exists!