The Invisible Web & College Research
The invisible web holds innumerable resources that lie beyond the reach of common search engines. But before diving into the invisible web, it’s important to learn what exactly it is and how to use it. Below are some handy definitions and distinctions that will help make searching the invisible web easier and safer.
The internet most people use daily and what search engines are able to index.
The majority of the internet. Pages within the invisible web can’t be indexed by search engines for various reasons, including because they aren’t linked to by any other page or they’re behind a paywall. The invisible web is also called the “deep web”
A small section of the invisible web that is anonymous and often the location of illegal activities. It’s sometimes called “Tor networks” and “onion pages”.
Why should students use the invisible web in college?
Can reduce the number of non-academic results
Finds resources that major engines like Google, Bing and Yahoo miss
Includes academic journal articles, which major search engines generally exclude
Frequently provides more specific and relevant results
Features databases which allow advanced search and filtering options
May include raw statistical, numeric and/or scientific information (such as census data)
Allows for faster research
How to Use the Invisible Web for College Research
Students accustomed to using Google and other surface web search engines may not know how to safely access the invisible web. Finding invisible web resources without a starting place can be difficult, so continue reading to learn tips and guidance for reaching and using the invisible web for college research.
How to find the invisible web
Although search engines such as Google don’t pull invisible web entries into their search results, they can be used to find entrances to the invisible web. For example, if a website requires a user to register before accessing its database or archives, those archives will likely not appear in a Google search result. However, the website’s main page might.
Students can also find the invisible web by using resource lists published by their university library, listed in textbooks and even suggested by professors.
How to browse safely
The basic internet safety rules used on the surface web also apply in the invisible web. Use good judgment when downloading items and when registering for websites. See the FBI’s Simple Steps for Internet Safety
for more information on the standard safeguards for using the internet (surface and invisible alike).
Avoid the dark web. Illegal activity frequently occurs on the dark web and government agencies try to monitor its content and those who access it. Do not use Tor networks or visit websites that require you to hide your IP address.
Remember that academic journal articles often cost money to access. Services like Sci-Hub and Library Genesis that promise free access to ebooks and articles from a variety of academic journals are considered illegal in the United States
Stay focused. Following unrelated but interesting links and advertisements can lead into a rabbit hole of non-academic sources, unwanted lewd content and even viruses.
Invisible web search tips for college students
Use the right keywords.
Don’t use complete phrases or sentences when researching; instead focus on nouns.
Use search modifiers and operators.
These tools can help refine results in databases, academic journal searches and most other places with a search bar. Google provides a rundown of some basic Boolean language it uses that may also be applied to databases and other search engines on the surface or invisible web.
Unpaywall is an extension that can help find free (and legal, unlike Sci-Hub and Library Genesis) copies of academic articles; it is available for Google Chrome and Firefox internet browsers.
Keep track of helpful websites.
Not only will this make citing sources and writing footnotes indescribably easier, but students can also return to these resources for future projects.
Databases & resources to search the invisible web
Department of Energy Pages
A database of articles funded by the US Department of Energy
A database of accessible global development studies research and articles
A database of financial and demographic information
A database of available Earth Science journals and articles
A multidisciplinary search engine for academic journals and articles
A database of academic journals and sources. Items listed in JSTOR are generally three years old or older.
A directory of Medieval Studies databases, journals, and more
Library of Congress
The largest library in the world; its website includes a catalog of holdings and extensive fully-accessible digital collections
Directory of Open Access Journals
A directory of accessible academic journals from nearly every discipline, many with downloadable articles
A search engine for academic sources; Microsoft Academic uses semantic search instead of traditional keyword search
A database of digitized United States historical documents
Penn World Table
A collection of international income and purchasing power data. The most recent entries are from 2007.
A database of plants native to the US; hosted by the United States Department of Agriculture
A collection of over 50,000 free ebooks, the vast majority of which are part of the public domain
A catalog of humanities and social science journals and books. Access to these is dependent on whether the student’s college or university subscribes to MUSE.
A database of biomedical citations, some of which link to fully-accessible articles and resources
A database of US Department of Energy scientific and engineering data
An archive of preserved webpages
A deep web search engine that functions similarly to Google and Bing. It also filters out adult content for a de-cluttered search experience.
A catalog of participating libraries’ holdings; students can use it to track down accessible copies of elusive resources
A database of scientific databases across the world
How to Evaluate a Resource for Academic Research
Finding resources is only the first stage of academic research. In order for college students to know how reliable and trustworthy a resource is, they must evaluate their findings before using them. Below are some questions to ask before using a resource.
Who wrote or produced the resource?
Academics and individuals with relevant degrees will know more about the topic and have more authority in the field than others.
Where was the resource published?
Colleges and universities produce academic research, so websites that end with “.edu” are often more reliable than “.com” sources. Government sources (which have websites ending in “.gov”) are also typically considered trustworthy. Use articles published in academic journals instead of those published in newspapers, magazines and similar popular outlets unless the research topic requires otherwise.
If the resource is an article from a journal, was it peer-reviewed?
Peer-reviewing is the process by which experts in a field review each other’s work before it is published. Typically, college students are encouraged to use only articles which have been peer-reviewed.
When was the resource published?
Depending on the topic being researched, something published even just a few years ago may be considered outdated.
Who was the resource written or produced for?
An article intended for individuals outside of the field or for children will be more general and less nuanced than an article written for fellow experts.
How academic does the resource appear?
Look for resources that reference and cite other pieces of research. Additionally, students should be wary of sources that use overly casual language.
When in doubt, reach out. When an article is indispensable for a student’s research or project but does not pass the majority of the above tests, students should ask their professors, course teaching assistants and reference librarians for guidance.
Other Research Resources
Although 21st century college students conduct a significant portion of their research using the invisible web, traditional methods shouldn’t be ignored. Libraries, librarians, note taking and physical books remain vital and powerful components of a student’s toolkit.
Below are some of these visible web and physical world resources that students can benefit from.
Although it still uses the surface web, advanced searches on popular search engines can help users significantly narrow down their results and find websites more specific to their needs. Google and Yahoo both have advanced searches available.
Evernote’s free Basic edition is note-taking and -organizing software that syncs across all platforms. It is available for desktops, tablets and smartphones.
Google Books sits in the grey area between the surface and invisible web. Regardless of where it belongs, however, Google Books can be an excellent resource for students. Google Books is an index of over 30 million books. While a significant amount of them only allow users to view a smattering of pages, students can still use Google Books to find titles and determine if they will be useful enough to request from a library.
InterLibrary Loan (ILLiad)
Over 16,000 libraries in 120 different countries participate in the Online Computer Library Center which hosts a variety of programs centered on sharing resources between member institutions. Students can especially benefit from ILLiad, the system through which participating libraries can loan each other materials for their patrons. If a local library does not have a resource, librarians may be able to help students get it through ILLiad. Some libraries also participate in regional loan programs which can help students get materials more quickly.
OneNote is available free for both Mac and PC; students can use this program to save and organize screenshots of research and to take notes.
Reference librarians are individuals employed by a university or library who specialize in information literacy. This means they know a wide variety of resources available–digital and physical–and how to access them. Reference librarians at colleges and universities may specialize in certain academic fields and can offer one-on-one assistance.
Zotero can be used to save, organize and export online research. This program can also connect with Safari and Chrome browsers.
Now that you’ve done the research, here’s some additional help on writing college papers.