If you have ever found yourself sitting at your computer desk, staring at the endless results of your research database with your cold mug of coffee nearby, wondering if you will ever find that perfect article for your research project on page 47… don’t worry, you ‘re not by yourself in this boat – the biggest problem facing a researcher today is NOT finding enough resources to conduct their research, but rather finding ways in which to carefully filter through the mountains of resource materials available to them today. Finding research articles is only the first step; the true skill of a researcher lies in how they develop the ability to quickly and accurately filter through their various publications to determine if the item will be beneficial to meeting the objectives of their study. The art of filtering through source material to determine if the paper meets the expectations of the researcher is what makes up a focused and coordinated screen paper vs. a haphazard and scattered literature review for the researcher. So let’s take some of the anxiety out of the process and begin developing a method for filtering through the overwhelming number of sources available that is effective for YOU and will help turn this overwhelming task into a more rewarding and pleasurable experience.
The Art of the Pre-Screen: Building Your Battle Plan
To find relevant research, you need to start with a plan, just like creating an outline before writing an essay. You won’t be able to find what you want without knowing what it is you’re looking for. Because many of us are tempted to just search for whatever we think might work and hope that by doing so we’ll find something that will help us answer our research question; this approach is very ineffective and will only produce wasted time. To create a plan to help you get the right types of articles to answer your research question, first identify all the major concepts of your research question so that you can form a “lexical net” of words or phrases that: 1) describe each key concept, and 2) include possible synonyms of each of the words/phrases describing each of the key concepts. Write down your estimated inclusion/exclusion criteria using entire words or phrases. Also create your inclusion/exclusion criteria based on what types of articles will help answer your research question? What types of publication dates do you want to accept for articles? Who do you want to be the subject of your research? You should write these on a notepad (paper or digital) and keep that open while using databases. Your objectives will become this list and also serve as a way to keep you from being distracted by other “interesting” videos that are not relevant. While this prep work may feel like it delays your screening process, it will save you many hours of rehashing or confusion later on, thereby turning the paper screening process into a proactive mission instead of a reactive one based solely on skimming.
Leveraging the Digital Gatekeepers: Tools and First-Pass Filters
Once you have your plan of attack, it’s time to start using your digital resources. Using only one research/database is generally not recommended. Use the advance searches of Google Scholar, PubMed, IEEE Xplore, or a website specific to your field, to begin your first automatic screening of articles. Will your keywords work together smoothly? YES! Use Boolean operators (AND, OR, NOT) with your keywords to cast a great net, but intelligently. This is where your inclusion/exclusion criteria are going to kick in, digitally. Use the search filters for date of publication, document type (i.e. review article, conference proceeding), and subject area (when available) to help narrow your research down from thousands to a few hundred. Finally, when the search results finally come up, do not be tempted to read the abstracts immediately. The first thing to do is scan your title and use the metadata to make your assessment. Be ruthless, if a title is clearly outside of your inclusion/exclusion criteria based on topic, method, or context you can mark it for exclusion. The main goal of this high-level screening stage is to facilitate the rapid identification and adherence to the approved protocol for your research. This is similar to how a customs officer performs a preliminary check of passports at the border (or ‘front door’) of a country. The initial ‘passport’ check will result in a set of documents being accepted (i.e., passing the initial visa check) to continue to a more detailed inspection of those documents.
The Abstract Deep Dive: Reading Between the Lines
The abstracts you’ve been left with from your first round of scanning will now need your full attention. This is where you will really begin the manual screening portion of your process. Read each abstract with your criteria document in front of you so you can assess whether it meets specifications as you read. Make sure to highlight or make notes about the objective of the study, the methodology used, the key results, and the conclusions of each abstract as you go. Continue to ask yourself, “Does this abstract reflect what I defined as necessary to my research?” A paper may seem to be relevant based upon its title alone, but if it used a methodology you previously excluded, then it does not meet your criteria. Another paper may relate to the general research area, but if they conclude something unrelated to your research question, then it also does not meet your criteria. Use the methods section within the abstract as a guideline for determining the relevance of the research, as it is usually the best indicator. As you scan abstracts, sort them into three categories—yes, no, maybe. The maybe category would include papers that could be considered as marginal (e.g., may use a related methodology or cover an additional aspect of your research question). Do not become too concerned about categorizing papers into the maybe category now; this will come later on in your process. The aim of this phase will be to generate a good collection of definite “yes” candidates for full text review. Maintain discipline. If you do not have a clear “yes” according to the pre-defined criteria, it will be considered a “no” for the time being. You can always come back and reassess at a later time.
Conquering the Full-Text Frontier: Beyond the Hype
Good news! All your “Yes” papers are now available for full evaluation and are now part of your curated shortlist. You can now organize all of the “Yes” papers into a single folder where the actual evaluation of the “Yes” papers takes place. To begin evaluating a “Yes” paper, you will want to read the abstract and the first and last sentences of each paragraph, and then read the conclusion and introduction of the paper, to establish whether or not the central thesis of the paper is suitable for your needs. Next, you will want to read through the methodology section, to establish whether it was conducted appropriately and to determine any limitations. Following this, you will review both the results and discussion section to determine whether the data supports the thesis and the strength of the evidence. As you look through each “Yes” paper for evidence, also look through the references to identify seminal works in your area of research that may have been overlooked. The other option is to use “Cited by” features from websites that provide the resources you will need to acquire updated articles that cite back to the original paper. As you evaluate the “Yes” papers, remember to take detailed notes systematically. In addition to highlighting each paper read, create a summary of each paper’s contribution (strengths, weaknesses, and relevance to your research question) in a reference management tool such as Zotero or Mendeley. This note-taking process provides an additional and essential layer of accuracy, providing a base of structured knowledge about these papers that will be used for your own research.
Maintaining Your Screening Sanity: Pro Tips and Mindset
Realistically, screening is tedious at any system. Developing sustainable screening habits will require that, with the right practices, you avoid burnout. One of the best ways to accomplish this is to create time blocks (Approximately 45 minutes of initial screening for every abstract, followed by a five-minute break). Our attention and judgment wane with time and fatigue. Secondly, use technology responsibly. Reference manager software will help you store your PDFs and organize notes about them. Some reference manager software applications even allow you to use an AI assistant to help with your initial assessment by providing keywords associated with the abstracts. Do not delegate the assessment of your work completely to an algorithm; consider these tools as assistants, not replacements. Thirdly, keep re-evaluating your “maybe” pile. You’ll know your core “yes” papers well, and you’ll have a better understanding of what constitutes a “maybe.” Lastly, remember that the process of screening is an iterative process; it’s not a linear process (you can go backward). If you evaluate a paper in your complete review and realize that you have a different understanding of the subject, you’ll have an opportunity to re-enter all the relevant databases using a modified keyword search. This is not a failure but rather an opportunity to gain new knowledge. In the end, you have a dynamic, living system that will evolve as you develop each project, making you a quicker researcher who evaluates research more critically each time. Now: take a deep breath, establish your criteria, and begin to screen for research that meets your needs as soon as possible. You will discover that there are a great number of fantastic papers available to you.
