Welcome — Reading Research, Designing Research
Welcome to Week 4 in the Research Methods in Social Science course for the University of Maine at Augusta. This online lecture is a companion to your textbook reading for the week and your problem set work. Be sure to read Chapter 4 (on designing research). Why have I added a section on reading research to the lecture? The reason for this unique pairing is that reading research and designing research really are not all that different. In each experience, the last thing you want to do is approach the experience from (reading or writing) the first page of a research article to (reading or writing) its last page. In both reading research and designing research, it is essential that you begin with identifying a research question, move to variables and method, and let the rest of the work (of reading or writing) flow from these.
Reminder: Midterm Exam Coming!
As you already know from having read and agreed to our course syllabus, all exams in the course are completed using pen and paper and will be proctored, requiring you to travel to an ITV site, University Center or another location near you to take the exams. During the first two weeks of the semester, as the syllabus explains, you should have visited the website http://www.learn2.maine.edu/exam and chosen a location at which to take your exam (or, if you live out of state, you should have already made arrangements with a proctoring service and then contacted me with specific information about those arrangements).
It is also your responsibility to contact the location you’ve chosen, before each exam, to arrange a specific day and 3-hour block of time within the period October 24-28. The day and time you schedule needs to be practical for you but also for site staff, so be sure to allow for some time to find the mutually agreeable slot you and they can work with. If you need help contacting any UMA campus or Maine center to arrange a specific day for your exam, you should have let me know by e-mail (firstname.lastname@example.org) by now — again, all these details are laid out in the syllabus.
How to Read and Evaluate a Research Article
Research Articles: Your Gateway to the Professional World
As an undergraduate student and a high school student before that, you may be familiar with textbooks as the dominant mode of written instruction. But in both the academic and professional sphere, social scientists most commonly accumulate and disseminate knowledge through research articles published in journals. Social scientists subscribe to prominent social science journals such as the American Sociological Review, Social Networks, and Criminology so that they can maintain an awareness of knowledge in their fields. When conducting their own research, they also may review the literature on a subject by searching journal articles through free services such as Google Scholar or paid services such as JSTOR and Academic Search Complete (available to you at the UMA Library Database page). Your journey as an undergraduate student should take you from lower-level classes dominated by textbooks to upper-level classes that incorporate a significant number of professional research articles.
By the time you have graduated with a Bachelor’s degree, you may not understand every method employed in professional research. Nevertheless, you should be able to read, understand, and usefully react to the theory and knowledge contained in a research article. Reading a research article is quite unlike reading any other form of writing. It’s a practiced skill that requires patience, repeated effort, and the correct technique. The patience and effort in learning to read a research article must be supplied by you; this reading is designed to introduce you the necessary technique.
Let’s begin by identifying the elements of a research article. Once we’ve identified these elements, we can think about how to read through those elements effectively.
Elements of a Research Article
Not every article in an academic journal is a research article. Some articles will focus entirely on theory or review of previous literature. Some articles are book reviews. Yet other articles are commentaries on previous research. An academic article is a research article if it contains an abstract, an introduction, a literature review, a methods section, a description of results, and a concluding discussion. If your article does not contain these elements, you most likely aren’t looking at a research article.
In just a few sentences, an abstract concisely describes the contents of the research article so that a reader may quickly determine whether the article is relevant and worth a closer read. The abstract identifies the question motivating research, names the essence of the data and methods used in research, and describes the essence of research findings. In the final sentence of the abstract, the authors often identify the broader significance of their research.
Introduction and Literature Review
The format of these sections may vary from journal to journal and from discipline to discipline. At times, a literature review is incorporated under the heading of an “Introduction.” Regardless, almost every research article contains an introduction and a literature review. An introduction explains why research is being carried out. That reason may be substantive (a real-world problem that needs to be solved), theoretical (the existence of two or more different theories that predict differences in research outcomes) or methodological (that previous research has gone about studying a subject in the wrong way).
Task: When you are reading an introduction, always look for the outcome of the study. What feature of social life are the authors attempting to explain? For instance, authors might be trying to explain the outcome of obesity: why do some people become obese?
In the literature review, the authors of the article most often note the prior studies that have attempted to explain the same outcome or similar social outcomes. In this section of the paper, the authors begin to treat the outcome of interest as a dependent variable. A dependent variable is something related to an outcome that can be observed in the social world and that can take on different values. For instance, a person’s weight is something that can be observed in the social world that related to obesity. It can take on different values (indicating higher or lower weight). What makes a person have higher weight? What makes a person have lower weight?
The most traditional approach of a literature review is for an author to name a single dependent variable (like weight), then look through earlier research articles and describe the independent variables associated with the outcome. An independent variable, like a dependent variable, is something that can be observed in the social world and that can take on different values. Independent variables are important because when they change, the dependent variable may change too. For instance, someone studying obesity may find a prior research article suggesting that the amount of food people eat leads to how much people weigh. In this instance, weight would be the dependent variable and amount of food eaten would be the independent variable. In this approach, you should expect to find reference to one dependent variable and multiple independent variables.
A less common approach in a literature review is for an author to name a single independent variable (like amount of food eaten), then look through earlier research articles and describe various different dependent variables that might change along with the independent variable. In this approach, you may find reference to one essential independent variable and multiple dependent variables.
Task: When you are reading a literature review, look for instances of variables. Based on how the authors describe them, try to determine which variables are independent variables and which variables are dependent variables.
Relationships Between Variables
As a consequence of reviewing previous research, the author of a research article almost always has some informed guess about the relationship between the independent variables and dependent variables of interest. This informed guess is referred to as a hypothesis, can usually be described in one sentence, and describes one of three possible relationships:
- In a positive relationship, the dependent variable rises when the independent variable rises, and the dependent variable falls when the independent variable falls. Example: “If the amount of food a person eats rises, the person’s weight will rise.”
- In a negative relationship, the dependent variable falls when the independent variable rises, and the dependent variable rises when the independent variable falls. Example: “Higher consumption of the stimulant methamphetamine leads to weight loss.”
- When there is no relationship between an independent variable and a dependent variable, that means that when an independent variable rises in value, the dependent variable does not tend to fall or rise. Example: “No matter how many times little Lucinda jumps up and down, the number of candies her mother provides her each month will remain the same.”
In a graph plotting the relationship between an independent variable and a dependent variable, a positive relationship, a negative relationship and no relationship each has a distinctive appearance:
If you ever asked your geometry teacher in high school why you would ever need to know the equation for a line, here’s your answer. The slope of a line displaying a positive relationship between two variables is positive. The slope of a line displaying a negative relationship between two variables is negative. Finally, the slope of a line displaying no relationship between two variables is neither positive nor negative, but zero. A great deal of research in the social sciences (and the natural sciences, and business, and government, and medicine, and on and on…) can be summed up as trying to determine the slope of a line describing the relationship between an independent variable and a dependent variable. Those who master this understanding are on their way to mastering the real-life circumstances driven by the relationships between variables. A little knowledge can be powerful.
Stating Hypotheses or Engaging in Exploration?
The author of a research paper often states his or her hypotheses explicitly, but sometimes only states the hypotheses indirectly, leaving it up to you to state them formally. If the author is the first to conduct research on a subject, then the author may have no prediction to make at all, and is only engaged in exploratory research; in this case the author should state a research question that is like a hypothesis but does not predict how an independent variable and dependent variable are associated (Example: “How does the price of tea in China affect the fuel efficiency of my pickup truck?”)
Task: When you are reading a literature review, identify the hypotheses or research question. If a hypothesis is present, specify how the independent variables and dependent variables are predicted to relate to one another: positively, negatively or not at all?
While the literature review summarizes previous research findings along with general ideas and predictions, the methods section of a research article gets very particular in describing the design of research carried out by the article’s authors. The methods section should describe:
- The subjects of research, be they people, groups, communities, media, cultural forms, or other social objects.
- The instrument (tool) used to study research subjects — observation, tracking online, interviews, questionnaires, use of existing data, or other methods.
- Precisely what about the subjects is observed. The best research articles will operationalize independent and dependent variables. To operationalize a variable is to describe exactly how values for a variable will be measured. For instance, if weight is a variable, it may be operationalized in a study in units of pounds or kilograms. If obesity is a variable, it may be operationalized differently than weight, since a person who weighs 180 pounds may be obese if they are short but slim if they are 6 foot 4 inches tall. Obesity may be operationalized instead in terms of body-mass index. Finally, the amount of food a person eats may be operationalized in terms of kilocalories of food as indicated on a subject’s food diary.
- The means by which resulting data are analyzed. Such means may be intuitive to you, but may also be very complicated and confusing. At the undergraduate level, don’t worry too much about understanding the method of analysis.
Task: When you are reading a methods section, find out who or what the subjects of research were. Also be able to describe how each variable is operationalized (that is, exactly how the values of each variable are measured in the real world).
The results section of a research article reveals the relationships among variables uncovered in the authors’ study. Because methods of analysis may be very complicated and confusing, the tables, figures and text describing the results of those methods may also be confusing to you. Two tips will help you to understand results better.
First, in quantitative studies tables of results look for columns of data labeled as “coefficients” or “slopes.” A slope describes the relationship between an independent variable and a dependent variable, and like any number can be positive, negative or essentially zero. With few exceptions, if the slope associated with an independent variable is a positive number, that indicates a positive relationship between the independent variable and the dependent variable. Also with few exceptions, if the slope associated with an independent variable is a negative number, that indicates a negative relationship between the independent variable and the dependent variable. Sometimes, however, a slope may be too close to zero in a study for its authors to conclude that the apparent negative or positive relationship reflects some broader truth. In that case, the effect will be called statistically insignificant, and to be safe in such cases the author concludes that there is no evidence of a relationship between variables.
Second, as you gain confidence in interpreting tables of research results, you may be able to understand what a table is telling you without referring to surrounding text. However, it is safest and wisest to read the text accompanying a table to find help in interpreting those results. A well-written results section will communicate the author’s understanding of the statistical significance and positive or negative direction of effect in the relationship between each independent variable and the dependent variable.
Discussion or Conclusion
In the discussion or conclusion section, the author steps away from the particular consideration of the particular relationships between particular variables, and considers broadly what the implications of the research results may be. Were findings consistent with some hypotheses and inconsistent with others? Do these findings contradict, support or extend findings in previous research on the subject? What are the implications of research findings for action in the real world? What sorts of further research are required to develop understanding of the research topic? These are the considerations of the discussion section.
This section is an appendix in which all sources cited in the full text of the article are listed so that an interested reader may check the author’s claims and learn more about the topic.
How to Read a Research Article
Before we consider how to read a research article, let’s first consider how not to read a research article:
- Pick up the article and read it from beginning to end.
- Be sure you understand every word written in a sentence before moving on to the next sentence.
- Throw down the article in despair.
Research articles aren’t meant to be read like other pieces of writing. Instead, they’re meant to be read in pieces by different audiences. Experts will certainly read the entire article, but newcomers to a subject may gain knowledge by only reading a few parts of the article. Remember that the vast majority of readers will be unable to understand some aspects of a article. Your job is to gain as much understanding from an article as possible. Try following these easiest-to-hardest steps in order:
- Read the title to get a rough idea of the subject of the article.
- Read the abstract to understand what the article is trying to accomplish.
- Skip to the discussion or conclusion section of the article and see if you can figure out what the authors found in their research.
- Head back to the introduction and literature review to identify the dependent and independent variables in the author’s research.
- Advance to the methods section and note how each of the variables is operationalized. In the first round of reading, ignore technical details.
- Review the research results, figuring out the type of relationship (positive, negative, none) between each independent variable and dependent variable. In the first round of reading, ignore technical details.
- Start asking critical questions. A few examples:
- do the authors honestly and completely describe the current state of knowledge regarding their research question?
- did the authors clearly describe variables and hypotheses?
- as operationalized, do the variables measure what the authors say they measure?
- do the results described in the results section of the article support the authors’ conclusions?
- To answer those critical questions, reread the paper more closely.
If you proceed in this order, then even if you don’t make it to the final step you’ll have gained some essential understanding regarding the research described in the article. Understanding and successfully reading a research article isn’t a simple task; it’s a skill that has to be practiced for you to become proficient at it. But with practice, you’ll be surprised at how quickly your comprehension of research articles improves. You’ll have taken a significant step toward becoming a scientifically literate consumer of knowledge, independent of major media spin.
Now that you’ve gotten to the end of this week’s lecture, I’ll share a little secret with you: the logical order for writing a research proposal is almost identical to the logical order for reading a research paper. This isn’t an abstract gem for you to absorb: one of your two main tasks in the second half of the semester is the composition of a complete research proposal. Now that you know what the elements of a research paper are, you should be struck by how similar those elements are to the elements of a research design as described in your reading for this week. As a matter of fact, a research proposal can be thought of as the beginning of writing a research paper, right up through the methods section, skipping the results and conclusion, and finishing up with your references.
This implies that to write a research proposal, you might want to try following the same order of tasks that you follow in reading a research paper. The main difference is that you are no longer a passive reader; you are now the author of research. Following these easiest-to-hardest steps in order:
- Play with titles, writing them as an initially rough (and later, polished) expression of the research question of your project.
- Try writing an abstract expressing what your research is trying to accomplish. Note where the holes are in your abstract by including [brackets] as placeholders
- Identify the dependent variable in your research. Read relevant work done by scholars before you to find what independent variables have been used to explain variation in your dependent variable, and write up a literature review to describe patterns in that pre-existing scholarship.
- Use the literature review to identify methods used by previous scholars, and to inform your own choice of method. Incorporate independent variables from the literature review. Operationalize the independent variables and the dependent variable, too. In the first round of writing, ignore technical details. Fill these in later.
- Start asking critical questions of your own work, reading and re-reading each draft as if you were reading someone else’s research article. After you have finished a draft, let it sit for a few days, then pick it up, read it and ask:
- do you honestly and completely describe the current state of knowledge regarding your research question?
- do you clearly describe variables and hypotheses?
- as operationalized, do the variables measure what you say they measure?
- To answer problems raised by consideration of those critical questions, rewrite your proposal more closely and fill in needed detail.
Writing a research proposal is not a project for a day or a week. It’s a process that involves much consideration and much revision, which is why the second half of the semester is devoted to your individual proposal and your group project. There are many steps in both, but if we take those steps just a few at a time, you’ll be amazed at the end of the semester by how much you have accomplished.