Content analysis is a systematic, quantitative process of analyzing communication messages by determining the frequency of message characteristics. Content analysis as a research method has advantages and disadvantages. Content analysis is useful in describing communicative messages, the research process is relatively unobtrusive, and content analysis provides a relatively safe process for examining communicative messages, but it can be time-consuming and presents several methodological challenges. This entry identifies several advantages and disadvantages of content analysis related to the scope, data, and process of content analysis.
The scope and advantage of content analysis is as a descriptive tool. Content analysis can be used to describe communication messages. Content analysis focuses on the specific communication message and the message creator. It is often said that an advantage of content analysis is that the message is “close to” the communicator; that is, content analysis examines communicative messages either created by or recorded from the communicator. Researchers can examine the manifest (the actual communicative message characteristics) and latent (what can be inferred from the message) content of a message. Researchers can use content analysis to study communication processes over time. For example, a communication scholar might be interested in the metaphors presidential candidates have used in speeches during war time.
While content analysis is used to describe communicative messages, content analysis cannot be used to draw cause-and-effect conclusions. Identifying and describing the characteristics of a message is not enough to make claims of what caused or was caused by a message. Content analysis can, however, be combined with other methods to make causal claims, or the description developed through content analysis can be used as a starting point for future causal research. Describing the messages a mother uses to deny a young child’s request is not sufficient to determine the child’s behavior, but combined with other methods (e.g., experimental methods), the child’s behavior could be predicted.
Content analysis is a beneficial research method because of the advantages in collecting data and analyzing quality data. Content analysis can be applied to various types of text (e.g., advertisements, books, newspaper articles, electronic mail, personal communication), and therefore is useful for studying communication from a variety of different contexts. Many times, content analysis can be conducted on existing texts, and therefore the work of collecting data may be minimal (though searching through decades of newspaper articles is a time-consuming process as well). Since content analysis can be used to study communication processes over time, it is useful for studying historical contexts, because describing messages over time can help researchers identify trends in messages over time and subsequently explore the historical context in which the messages changed.
Content analysis also benefits from the data, or communicative messages, coming from the source, or communicator. Data straight from the source relieves several methodological issues (which will be described in greater detail later in this entry). Additionally, data is often readily available for content analysis. For example, print resources are already in an analyzable format, as is written correspondence. Transcripts of videos and radio shows, music lyrics, and the like are readily accessible on the Internet. Also, many texts (e.g., newspaper, books) are available for public consumption and therefore access to texts is easier, making research using content analysis relatively unobtrusive. Once the message has been shared, the researcher only needs the data, and not the source, to conduct the analysis. This is an important benefit of content analysis, as many analyses can bypass human subjects boards because the research neither involves nor affects actual participants; however, some content analyses require data collection from human sources and must therefore receive appropriate approval before the data is collected and research is conducted. Content analysis also affords researchers richer data; that is, because actual communicative messages are collected and analyzed, researchers are exposed to more detailed data than they could obtain through survey research, for example.
Analyzing recorded (e.g., audio, video, print) communicative messages helps to prevent two disadvantages that are characteristic of other research processes: participant recall and recall bias. Some research methods (e.g., interviews, focus groups, diary method) ask participants to recall a situation and what was said and either share the story verbally or write out the account. Research shows that individuals’ abilities to recall information accurately, even a short time after the communicative exchange, are very low. Content analysis uses recorded data, and therefore avoids the issue of misremembering. For example, analyzing the actual discussion will be more accurate than asking a participant to recall what was said and analyzing that response. Additionally, content analysis avoids the issue of recall bias. Oftentimes, participants in the same situation will recall the situation and the communication messages differently. This is frequent in many communicative situations, but particularly common in conflict communication. Because content analysis uses recorded texts, discrepancies between participant accounts are avoided and the data are arguably objective. However, content analysis does not avoid recall bias between the researcher and the participant. One advantage of content analysis is that it removes human participants from the process; however, when a text alone is analyzed without feedback, input, or reflection from the participant, a researcher may misinterpret the latent content of a message; that is, the researcher may misinterpret the intention of the message or infer a different meaning from the message. This is particularly troublesome when analyzing content between close individuals who may frequently use personal idioms in conversations. Without the participants, the researcher is left to analyze the manifest content and may misinterpret the message without considering the latent content imbedded in the idioms.
Analyzing recorded messages has specific advantages, but two major issues arise. First, content analysis cannot study what is not recorded; therefore, if a speech, conversation, or other communicative message is not recorded in some way, the message cannot be analyzed. This could include either an entire population of potential texts (e.g., a series of speeches never recorded, conversations not recorded) or parts of the population (e.g., missing a volume of a newspaper, or a film in a series) that may be excluded from analysis, leaving the population of texts incomplete. Second, content analysis can miss key “real-time” features from the communicative exchange. Because content analysis is focused on the specific communicative message, and analyzed texts and characteristics of the message, important aspects to understanding the message can be excluded. This is particularly true of nonverbal communication, including body language, eye contact, inflection, and the like, which cannot be considered in the analysis. This is troublesome because just as the communicator has insight, the nonverbal communication provides insight as well. For example, a researcher may interpret the manifest content in one way, but miss that the message was delivered sarcastically and therefore should be interpreted differently.
Content analysis also has benefits as a process. Specifically, content analysis is a relatively “safe” process. In many research processes, if an error is made throughout the process, a project may have to be terminated or the researchers may have to start over with a new sample. Because content analysis examines texts and is removed from the original communicators and their potential to bias the process, errors are fixed more easily and entire projects are not lost. Say, for example, two researchers are coding the messages and realize they are coding messages differently. The researchers can go back to the text and recode based on the specific error. However, if a survey has a fundamental error and is distributed to a sample of 400, the survey has to be fixed and distributed to a new sample of 400, which can be arduous, time-consuming and sometimes quite expensive. Repeating part of the process in content analysis tends to be easier than in other projects, and relatively less costly and time-consuming.
Though content analysis is a relatively safe process, the process has its disadvantages. First, because content analysis analyzes texts, finding a representative sample may be difficult. Researchers identify several issues in finding representative samples: searching through newspaper articles and other data sources is time-intensive, transcripts may not be perfectly accurate, researchers might select convenience samples and miss key pieces of data, and access to particular texts may be restricted, to name a few.
Second, coding issues in content analysis make it difficult to generalize across content analyses. Researchers studying the same variable may operationalize the variable differently and therefore code the results differently, and therefore it is difficult to make inferences across studies. For example, researchers studying compliance-gaining or influence messages could use different typologies of messages to analyze conversations. When different coding categories are used in different studies, dissimilar codes can make it difficult to generalize results across studies.
Third, content analysis can also be time-consuming, complex, and labor-intensive. For example, audio recorded messages often need to be transcribed before analysis is conducted. In other cases, a population may be every newspaper article written on a specific presidential election, and while those articles would not need to be transcribed, the sample would be quite large. Collecting a population and/or sample of such an extensive collection would take a great deal of time, particularly if permission was needed to access the material. Coding and analysis of a large volume of material would also be time-consuming.
Finally, other major issues emerge when conducting content analysis related to coding. First, researchers may code messages too narrowly or too broadly. Coding units should be exhaustive and all coded units should fit into a category; however, sometimes coding units are too narrow and important nuances of the message may be missed. It is important for researchers to remain attentive to their research questions and hypotheses to avoid coding too narrowly or too broadly. Using coding units that are specific words, rather than phrases, could affect the interpretation of the message, depending on the purpose of the study.
Other issues are related to coding reliability and validity. Content analysis utilizes multiple coders, and intercoder reliability, or the amount of agreement between coders on coding decisions, is important for the results of the analysis. In content analysis, intercoder reliability is calculated for two different types of coding decisions: unitizing reliability and categorizing reliability. Unitizing reliability refers to the amount of agreement between coders on what is to be coded. Unitizing reliability is typically fairly high when units have natural beginning and ending points; for example, a sentence has a clear beginning and end. Coding is more difficult when there are not clear sentences, or beginning and ending points. Coding units like phrases in a conversation, themes, and stories is more difficult. After coders have identified units, each coder separately decides in which category to place the unit. The more the coders independently place units into the same categories, the higher degree of a second type of intercoder reliability, categorizing reliability.
Intercoder reliability can be measured using a number of different statistics, and the particular statistic should be chosen based on the nature of the coding. Percent agreement is the easiest, and therefore most popular, measurement. Imagine two coders, A and B. Coder A codes a unit “1” and coder B codes a unit “1.” The two coders assign the same code, and therefore, there is 100% agreement. If Coder B had coded the unit “2,” the coders do not assign the same code, and therefore agreement is 0%. In a scenario where three coders, A, B, and C, code a unit 1, 2, and 2, respectively, percent agreement would be 33.33%. One of the three pairs (A/B, B/C, and A/C) agrees on the code, but the other two pairs do not, and therefore there is one-third agreement. Percent agreement is useful for diagnostics during coding, but is not sufficient alone for publishing results, and therefore, other statistics are widely used. Scott’s Pi and Cohen’s kappa are two other statistics used specifically if there are only two coders. Both statistics improve upon percent agreement by including a calculation for chance in their equations, which compares observed agreement with expected agreement. Fleiss’s kappa is similar to these statistics, but is recommended for projects with three or more coders. The final statistic, Krippendorf’s Alpha, is the most reliable, but most complicated statistic. It is recommended for three or more coders, but differs from Scott’s Pi, Cohen’s kappa, and Fleiss’s kappa because rather than measuring observed and expected agreement it compares observed and expected disagreement.
In order to improve intercoder reliability, researchers must have clear operational definitions, code carefully, and carefully train coders on how to categorize message characteristics into coding units. Although reliability is a concern, content analysis is one of the most replicable research methods. Validity is another concern, or whether the coding scheme fits with the desired message analysis (i.e., measures what it is supposed to) and whether the coding scheme is parsimonious, or simple enough to explain the communicative phenomenon. One way to ensure parsimony is to examine the “other” category to determine if the category is too broad and contains important data. Using the “other” category as a catch-all may mean valuable categories or units are neglected. Content analysis is criticized by some scholars who say that the process of coding and counting frequencies of messages is too simplistic and therefore does not provide a thorough analysis of communicative phenomenon.
In summary, content analysis is useful as a descriptive tool, has broad application, is relatively unobtrusive, and is a fairly “safe” research method. Content analysis can be time-consuming, labor-intensive, limited by available texts, and can present challenges to study reliability and validity, but ultimately is a useful heuristic tool for future research and as a method for describing communicative messages.