How to Read Journal Articles in Eight "Easy" Steps

How to Read Journal Articles in Eight “Easy” Steps

by Dave Wagner (dvwgnr@gmail.com)

October 2005 (Revision 2)

As you work on assignments and consult outside resources, you will notice a broad range of opinions expressed toward the topics that you are researching. Try to use a critical eye toward practitioner-oriented and even scholarly articles. Here is a simple method for disassembling all research articles to ascertain their usefulness (Flaschner, 2003; Trochim, 2001):

1. Identify the article and scholarly journal in APA style. Why was this article written? What is the driving force or main purpose behind this article? Could the article best be classified as reporting, descriptive, explanatory, or predictive/causal?

2. In a nutshell, what is the article really about? What is the central hypothesis or main proposition that the author is trying to express/explore? Is the main hypothesis a measure of association or a measure of difference?

3. Construct Validity? Is there a flow of ideas from referenced, research literature toward the central hypothesis or main proposition of the article? Briefly explain it. (Contrast that ideal with armchair anecdotes and subjective opinion that may not apply beyond the current setting.) What references were quoted? How many? Are they scholarly or relevant to the subject?

4. More Construct Validity? In the instance of a cause and effect relationship being described (i.e., the reliance on advertising revenue causes online media businesses to fail), look for a description of how the cause (i.e., the independent variable) and the effect (i.e., the dependent variable) are being measured. What is the level of measurement (i.e., nominal, ordinal, interval or ratio)? What is the unit of analysis (i.e., individual, group, corporate, societal)? Do the units of analysis match between variables? Sometimes the article is only about how causes are related to other causes or effects are related to other effects. Try to ascertain how the proposed relationship in the hypothesis(es) was measured or could be measured.

5. External Validity? Look at the sample. Did the author look at a large sample, multiple cases or are the conclusions drawn from one specific instance or no sample at all. Having no sample does not make the author’s conclusion wrong but it does open the point to investigation. Could the sample be generalized to other samples, settings or populations? (e.g. If students were asked to rate the taste of soft drinks, does that apply to other consumers or not?)

6. Internal Validity? Examine the setting in which the data was collected. Observation. Review of documents. Survey. Experimentation. Personal interview. Did the data collection process make sense? Were there any data collected to support the author’s central hypothesis?

7. Conclusion Validity? (Or Statistical Conclusion Validity?) Was there any statistical or qualitative analysis of the information collected that would support the hypothesis? Identify the five steps of hypothesis testing as best you can. Identify the statistic. Are the statistical methods appropriate for the level of measurement of the data?

8. What are the implications of the research and the holes (i.e., opportunities) in the arguments that might lead to future research? Perhaps you can suggest some research to extend the article.

References

Flaschner, A. B. (2003). Touro University International RES610 Advanced Data Analysis Coursework.
Trochim, W. M. K. (2001). The Research Methods Knowledge Base, 2nd. Ed., Cincinatti, OH: Atomic Dog Publishing.

SPSS Transform Recode into Different Variable in MS Excel to Reverse Response Scales

While SPSS allows one to perform some interesting transforms on variables from within the software package, it is often much cleaner to perform data transformations and variable computations from with MS Excel before data Import into SPSS.

While using MS Excel, suppose your response data for a 1 to 10 “Likert scale” is in column B and you want to recode it into Column C:



Use the Choose Function in MS Excel to reverse scale as follows:
1. Enter “=CHOOSE(B1, 10,9,8,7,6,5,4,3,2,1)” without the quote marks into Column C
2. Copy the function in Step 1 down the column for the remainder of Column C.
3. Use Paste Special Values to copy the values in Column C over the function.
Note: The Choose Function uses an “index” scheme to decide how to replace the values. For example, the first index location #1 appears in the function argument list immediately after the cell B1, so a response value of 1 becomes 10, and so forth.

Mastering Basic Statistics

Trust me. You can do statistical analysis. The basics of statistics can be mastered... Forget about those mind-numbing textbooks for a second. Descriptive Statistics are all about what the data looks like. Inferential Statistics are all about whether two sets of data are different or if two sets of data have a relationship.

Descriptive Statistics are a summary of what the sample data looks like, such as the measure of central tendency (e.g., mean for interval data) and measures of dispersion (e.b., standard deviation (SD) for interval data). Data that is dispersed about a mean like the bell-shape is normally distributed (i.e., 68.26%in 1 SD, 95.44% in 2 SD, 99.7% in 3 SD). The randomly drawn sample is best but rarely possible, so a non-random or convenience sample can be used with justification.

When compiling descriptive statistics, you need to know whether the sample data (i.e., level of measurement) is nominal (yes, no, or a label), ordinal (in some kind of order such as doneness of meat: rare, medium rare, medium, or well done) or a number that has order and the value means something (such as "that movie is an 8 on a scale of 10"). You also need to know the unit of analysis, such as the individual, group, organization, or society. Descriptive Statistics tell us what Inferential Statistics we can safely use to draw conclusions.

Inferential Statistics are how we make a decision about the POPULATION guided by what the Descriptive Statistics have told us about the SAMPLE data using probability theory. There are two types of decisions: Measures of Difference and Measures of Association. Measures of Difference (z, t, F, etc.) test differences between a number and sample, two samples or more than two samples. Measures of Association (r, correlation, regression) test whether variables move together and possibly whether there is some causal relationship. (Causal relationships are tricky to prove so be careful about saying X causes Y.)

When applying Inferential Statistics, the types of measures of difference or measures of association that can be used are governed by the level of measurement, the number of samples you are comparing, whether the sample is random/independent, and if the data is tightly dispersed about the mean like the normal distribution. When you are comparing samples, you have to make sure that the unit of analysis in each sample aligns with the other samples and your research question. (e.g., students in a classroom vs. a classroom of students, such as can a single student be judged by being in a particular class or should the particular class be judged by a single student.) Test statistics are calculated from sample data and critical values are looked up on a distribution (probability) table, and you compare these two in hypothesis testing. If you see a low p value, that is good.

All good quantitative research uses variations of the above instances to boil the research question down to a testable hypothesis for a large sample for descriptive, exploratory, or causal/experimental research. All good research articles explain how construct validity (i.e., theory or practical problem), external validity (i.e., how and why the sample was chosen), internal validity (i.e., why they think they saw is what they saw) and conclusion validity (i.e., how the descriptive and inferential statistics support our discussion) are achieved. A sample size of one in qualitative research might use ethnography, action research or other methods to build a case study or foundation for quantitative research.

That's it. That's about all a business manager or MBA must know about statistics. Of course, there is a lot more that you could know, but the basics can be mastered.

Broadcast News Media Research Indicated Bias for Senator Barack Obama

A broadcast news media study by the Center for Media and Public Affairs at George Mason University found that coverage during the 2008 U.S. Presidential campaign of Senator Barack Obama was 65% positive, but coverage of Senator John McCain was only 36% positive.

According to the study by researchers at George Mason University, there was a documented media bias for Obama and against McCain. Did the bias influence voters? I don't know. Was the study relevant news that was largely ignored. I don't know.

The link below has the details. Judge for yourselves...

Source: http://www.cmpa.com/media_room_press_10_30_08.htm

Additional References: Pew Charitable Trust Study of Print Media: "The media coverage of the race for president has not so much cast Barack Obama in a favorable light as it has portrayed John McCain in a substantially negative one, according to a new study of the media since the two national political conventions ended."

Source: http://journalism.org/node/13307

References

The Center for Media and Public Affairs at George Mason University, http://www.cmpa.com/media_room_press_10_30_08.htm

Pew Charitable Trusts Excellence in Journalism, http://journalism.org/node/13307

Unwelcome Effects of Public Opinion Research

Unwelcome Effects of Public Opinion Research

The importance of the public opinion survey / poll has gained prominence in presidential races, because of the economy and efficiency of mass opinion polling over the telephone and the Internet. For example, with a relatively small sample size of just under 400 randomly selected participants one can gain a reasonable understanding of the opinions of up to 1,000,000 persons, within a margin of error. The miracle of statistical inference.

A sample of approximately 1,500 randomly drawn individuals may be projectable across the entire nation. The implications are clear. An unscrupulous candidate, who strongly desires to be elected, may communicate only those messages that increase his/her favorable ratings in the polls. On the other hand, a candidate with integrity may use the pollster to determine those messages springing from his/her political ideology that need fine tuning to appeal to the largest group of voters.

Appealing to the largest group of voters is similar in concept to the responsiveness that all politicians in a majoritarian form of democracy must face. Public opinion polling should be used only by politicians and news organizations to gain a better understanding of their audience, but polls alone should not be considered news and should not be reported in a way that will shape public opinion. Is that too much to ask? Is that unrealistic? Perhaps.

In sum, honest and disingenuous politicians alike, and news organizations with a specific agenda, may find the pollster an indispensable member of the team, but their is a societal cost.

Reference

Janda, K., Berry, J.M., & Goldman, J. (1995) The challenge of democracy: Government in America, (4th Ed.). Boston, MA: Houghton Mifflin.

Rumsey's Ten Common Statistical Mistakes

Rumsey's Ten Common Statistical Mistakes

• Misleading Graphs
• Biased Data
• No Margin of Error [reported]
• Non-random Samples
• Missing Sample Sizes (i.e., not reported)
• Misinterpreted Correlations
• Confounding Variables (i.e., outside influences not discussed)
• Botched Numbers
• Selectively Reporting Results
• The Almighty Anecdote
  

Reference

Rumsey, D. (2003). Statistics for dummies. New York: Wiley.

Rumsey's Ten Criteria for a Good Survey

Rumsey's Ten Criteria for a Good Survey

• Target Population Well Defined
• Sample Matches the Target Population
• Sample is Randomly Selected
• Sample Size is Large Enough
• Good Follow-Up Minimizes Non-Response
• Type of Survey Used is Appropriate
• Questions are Well Worded
• Survey is Properly Timed
• Survey Personnel are Well Trained
• Survey Answers the Original Question
  

Reference

Rumsey, D. (2003). Statistics for dummies. New York: Wiley.

Understanding Null and Alternative Hypotheses

Is Grandma's Freezer Cold? (Understanding Null and Alternative Hypotheses)

When approaching business research, managers are sometimes confused by the concepts of the null and alternative hypotheses. The concepts are incredibly useful though, when the decision can be framed as a binary choice.

The null hypothesis embodies the condition that nothing has changed. For example, if we wished to learn if deep freezers were cold inside, we could think of the research in terms of null and alternative hypotheses.

The null hypothesis would be that the freezer in our sample is cold inside, which would be the normal condition. The alternative hypothesis would be that the freezer in our sample is not cold.

Therefore, to draw our sample, we walk up to Grandma's deep freezer, open the door, and stick our hand inside. Yes, Grandma's plugged in freezer is cold inside.

Internal validity, which means that what we saw what we thought we saw, is supported because we sensed that the freezer was cold with our own hands.

External validity is good in this case, which means that we can project our sample on the population of freezers that are plugged in (i.e., we did not check air conditioners, tap water, or ovens, but a freezer).

Construct validity, which is the theoretical background of measuring the temperature of freezers by sticking your hand in them, is supported because we have stuck our hand in all sorts of places to ascertain temperature before.

Conclusion validity, or support derived from statistically drawing a conclusion about all freezers from our sample of one, is not very good because our sample is very small.

Did Babies Build Roads in Europe? Presumed Causation Between Correlated Variables

A frequent problem with interpretation of data in the social sciences and business research is presumed causation between correlated variables. Two variables can exhibit perfect linear correlation yet not be in a cause and effect relationship. Generally, we need to satisfy at least three stipulations to argue for a cause and effect relationship:

1. Temporal precedence -- the cause happens before the effect.
2. Association between the independent variable (i.e., cause) and dependent variable (i.e., effect) – a linear, geometric, exponential, logarithmic, or some other covariation exists.
3. No reasonable alternatives -- upon careful inspection there are no other reasonable explanations for why the cause would result in the effect.
   

For example, between the years of 1945 and 1962, there were dramatic increases in the number of new roads built in Europe and the number of live births in the United States. (Note that I read this comparison somewhere but I do not recall; I use it frequently when teaching undergraduate statistics, because the face absurdity of the comparison makes the lesson easily remembered by students.) Were babies building roads in Europe? Not likely. Were roads in Europe making it possible for more babies to be born in the good 'ole U.S.A.? Not likely. See, variables can be perfectly correlated and probably unrelated. That is, there is no direct relationship between those variables; a confounding, third variable could be related to both of the correlated variables, which we might assign in this case to the drastic social upheaval that occurred during World War II.

In business research, causation and correlation are frequently confused as well. For example, are the dollars invested in showroom inventory the cause of sales revenue at the retail furniture store? Are the dollars of sales revenue generating investments in new showroom inventory? Still, is a third, confounding variable, such as consumer demand, somehow affecting both? In many cases, the discrete causal variable is not being measured, but at least the three stipulations above must be satisfied to argue for causation between any two known variables.

Data Collection Examples and Strategies

New Housing Development. A personal interview may be the best way to adjust the stream of questions to probe about specific problems experienced by the residents of a new neighborhood. The fact that the respondents live in a new neighborhood may make telephone and mailing lists unavailable. The open-ended nature of a study about what they like and dislike about life in a small subdivision may require probing on the part of the interviewer. An advantage of a personal interview in this situation is that the respondents are physically close together and that the newness of the neighborhood may make the respondents more approachable.

College Students Voting for Student Government. The university environment would make it difficult to use telephone or mail surveys. Some students may not have a telephone number, and if they do, they may be difficult to reach or they may be from only one particular socio-economic class. A mail survey would probably take too long to reach the students and be responded to, especially with off campus students. A personal interview including a small set of screening questions such as age, gender, ethnic background, and income conducted in the student union building would probably yield results accurate enough to project a favored candidate for the presidency.

Human Resources Professionals in Grocery Distribution. A personal interview of this population would be extremely expensive. However, with only few major companies involved, it is desirable to contact almost all of the companies. A telephone interview is a reasonable compromise from contacting each one personally. A mail survey would probably result in very low response rate, and a higher response rate is needed because of the small population, so it is not recommended.

Attitudes toward Economic Outlook by Fortune 500 CFOs. This is a hard to reach, geographically spread out group of respondents. Personal interviews would be far too expensive. Mail surveys may not be returned in sufficient numbers. The closed-ended nature of the mail survey may limit the answers given. A telephone survey seems provide the best balance of expense, accessibility, and flexibility. Furthermore, a telephone interview would give the greater flexibility needed to probe and get predictions for the next year’s economic forecast. If the research budget is limited, then a mail survey would be the best choice.

Surveying Retail Pharmacies. This audience could be very numerous and spread out geographically. This topic would be of great interest to this audience, not always typical of mail surveys, so a high return rate would be possible. A mailing list for this audience could be easily secured. The expense of a personal interview would not be necessary, as the respondents are too many in number and too spread out geographically. Telephone interviews would yield good results, but the extra expense would not be justifiable.

Basic Research Concepts: A Handy Reference

Applied research—Using basic research methods, the outcome of the research is applied to a business problem.

Basic research—The clinical or scientific method of research. This type of research is often done in laboratories or under controlled circumstances.

Four Types of Research Methods: Reporting—A summary or incomplete review of existing data; Descriptive—Most often used in marketing or sales. This type of research always asks who, what, when, where, why, and how in research questions; Exploratory—Using focus groups or a small study to get a feel for the problem; Predictive/Causal—Research conducted where one unit is held steady, while the experiment is conducted on the other unit, which tests whether the experiment itself is the reason for change in the experimental unit.

Time Factors for Research: Longitudinal—A research study performed on a sample over a period of time; Cross-sectional—A research study done only once, which provides a snapshot of what is occurring somewhere at a particular time.

Four Types of Research Validity: Construct Validity—the theoretical or practical underpinnings of the hypotheses and measurement of the variables; External Validity—the comparability or generalizability of the findings to other samples and settings; Internal Validity—for descriptive, explanatory, or causal studies this is basically an answer to the question of whether we saw what we thought we saw; Conclusion Validity—did we use the proper statistical tools to draw these conclusions?

Operational definition—is a definition stated in terms of specific testing or measurement criteria. These terms must have empirical referents, which means we must be able to count or measure them in some way. The object to be defined can be physical one (i.e., a machine tool), or it can be abstract one (i.e., achievement motivation).

Level of Measurement—the characteristic of the data with respect to alphabetic and numerical values assigned to represent it, such as the measures of variables on surveys. Data can be represented at four levels of measurement:

• Nominal—e.g., Male/Female; The word nominal means in name only. Nominal variables are used on surveys to describe or identify the population being sampled;
• Ordinal—e.g., Rare, Medium Rare, Medium, Medium Well, Well Done; An ordinal measure can capture how a person feels on an issue, which is the case when the distance between each of the measure cannot be determined scientifically;
• Interval—e.g., Temperature or 1 to 5 satisfaction scales; with Interval measures, the distance between each unit of measure has a precise distance. For example, how long does it take ten trucks on the loading dock to unload a full container of merchandise?
• Ratio—Age in years, relative time, relative distance or relative temperature. Ratio data is captured with absolute measures; height, weight, distance, and money are all examples of ratio data.

Unit of Analysis—a classification of the individual, group, company, or societal unit under study. It is relevant because comparison of data from different units of analysis is frequently used to draw conclusions that while they seem logical are, in fact, erroneous.

For example, predicting the outcome of local elections based on a national survey or predicting the outcome of national elections based on a local survey. This fallacy involving misapplication of the unit of analysis is related to the ecological and exception fallacies. Consider this important issue in research, especially when using secondary data (i.e., data collected by somebody else for a different research question), as it is not always clear whether one is examining the individual, group, company, industry, etc.

For example, news commentators sometimes compare mismatched units of analysis and draw conclusions that may not be correct. If one draws conclusions about a group from one individual case, that is the exception fallacy. If one draws conclusions about an individual because they are part of group, that is an ecological fallacy.

For example, you know of several people who are Razorback fans and observe that they each own a red pickup truck. If you then meet a Razorback fan at the university, can you assume that they own a red pickup? No, because of the potential for ecological fallacy; you've erroneously assigned a group attribute to an individual. If you then meet red pickup trucks on the road, should you yell “Soooeee...” out the window at each one of them? No, because of the exception fallacy; it is possible that you have assigned an individual attribute to the entire group. The problem in both cases is that the unit of analysis of the information under examination does not match the type of research question at hand. Hence, there is a possibility of committing a unit of analysis fallacy.

Business Research Push-ups

Ascribing Causes to Events

The post hoc fallacy generally describes a major problem with reaching and inductive conclusion that covariation between two variables indeed exists when the variables cannot be manipulated (to verify the conclusion.) Causal inferences are also only predictive and presumptive, not absolute. Therefore, conclusions based on causal inferences may only be temporary in nature -- changes in the conclusion will come if strong predictions of other causal inferences are found. In other words, large problems may result when other variables (not under study) may be responsible for actions we have ascribed to one particular variable.

Major Sources of Measurement Errors

The four major sources of measurement error are the respondent, various situational factors, undue influence by the measurer, and the instrument used to measure the response. The respondent may be a source of measurement error in a home telephone interview conducted in the evenings on the topic of political candidates when s/he gives imprecise answers just to end the interview and get off the telephone. An additional person such as a spouse present during an interview on the topic of attitudes toward a new car model may influence the responses given. An interviewer in a face-to-face interview may inadvertently lead the respondent to give certain answers by using a particular tone of voice. A long survey containing many questions worded at a higher level of vocabulary than the respondent is accustomed to will cut response rates.

Measurement Scale Validity More Important than Reliability

Scale validity is more important to the measurement process than reliability. Reliability is concerned with freedom from random error or instability that can be present in a measurement device. A measurement device can be reliable, but not it does not have to be valid. Validity is more critical than reliability because it refers to whether what we wish to measure is actually measured. However, a measurement device cannot be valid if it is not reliable.

Difficulty of Determining Content Validity

Content validity of the measurement scale items is not the most difficult type of validity to determine. The evaluation of content validity requires judgment and intuition which may be difficult for some to accomplish. Once again, the criterion-related validity is not simple, but it can be determined by correlation of the scores. Construct validity is the most difficult to determine because one must consider whether the construct applies to (i.e., supports or refutes) the theory in question.

Reliable Measures May Not Be Valid

A valid measurement is reliable, but a reliable measurement may not be valid. Once again, a measurement instrument can be reliable, but not it does not have to be valid. For example, a person may wish to measure a room. Believing one’s foot to be twelve inches in length, one steps across the room placing feet end on end. The process is repeated twenty times. Therefore, the conclusion might be that the length of the room is 240 inches. Later it is learned that the foot is only 11.5 inches in length. The person’s foot is a reliable measurement of the length of the room, that is, they get twenty foot-lengths every time, however, this is not a valid measurement of the room in standard inches.

Instrument Stability and Equivalence

Stability and equivalence are not identical terms. Both stability and equivalence are factors of reliability, but they refer to different ways in which reliability can be reduced. Stability refers to changes in the items that are being observed while equivalence refers to variations in how they are being observed.

Rating and Ranking Scales

Rating scales can be used easily to judge properties of objects against specified criteria without comparison to other objects, whereas ranking scales classify objects by requiring a choice between the objects. Rating scales can be time-consuming to construct, whereas the procedure for ranking objects can be difficult to administer. Rating objects can often be influenced by poor or careless judgments by the person doing the rating--the halo effect, leniency, and central tendency noted by Cooper and Schindler (2003) can all be found in the normal personnel review process, for example.

Ranking objects against one another may eliminate the need for development of an absolute set of criteria to judge by as required by rating scales, but judging more than two objects at a time may lead to misinterpretation of the exact level of attitude expressed for any one object. This is especially true when some of the items are equally matched in the positive attitude that can be evoked from the respondent. A perfect, yet simple, example of this vote splitting problem was seen in the 1992 U.S. Presidential elections where Bill Clinton, George Bush, and Ross Perot each received 41%, 37%, and 19% of the vote, respectively. One interpretation is that Clinton was the most favored candidate (i.e., had a mandate), whereas another interpretation is that Bush and Perot were similar enough in ideals that they were campaigning for the same votes, and therefore, had one of them not run, the remaining one would have been elected handily.

Likert and Differential Scales

Likert scales (i.e., a summated scale) can typically be created more easily than differential scales such as the Thurstone Differential Scale. This creation advantage that the Likert scale has may be due to the Likert being constructed through Item Analysis, rather than differential scale which is created through consensus. Differential scales are complicated and expensive so other methods like the Likert scale are preferable for business research. For some types of research, the cost of having many knowledgeable judges agree on the rating of items included on the differential scale may produce better results than using the pre-testing process of a summated scale.

Unidimensonal and Multidimensional Scales

Unidimensional scales cumulatively measure attitudes that are extreme to less extreme, so it is possible to understand which individual items the respondent judged positively or negatively. However, not all concepts and constructs can be adequately assessed in this way because the items being studied may be correlated in more than one way, that is, they are multidimensional. Construction of a scale using the Semantic Differential method may indeed reveal more dimensions and so the measurement instrument must be narrowly focused to measure a concept unidimensionally. Somewhat ethereal concepts such as organizational image or brand image may be difficult to assess with a cumulative (i.e., unidimensional) scale.

Methods of Survey Measurement Scale Construction

The five methods of scale construction are the arbitrary approach, the consensus, item analysis, the cumulative approach, and factor scales (Cooper & Schindler, 2003). The arbitrary approach is a commonly used method that describes scale construction that occurs as the measurement instrument is being developed. Responses are scored based on the subjective judgment of the researcher, which may be good or bad. This quick and inexpensive method can be very powerful in the hands of an experienced researcher.

The consensus approach involves scale construction by a panel of judges (i.e., presumably knowledgeable) who weigh each item for relevance, clarity, and level of attitude it expresses. The panel of judges can produce a better scale for the measurement instrument, but it does so at the expense of time and money.

The item analysis approach to scale construction actually analyzes how well the items included in the measurement instrument discriminate between indicants of interest. The values assigned can then be totaled to measure the respondent’s total score. The Likert is a one common and very effective Item Analysis (i.e., summated) scale.

The cumulative approach to scale construction ranks items based on the degree to which they represent a certain position held about the item of measurement. In particular, the Guttman scalogram attempts to measure unidimensionality, that is, if the responses fall into a pattern of the most extreme position also including endorsements of all positions that are less extreme.

The factor scale is based on the correlation between items and the common factors that they share. Factor scales deal with the problem of there being more than one dimension to an attitude toward an item and the fact that there may be more dimensions not yet known. The appropriateness of each of the five methods of scale construction depends on the research objective and the type of measurement instrument, etc. Scale construction through the arbitrary and item analysis methods may be less expensive and completely adequate for some topics. Scale construction through the consensus, cumulative, and factor scaling methods are more time consuming, expensive, and would be more appropriate for measurements involving complex judgments.

The impact of the scale construction technique chosen on the scaling design could be as important as the information that is hoped to be derived from the research. The reason is that the expense of scale construction may be too costly or time consuming, or the scale construction method may not fit the measurement of the judgments being made by the respondent. Therefore, the selection of the scale construction technique is important.

Probability Sampling and Nonprobability Sampling

A probability sample is necessary when a true cross section of the population is needed to properly achieve research objectives. It is most likely that the sampling phase of a project requiring a probability sample would need to be funded because of the expense in pursuing the need to know the population members from which to draw, the personnel involved, and the larger size of the sample to produce a sample with the desired degree of confidence.

A nonprobability sample is sufficient when it is not necessary to comprehend a single item represented in the whole population. Clearly, if the researcher is seeking only to gain a “feel” for the level of presence of certain items within a population gathered with a nonprobability sampling technique such as the judgment or quota sampling methods, the sampling procedure will be less expensive than a probability sampling and will probably suffice. A good example of this would be conducting sampling for exploratory research.

Random, Cluster, and Stratified Samples

A simple random sample is most appropriate when a list of the population elements is known and can be easily randomized. The simplicity with which the sampling procedure can be established and executed is a major advantage. A cluster sampling is most appropriate when the expense of a simple random sampling exceeds the budget and clusters that are internally heterogeneous and externally homogenous can be identified. In other words, obtaining a list of population elements that are naturally grouped into heterogeneous clusters that can be sampled may be easiest. A stratified sample is appropriate if a complete population list that would facilitate a simple random sampling is unavailable, and preferable, if the population can be stratified on the primary variable that is being studied. A stratified sample can also improve statistical efficiency, if it results in the elements within the stratum being more alike each other (i.e., homogeneous) and different from the other stratum (i.e., heterogeneous).

Finite Population Adjustment Factor

The finite population adjustment factor applies when the sample size is five percent or more of the total population and it can be used to reduce the required size of a sample to produce a desired level of precision (i.e., confidence). If the size of the sample is a budget concern, then adjusting the size of the sample with the finite population adjustment factor may be appropriate.
Disproportionate Stratified Probability Sample

The statistical efficiency of the entire sample can sometimes be increased if a larger sample is taken within one of the strata. This may be a good idea if the stratum is larger, more variable internally, and if the whole process of sampling is less expensive within the stratum.


Reference

Cooper, D.R., & Schindler, P.S. (2003). Business research methods, (8th ed.). Boston, MA: McGraw Hill.

To Make or Buy Research

Managers are continually faced with the decision to make or buy research to support important decisions. Managers need to know how research is conducted and how good research can be identified. Buying research requires that the research methodology and all phases of the collection and analysis of the research data be understood by the managers in order to make an informed decision about the value of the study -- the study might be inappropriate with respect to scope or may not have been conducted scientifically. If good research is not available or affordable within the manager’s budget, research of some kind must be undertaken. Conducting their own research requires time to properly structure the research, carefully consider the goals of the research, plan, and execute the data gathering phase, and analyze the collected data -- the study must be conducted with a clear idea of the research problem (i.e., it must be a researchable question) and by a person with the skills necessary to undertake such a study. If the study is to be conducted in house, the delegation of research to another party besides the manager often sheds bias and leads to results that are more relevant.

Researchable Questions

Consider also that some questions are answerable by research and others are not. In order to fall into the category of a researchable problem, a problem must be subject to data collection in the “real world” or least the procedures designed to perform data collection must be possible to execute. One general example is that of an ill-defined problem -- so little may be known about the exact size and scope of the problem (as it has been initially stated or understood) that it is not possible to form clear hypotheses for testing because of the existence of too many interrelated variables. That is, with some problems, it is impossible to hold a variable constant through assumption to test other related variables through data collection.

Examples of unresearchable management problems:

a. If iBOX corporation spends $5 million to develop and launch the new iBOX, the first product of its kind, will it succeed and make money?

b. How much money should we spend on marketing our products?

c. Should we risk announcing the development a new product to foil our competitors even though we have not yet begun the design phase?

Examples of unresearchable management problems altered/restated in researchable form:

a. In order to break even in the first year, iBOX corporation must sell 50,000 units of the iBOX to its five major market segments -- estimate the size of these market segments, and assuming a 1% penetration of these segments, is it possible to exceed 50,000 units in sales in all market segments combined?

b. At what point does the use of a particular marketing medium such as the trade show or print advertising to provide sales leads that close within 90 days appear to plateau or cease to scale up?

c. What effect on our competitor’s actions have previous pre-announcements of new products in the industry had?

Researchable Applied Studies

The following are researchable applied studies that managers in various contexts may find useful:

a. A Descriptive study by a manager of the men’s furnishings department at a national chain: Product Sales Analysis for previous year grouped by male demographic group. This study would help the manager to understand which clothing lines are selling well to his target customer.

b. A Predictive study by a plant manager at a Ford auto assembly plant: Quality Circles Implementation and Impact Study. This study will help the plant manager to raise quality levels at the assembly plant by understanding when, how, and why the workers strive toward higher quality.

c. A descriptive study by a director of admissions at a large state university: Regional demographic studies of high school seniors and their corresponding college selection in the surrounding geographic area for the last four years. This study will help the director understand how college selections are made among his/her target audience and perhaps learn enough to make related changes at the university.

d. A predictive study at an investment analyst at an investment firm: Economic Forecast for the coming year. This study will help the analyst to understand in what ways the economic indicators will unfold into the future economic outlook, and therefore guide investments based on that knowledge.

e. A reporting study for the director of personnel at a large metropolitan hospital: Regional Medical Worker Wage and Benefit Survey. This study will assist the director in understanding how other similar hospitals are compensating their workers.

f. An explanatory study for the product manager for Crest toothpaste at Procter & Gamble: Study of Relationship between Water Fluoridation in Urban Water Systems and Tooth Decay among the Local Citizenry. This study will give the product manager the empirical backing to request that more fluoride be added to Crest in order to enhance its cavity fighting power.

Reference

Cooper, D.R., & Schindler, P.S. (2003). Business research methods, (8th ed.). Boston, MA: McGraw Hill.

Commonly Confused Research Terms

Concept vs. Construct

A concept is a bundle of meanings and characteristics created by classifying and categorizing objects or events. A construct is an image or idea specifically invented for a given set of research and / or theory-building purposes. Concepts are often borrowed from another field of study and then applied in a new way during a research study. An abstract concept can often be considered a construct. For example, pass percentage of completion for the quarterback in football is a relatively well-understood concept that could be considered a construct, while ability to win is also an important, but abstract concept. Both pass percentage of completion and ability to win are skills expected of a good quarterback that can be researched and quantified to some degree, but there are vast differences in their measurability and precise definitions. Furthermore, a researcher may build a construct from other simpler concepts to assist in a theory building exercise. For example, pass percentage of completion could be one of concepts that comprises a construct named ability to win.

Deduction vs. Induction

Dewey’s so-called Double Movement of Reflective Thought (Cooper and Schindler, 2003) refers to the use of Induction and Deduction in tandem; it is an essential part of hypothesis formulation and testing prevalent in scientific thought processes. Deduction is a form of inference that presumes to be conclusive in that the conclusion follows from the reasons given and the reasons actually imply the conclusion (and represent a proof.) Induction is another form of inference in which a conclusion is drawn from one or more facts, and the process of inducing a conclusion represents an inferential jump beyond the evidence presented. In effect, Induction produces hypotheses that are tested through Deduction; Deductive / Inductive processes only produce hypotheses that must be later confirmed with data collection and analysis.

Operational Definition vs. Dictionary Definition

Clear definition of terms is an essential element of research. Dictionary-based definitions are provided through language synonyms. Often times, these synonyms are used to define one another or related word groups of similar, but not precisely the same meaning. This lack of precise definition is why dictionary definitions of concepts are unsuitable in a research environment. Operational definitions, however, are instead defined in terms of specific testing criteria or operations. An operational definition is one in which we must be able to measure, count, or collect data to verify a particular meaning. The operational definition of a concept must specify the characteristics to study and how those characteristics are to be observed, so that another researcher would classify the objects likewise.

Concept vs. Variable

A concept is a set of characteristics created by classifying objects or events that have common characteristics in two or more observations. Meanwhile, a variable is a synonym for the property under examination in an empirical setting; that is, a variable is defined in such a way that it takes an observed value at an instant, when we measure it. Observed variables may comprise an overall concept or construct in research, if they possess common characteristics.

Hypothesis vs. Proposition

Although there may be some dispute over the exact definition of a proposition and its relationship to a hypothesis, a proposition can be defined simply as a statement about concepts which can be judged to be either true or false, if the concepts are rooted in observable phenomena. A hypothesis is different in that it specifically refers to a variable or variables that can be empirically tested. A hypothesis is also more formally stated than a proposition, so that it can be accepted or rejected based on the actual observed phenomena. A hypothesis may describe the existence or magnitude of some variable, or it may describe a relationship between two variables with respect to some case. The most important role for the hypothesis in research is that it guides the direction of the study. In more lengthy research articles, it is common to find propositions stated quite generally that are later refined into testable hypotheses.

Scientific Method vs. Scientific Attitude

The Scientific Method can be thought of as the mind of science while the Scientific Attitude is the spirit driving scientific inquiry. The scientific method is structured inquiry into a well-defined topic. The scientific attitude is the underlying curiosity, imagination, and motivation to capture the most important element of a problem and apply the best technique for observing related phenomena in the most natural state. By analogy, business travel is a set of methods (i.e., scientific method) for traveling to the city of Chicago, whereas, desire to get to Chicago is the spirit of motivation (i.e., scientific attitude) to apply those business travel methods in the most optimal way. A scientific attitude in research provides the motivation continually to pursue inquiry in a scientific manner.

Examples of Variable Operationalization

Perhaps the distinguishing feature between concepts and constructs can be understood best as steps from an "abstract" research question toward a "concrete" variable. Consider these progressive stages in operationalization:

1. Research Question: Do parents have affection and concern for their children?
2. Conceptualization: Love (i.e., the concepts affection and concern might be conceptualized as "love")
3. Theoretical Construct: Parental Love for Children (i.e., a specific type of love is identified).
4. Operationalized Construct: Parental Love for Children as Time Spent with Children
5. Measurable Variable: Parental Love for Children as Time Spent with the Child per day.
6. Alternative Hypothesis: Parental Love for Children will be positively correlated with happiness of the child (not operationalized in this example).
   

The underlying lesson is that variables can be operationalized in multiple ways. In operationalizing Parental Love for Children, we could measure time spent with children, money spent in their care, number of words spoken to them, a qualitative analysis of themes discussed when speaking to the child, or we could even administer a survey to the child to ascertain perceived love as a proxy for parental love. All of those ways to measure love approximate the actual love experienced. There are many ways to operationalize a variable and these ways are determined (or bounded) by the original conceptualization.

An example of the many ways that variables can be operationalized, either optimally or otherwise, can be explained in attempting to measure whether Aunt Sally is at home baking pies on Sunday night after church. In order to test whether Aunt Sally is at home, we must decide how we are going to operationalize what the variable at home means for purposes of measurement. Do we see the lights on in the house? Does she answer the telephone? Does she answer a knock on the door? Does she answer her email? Is the Sunday paper still out front? Has she picked up her postal mail from the mailbox? Did she update her Apple Pie Blog website? Can you see her baking pies through the kitchen window? Is her Harley still parked in the driveway? Is the dog tied up out back? Are there spent shotgun shells on the back porch? Measuring whether Aunt Sally is indeed at home can be accomplished in many ways but there are only one or two best ways to stipulate an operational definition to arrive at a testable hypothesis.

Reference

Cooper, D.R., & Schindler, P.S. (2003). Business research methods, (8th ed.). Boston, MA: McGraw Hill.

Defining Business Research

Research is systematic inquiry of a topic for the purposes of solving a problem. Pure research is theoretical in nature and deals with problem solving which has little direct impact on business action items or policy decisions. Applied research has a more practical foundation in that its resulting conclusions have a direct influence on specific actions taken or policies developed. Pure research is often not the first type of research to take place in a given field of investigation. Applied research is more often the first type of research undertaken because it involves addressing a specific problem. After the problem is more adequately defined, pure research studies defined from what was concluded in applied research studies can tackle the more complex issue of developing a more complete theoretical framework for understanding a whole category of related business problems.

Establishing a definition of research is important in that it sets a minimum threshold for what is considered to be an original and valuable inquiry as opposed to studies which simply report what is commonly understood within a body of knowledge. Descriptive research is useful for defining a subject, but it is often not deep enough to provide working models of processes or explanation of phenomena. Explanation of a phenomenon that required a more penetrating study of the subject data and could lead to a predictive model. Prediction involves the development of a working model that forecasts certain outcomes based upon specific courses of action or natural occurrences. Mere reporting of a set of circumstances in a well-understood body of knowledge is often of little value in that it does not advance the body of knowledge, so it does not qualify as research, per se. In a general sense, a study, regardless of its nature, which advances understanding in a specific area such as a business problem, uses systematic, pre-defined procedures, and has specific goals may be considered to be research.

Reference

Cooper, D.R., & Schindler, P.S. (2003). Business research methods, (8th ed.). Boston, MA: McGraw Hill.