Characteristics of Research Design
The likelihood of success of a research project depends on how well it has been designed.
Advantages/Disadvantages of the Subject Designs
- it is commonplace that a study utilizes more than one independent variable, each having several levels or treatments
- the ability of the investigator to differentiate between a new independent variable and a treatment level is critical to selecting the appropriate analysis for a given study
- three types of research designs
- study in which the same group assesses more than one treatment level
- at least one level of the independent variable receives on treatment (called the control condition)
- another level receives a second treatment (called the research condition)
- size of the effect of the treatment levels of the independent variables is determined by comparing performance between the levels of the independent variable
- example: sex (male, female), hearing sensitivity (normal, impaired) age (old, young)
- incorporates the within-subject and between-subjects design concurrently
Advantages/Disadvantages of the Subject Designs
Frequently Asked Questions
- What is the difference between an independent variable and a dependent variable?
The independent variable is the treatment, or what is being manipulated. The dependent variable, on the other hand, is the outcome of interest (i.e., what is being measured), which changes in response to a treatment (i.e., independent variable).
- What is reliability?
Reliability is the consistency of a test to estimate performance similarly over time.
- What is validity?
Validity refers to the extent to which an instrument measures what it is intended to measure. The central distinction to validity is that one is confirming not the instrument itself, but the purpose for which it is being used.
- What is internal validity and how are threats minimized?
Internal validity is defined as the approximate truth about inferences regarding cause and effect relationships. In research, internal validity is believed to be the most important aspect considered in research design. Stated differently, the only reason that performance varies between groups should stem only from the independent variables selected. If this objective is not met, then there is some degree of threat to internal validity. Differences between the independent variables increase when the investigator accounts for historical threats, maturation threats, instrumentation threats, mortality threats, and statistical regression threats.
- What is external validity and how are threats minimized?
External validity refers to the generalizability of a study or the ability of the investigator to report that the performance measured on the subjects recruited (i.e., sample) is representative of performance found in the global population. If this objective is not met, then findings might be threatened by a placebo effect, Hawthorne Effect, carryover effect, or treatment interaction effect.
- How are the null and alternative hypotheses related?
The null hypothesis (i.e., Ho) assumes that there is no statistically significant difference between the sample mean and the population mean. In other words, there is no difference in performance between the sample and population groups. The alternative hypothesis (i.e., Hi) is the opposing view of Hoand assumes that there are statistically significant differences in performance between the sample and population.
- What is an effect size?
Effect size is a descriptive measure that indicates the standardized difference—based on a z-score—between the alternative (i.e., sample) mean and the null (i.e., population) mean. This descriptive statistic allows for the comparison of effect sizes taken from different distributions and also provides a measure of treatment effectiveness (i.e., clinical significance).
- What is statistical power?
Statistical power is the probability that results of the data will conclude that the variables being compared have a causal relationship. In audiology, statistical power is set to a probability level of 0.8. This means that the probability (i.e., likelihood) of the data detecting a relationship between variables is 80 out of 100 (i.e., 80%). The higher the statistical power, the more the investigator increases the possibility of rejecting the null hypothesis and accepting the alternative hypothesis. Statistical power is affected by differences between means (including effect size), standard deviation, sample size (e.g., increases when correlation is poor between treatment levels; increases with two-tailed error compared with one-tailed error), and significance level.
- What is a confidence interval?
The confidence interval for the mean (CIx) is a descriptive statistic that provides the investigator with a range of values below and above the mean, with the intent of indicating the range of the population mean. The range is based on the estimated mean and standard error derived from results obtained from the sample group. The CIx is also used to determine statistical significance. That is, if the CIx for the population and sample overlap, performance is not statistically significant between groups. Conversely, if the CIx for the population and sample do not overlap, then performance is statistically significant between groups.
- What is the relationship between clinical and statistical significance?
Statistical significance is not the same as clinical significance. That is, statistical significance is used in the context of null hypothesis significance testing. Clinical significance, on the other hand, provides information about the success of a treatment in aiding a patient.