HOW TO DESIGN AND EVALUATE RESEARCH IN EDUCATION
Fraenkel J.R. Wallen, N.E
WAYS OF KNOWING: Sensory experience (incomplete/undependable), Agreement with others (common knowledge wrong), Experts’ opinion (they can be mistaken) and Logic/reasoning things out (can be based on false premises)
WHY RESEARCH IS OF VALUE: Scientific research (using scientific method) is more trustworthy than expert/colleague opinion, intuition, etc.
Scientific Method (testing ideas in the public arena)
– Put guesses (hypotheses) to tests and see how they hold up
– All aspects of investigations are public and described in detail so anyone who questions results can repeat study for themselves
– Replication is a key component of scientific method
Scientific Method (requires freedom of thought and public procedures that can be replicated): Identify the problem or question, clarify the problem, determine information needed and how to obtain it, organize the information obtained, and interpret the results.
TYPES OF RESEARCH
- Some of the most commonly used scientific research methodologies in education are experimental research, correlational research, causal-comparative research, survey research, content analysis research, qualitative research, and historical research.
- Experimental: Researcher tries different treatments (independent variable) to see their effects (dependent variable). In simple experiments compare 2 methods and try to control all extraneous variables that might affect outcome. Need control over assignment to treatment and control groups (to make sure they are equivalent). Sometimes use single subject research (intensive study of single individual or group over time)
- Correlational Research: Looks at existing relationships between 2 or more variables to make better predictions.
- Causal Comparative Research: Intended to establish cause and effect but cannot assign subjects to treatment/control; Limited interpretations (could be common cause for both cause and effect); Used for identifying possible causes; similar to correlation.
- Survey Research: Determine/describe characteristics of a group; Descriptive survey in writing or by interview; Provides lots of information from large samples. Three main problems: clarity of questions, honesty of respondents, return rates.
- Ethnographic research (qualitative): In depth research to answer WHY questions. Some are historical (biography, phenomenology, case study, grounded theory). Ethnographic research is one form of qualitative research. Another common form of qualitative research involves case studies.
- Case Studies: a detailed analysis of one or a few individuals.
- Historical Research: Study past, often using existing documents, to reconstruct what happened, Establishing truth of documents is essential.
- Action Research (differs from above types). Not concerned with generalizations to other settings. This is a type of research by practitioners designed to help improve their practice.
- Content analysis research involves the systematic analysis of communication.
- Each of the research methodologies described constitutes a different way of inquiring into reality and is thus a different tool to use in understanding what goes on in education.
GENERAL RESEARCH TYPES
- Individual research methodologies can be classified into general research types.
– Descriptive (describe state of affairs using surveys, ethnography, etc.)
– Associational (goes beyond description to see how things are related): correlational/causal-comparative
– Intervention (try intervening to see effects using experiments or quasi-experiments)
- Quantitative and qualitative research methodologies are based on different assumptions; the purpose of research, the methods used by researchers, the kinds of studies undertaken, the researcher’s role, and the degree to which generalization is possible.
– Quantitative (numbers): Facts/feelings separate; World is single reality; Emphasize casual relationships; Researcher removed; Established research design; Experiment prototype; Generalization emphasized.
– Qualitative (verbal descriptions): Socially-constructed multiple realities; Concerned with understandings from viewpoint of participants; Participatory; Flexible, emergent, research designs; Limited generalization. Natural setting is direct source of data. Data collected in the form of words or pictures. Concern with how things occur. Inductive data analysis-not based on hypotheses. “How people make sense of their lives”
- Meta-analysis: Locate all quantitative studies on a topic and synthesize results using statistical techniques (average the results). Effect sizes.
CRITICAL ANALYSIS OF RESEARCH
Critical analysis of research raises basic questions about the assumptions and implications of educational research.
OVERVIEW OF THE RESEARCH PROCESS
Problem statement that includes some background info and justification for study
– Exploratory question or hypothesis (relationship among variables clearly defined)
– Definitions (in operational terms)
– Review of related literature (other studies of the topic read and summarized to shed light on what is already known)
– Subjects (sample, population, method to select sample)
– Instruments (tests/measures described in detail and with rationale for their use)
– Procedures (what, when, where, how, and with whom);
– Give schedule/dates, describe materials used, design of study, and possible biases/threats to validity
– Data analysis (how data will be analyzed to answer research questions or test hypothesis)