- science vs. pseudoscience
- chemistry vs alchemy
- creationism
- astrology
- homeopathy
- Holocaust denialism
- climate skeptics
Science | Methods | Research
Schedule
| Time | Monday | Tuesday | Wednesday | Thursday | Friday |
|---|---|---|---|---|---|
| 10-12 | common ground: philosophy of science | presentations qualitative papers | presentations quantitative papers | time, space, and social fabric | Urban water ecology |
| 13-15 | reading time | lecture qualitative methods | lecture quantitative papers | lab exercise | lab exercise |
Schedule
| Time | Monday | Tuesday | Wednesday | Thursday | Friday |
|---|---|---|---|---|---|
| 10-12 | systems thinking | advanced topics: Machine learning | group work | assignment 11:00 | Q&A | free |
| 13-15 | lab exercise: CLD | advanced topics: networks | Intro to GIS | free | free |
Outline
What is science?
How is scientific knowledge different from other forms of knowledge production?
What is the scientific method?
Research methods do not exist in vacuum
Research design: Why do we need one? or is there more than one?
Crystalized science: the research paper
0. Who are you?
1. What is science?
1. What is science?
“the natural and social sciences and the humanities are all parts of the same human endeavour, namely systematic and critical investigations aimed at acquiring the best possible understanding of the workings of nature, people, and human society”
Source: Hansson, Sven Ove, “Science and Pseudo-Science”, The Stanford Encyclopedia of Philosophy
1. What is not science?
- Difficult to demarcate:
- Timeless?
- Verificationism (Circle of Vienna)
- Falsificationism (Popper)
- problem-solving (Kuhn)
- scientific progress (Lakatos)
Non-scientific phenomena: religion, metaphysics
Science
- Why do we care about the problem of demarcation?
- healthcare
- expert testimony (law)
- environmental policy
- science education
- journalism
- possible + discipline-specific criteria
Pseudoscience
“Progress in science is only possible if a research program satisfies the minimum requirement that each new theory that is developed in the program has a larger empirical content than its predecessor. If a research program does not satisfy this requirement, then it is pseudoscientific.”
“an epistemic field is pseudoscientific if it cannot be incorporated into the existing network of established sciences (Reisch 1998; cf. Bunge 1982, 379).”
Source: Hansson, Sven Ove, “Science and Pseudo-Science”, The Stanford Encyclopedia of Philosophy
Pseudoscience
“…“the community of practitioners makes little attempt to develop the theory towards solutions of the problems, shows no concern for attempts to evaluate the theory in relation to others, and is selective in considering confirmations and disconfirmations” (Thagard 1978, 228)”
Source: Hansson, Sven Ove, “Science and Pseudo-Science”, The Stanford Encyclopedia of Philosophy
Science
“the intellectual and practical activity encompassing the systematic study of the structure and behaviour of the physical and natural world through observation and experiment” – Oxford Dictionary
“Science is a systematic search for knowledge whose validity does not depend on the particular individual but is open for anyone to check or rediscover” – Stanford Encyclopedia of Philosophy
Logical inference & Science
- Epistemic norms (Merton 1942)
- universalism
- communism
- disinterestedness
- organized scepticism
- Deductive: reasoning from premises to reach a logically certain conclusion (no epistemic uncertainty)
- Inductive: reasoning generalize conclusions from specific cases to general rules (epistemic uncertainty)
- Abductive: reasoning enables the inference of possible premises that lead to a conclusion (epistemic uncertainty)
Source: Hansson, Sven Ove, “Science and Pseudo-Science”, The Stanford Encyclopedia of Philosophy
Andersen, Hanne and Hepburn, Brian “Scientific method”, The Stanford Encyclopedia of Philosophy
Break (10mins)
3. What is the scientific method?
High-school version
Source: KhanAcademy
Hypothetico-deductive
- Theory
- Hypothesis
- Data collection
- Findings
- Hypotheses confirmed or rejected
- Revision of theory
Theory
- Generalized explanations of how nature works.
- Theory is the answer to the question why? -> causal reasoning
- Well-confirmed type of explanation: empirical support of a falsifiable claim
- Theory ≈ hypothesis
- Law: descriptive account of how nature behaves under certain conditions
- Theories are analytical tools for understainding, explaining, and making predictions
- Natural language, formal language -> follow principles of rational thought or logic
In social sciences:- Grand theories
- Middle-range theories
- Grounded theories
Epistemology (theory of knowledge)
What is (or should be) regarded as acceptable knowledge in a discipline?
Epistemology: Positivism
- Phenomenalism: only phenomena and hence knowledge confirmed by the sense can genuinely be warranted as knowledge
- Deductivism: the purpose of theory is to generate hypotheses that can be tested and that will thereby allow explanations of laws to be assessed
- Inductivism: Knowledge is arrived at through the gathering of facts that provide the basis for laws
- Objective: Science can be conducted in away that is value free
- Scientific statements ≠ normative statements
Epistemology: Realism
The hypothetico-deductive method is applicable to social sciences and there is an external reality independent of our observation of it
1. Empirical realism: objects exist in reality independently of our conceptual scheme (naive realism)
2. Critical realism: mind-dependent aspect of the world that reaches to understand the mind-independent world.
Epistemology: Interpretivism
Nature ≠ Social
Subjective meaning of social actions: actors form meanings and act upon them
- Hermeneutics: explanation vs. understanding of human behaviour
- Phenomenologist: attempts to see things from that ‘person’s point of view’
Ontology
What is knowable, what is real?
Ontology
Is the social world external to social actors or something people are on teh process of fashioning?
- Objectivism: ontological position that asserts that social phenomena and their menaings have an existence that is independent of social actors
- Constructivism: ontological position that asserts that social phenomena and their meanings are continually being accomplished by social actors
Source: Bryman (2008) Social research methods. Oxford
- “intellectual predilection for stressing the active role of individuals in the social construction of social reality”
- researcher own account: construction
- knowledge is indeterminate: postmodernism
- mutually incompatible with objectivism and realism
A binary system for a non-binary reality?
Research methods do not exist in vacuum
- Ontological and epistemological assumptions feed into what research questions are asked
- Influence the choice of qualitative vs quantitative methods
- Values: personal beliefs and feelings of the researcher
- Interfere / intrude?
- Research cannot be value free: but how to ensure that your values do not intefere with the research process?
- How to be self-reflective and open about the part played by such factors: personal bias?
5. Research design
Research design
Research design ≠ Research methods- Design: framework for collection and analysis of data
- Decision on priorities and relevant dimensions
- Causal variables
- Generalization to larger groups
- Method: technique for collecting data
Criteria of scientific research:
- Reliability: repeatable findings / results
- Replicability: allow others to verify results
- Validity: integrity of the conclusions
- Measurement: does a measure captures well a concept?
- Internal: causality, confounder and counterfactuals
- External: can results be generalized?
- Ecological: are results valid on everyday settings? or artifacts of researcher choices and methods?
Source: Bryman (2008) Social research methods. Oxford
Experimental design
Leveraging the power of comparison. Manipulation of an independent variable to determine the effects on a dependent variable. Groups are treatments, or levels on the independent variable. NOT all variables can be manipulated (unfeasible, unethical)
- Lab experiments: researcher has more control over experimental conditions.
- higher internal validity, but external?
- Field experiments: randomized control trials
- experimental and control treatments
- random assignment
- dependent vars: before - after
- The difference must be explained by the treatment because of randomness
- Quasi-experiments: Natural experiments
- no random assignment
- interventions (policies, programs)
Cross-sectional design
- Collection of quantitative or qualitative data on several cases in order to detect patterns of association
- Single point in time
- Examples: surveys, questionnaires, semi-structured interviews
Criteria of scientific research:
- Reliability: depends of representative samples
- Replicability: typically replicable
- Validity:
- Measurement: does a measure captures well a concept?
- Internal: Weak – hard to infer causation, only associations
- External: strong if randomized
- Ecological: unlikely, the instrument can bias the answers
Longitudinal design
- Multiple observations of same units over time
- Units: patients, cities, business, countries, schools
- Panel studies: sample over time. Balanced vs unbalanced.
- Cohort studies: cohort of people that share something: born the same week, married the same date
Criteria of scientific research:
- Reliability: depends of representative samples
- Replicability: typically replicable
- Validity:
- Measurement: does a measure captures well a concept?
- Internal: Strong, less ambiguity on direction of causal effects
- External: strong if randomized
- Ecological: depends on sampling unit
Case study design
- What is a case? a single: community, school, family, organization, person, event…
- Typical methods: ethnography, field notes
- Generalizability and external validity? Limited, but that was probably not the purpose of the study to start with.
- Types of cases: critical case, extreme or unique, exemplifying case, revelatory case, longitudinal case
Yin. 2003. Case Study Research. Applied social research methods series, Vol 5
Comparative design
- Multiple cases being compared
- Examples: cross-cultural, cross-national
- Methods: survey based, ethnographic, document analysis, QCA
- Make sure data is comparable in terms of categories and data-collection methods
1. Rihoux B, Marx A. QCA, 25 Years after “The Comparative Method”: Mapping, Challenges, and Innovations. Political Research Quarterly. 2013 Feb 12;66(1):167–235.
2. Ragin CC. Redesigning Social Inquiry: Fuzzy Sets and Beyond. Univ. of Chicago Press; 2008
No consensus
Too many stereotypes, but in practice what do scientists do to produce new knowledge?
Crystalized science: the research paper
Your choice of design & method depends on your research question
- Introduction:
- Scientific problem
- Research question
- Methods: what you do
- Results: what you found
- Discussion: how it fits the larger picture
- Conclusions: new knowledge
- Not a research paper:
- Literature review
- Commentary
- Perspective
- Policy forum
- Editorial
- Feature
- Helps you to understand the field, open questions, scientific problems
- At best identify testable hypotheses
- Reorganized old knowledge but do not produce new knowledge
Assignment
- Choose and read 2 papers, one qualitative and one quantitative
- Sign up here
- Prepare a speed talk 4 min summary (5 slides):
- Title
- What is the problem, research question, any expectations?
- What is the method: explain it in the context of the paper
- What did the authors found: new knowledge, lesson learnt
- Your opinion: was the method a good fit for the research question? Would you recommend it?
- Tuesday morning: presentation of qualitative papers
- Wednesday morning: presentation of quantitative papers
Re-cap
What is science?
How is scientific knowledge different from other forms of knowledge production?
What is the scientific method?
Research methods do not exist in vacuum
Research design: Why do we need one? or is there more than one?
Crystalized science: the research paper
Further reading
Hansson, Sven Ove, “Science and Pseudo-Science”, The Stanford Encyclopedia of Philosophy
Andersen, Hanne and Hepburn, Brian “Scientific method”, The Stanford Encyclopedia of Philosophy
Bryman, A. 2008. Social research methods. Oxford University Press
Miles & Huberman. 1994. Qualitative Data Analysis. Sage
Yin. 2003. Case Study Research. Applied social research methods series, Vol 5
Booth, W; Colomb, G & J Williams. 2008. The Craft of Research. University of Chicago Press
Tack | Gracias
Questions?
email: juan.rocha@su.se
twitter: @juanrocha
slides: juanrocha.se/presentations/USES_scientific_methods
Stockholm Resilience Centre
Subscribe to our newsletter at www.stockholmresilience.org/subscribe
1
Science | Methods | Research
Juan C. Rocha