I. Know what the construct is (and isn't) using common sense.
This should at least give you face validity.
II. What can go wrong with a measure with good face validity?
And what can you do about it?
Problem Solution(s)
1. Poor reliability Reduce error due to inconsistencies in
weakens validity. administering the scale by standardizing how you
administer the scale. Reduce random error due to
inconsistencies in scoring by eliminating the
observer or simplifying/structuring the observer's
task.
2. Observer bias may Structure/simplify the observer's task, use
poison validity. multiple-choice items, eliminate the observer, or
use the "blind technique"
3. Subjects can bias the
results by
a. following demand Use blind techniques make faking hard by not
characteristics letting them know they are being observed, not
letting them know what they are supposed to be
faking, making it expensive to fake, using
physiological measures that may be impossible to
fake.
b. responding in the Make faking hard by not letting them know they are
social desirable way being observed, not letting them know what they
are supposed to be faking, making it expensive to
fake, using physiological measures that may be
impossible to fake.
c. succumbing to Change the wording of questions so that sometimes
response sets such as agreeing with an item means that you are in
the acquiescence support of a position, whereas sometimes
"yea-saying" bias. disagreeing means that you support that position.
use disguised questions use behavioral or
physiological measures instead of verbal measures.
4. Intuition may be See that your definition is consistent with theory
inaccurate leading to and research about the construct you wish to
not measuring the right measure. Make the case that you are measuring all
construct. the relevant dimensions of the construct and have
enough items to adequately represent each of these
dimensions: the content validity approach.Ex:
Love=lust, commitment, & trust Argue that
all the items of your subscales are measuring the
same construct by showing that the answers to
these items are correlated: the internal
consistency argument. Make the case that your
measure must be valid because it correlates with
other indicators of the construct: the convergent
validity approach Show that you are not measuring
the wrong construct by showing that your measure
does not correlate too highly with measures of
other constructs: the discriminant validity
approach.