The property of validity pertains, in general, to an argument. It is, however, often applied to an empirical method.

A method or proposition is valid to the extent that it provides access to the construct (concept) that represents the relevant epistemic interest. In particular, a scientific method is valid to the extent that it produces the results it is meant to produce. This implies that the criteria for determining the validity of a method are independent of that method and given beforehand.

If we presuppose

• the constructs themselves
• and the function of the method to connect constructs with data

then it follows that validity of a method essentially depends on the quality of the operationalization of the constructs: The better the constructs appealed to have been operationalized, the higher – ceteris paribus – the validity of a method relying on them.

Consider as an example the task of identifying, in a sample of persons, that subset who speak English as a second language above level x of the Common European Frame of Reference (CEFR). Assume that the construct itself (a person speaking a language at a certain level) is well-defined and that the other two requirements usually made for a method are fulfilled: the method is objective, i.e. the result does not depend on who applies it, and it is reliable, i.e. it identifies the same set of persons on repeated application. Now consider the following methods as candidates:

• Ask the persons in the sample whether they speak English above level x.– It is immediately clear that this method is not valid, for more than one reason: the persons may be ignorant of the levels, or ignorant of their own proficiency, or be biased.
• Define a mapping relation between a high-school diploma testifying so many years of English as a second language and the scale of marks given to high-school graduates, on the one hand, and the system of CEFR levels, on the other hand. Use the high-school diploma of the persons in the sample and map their marks for English onto the levels, including level x.– Although this method is often used, it is equally clear that it is not valid. Among other things, the criteria used by high-school teachers differ from the criteria defining CEFR levels. Moreover, language proficiency decays, and the older the diploma, the less reliable it is.
• Operationalize the CEFR levels, in particular level x, in terms of a language proficiency test and subject the sample to the test.– This method has a chance of validity, depending – among other things, but crucially – on the quality of the operationalization.

Consider as another example the goal of ascertaining the grammaticality of a sentence. The method of getting grammaticality judgments from a group of informants and calculating the average value of their judgments is valid to the extent that the latter does correspond to the grammaticality of the sentence. Whether or not this is the case must be determined independently of, and prior to the application of, the method.

A method can be valid only if it is reliable; but it may be reliable without being valid.