The English past tense system has recently been used to argue that formal grammatical categories (such as root, rule, and lexical item) may not be necessary to explain the acquisition and knowledge of language. Rumelhart and McClelland (1986) devised a connectionist model relying solely on phonological information; it is often suggested that any deficiencies of such a model can be remedied by supplying it with semantic information. These proposals are incorrect: Grammatical categories and abstract morphological structure are indispensable and cannot be replaced with semantics while preserving the patterns of psychological generalization in the system. Linguists have noted that irregular past tense mappings (e.g., fly/flew; stick/stuck) apply only when a verb's root is marked in the lexicon as having an irregular past. Because nouns are never so marked, verbs with noun roots—denominal verbs—are regular even if they are phonologically identical to irregular verbs, hence: flied out/*flew out to center field; high-sticked/*high-stuck the goalie. Experiment 1 shows that adult subjects are highly sensitive to this principle when rating regular and irregular past tense forms of novel versions of irregular sounding verbs: New verbs formed from nouns were judged as better with a regular past tense (e.g., line-drived was the preferred past of “to hit a line drive”): new verbs formed from verbs were judged as better with an irregular past tense (e.g., line-drove was the preferred past of “to drive along a line”). Experiment 2 replicated the results with noncollege-educated adults, showing that the effect is not due to prescriptive language training. Experiment 3 tested an alternative to the formal grammatical account proposed by Lakoff (1987): When a verb has two meanings, one with an irregular past and one with a regular past, the irregular will belong to the meaning that is more central. Using regression techniques and ratings data, we disconfirm this prediction: In the data from Experiment 1, judgments of regular and irregular forms of a new verb are shown to be affected by whether the verb is derived from a noun or a verb, but not by whether its new sense is near the center or the periphery of the sense of the word it was derived from. Experiments 4 and 5 explain the few apparent counter-examples by gathering independent evidence for a short-circuiting process: When a denominal verb appears to have an irregular past tense form, it is because speakers sometimes interpret such verbs as having been derived directly from a related irregular verb root, bypassing the relevant noun. The experiments serve as a straightforward demonstration that representations of formal grammatical categories and structures are powerful determinants of linguistic behavior, and are not reducible to semantics, phonology, or prescriptive training.