EFL Learners’ Comprehension of Scalar Emotion Verbs

This study investigated EFL learners’ comprehension of scalar properties of three types of emotion verbs, namely, fear type, liking and disliking emotion verbs and compare their performance with instructors and native speakers of English. The participants were 38 non-native pre-service teachers from ELT department at a state university in Turkey, 11 ELT instructors at different universities and 10 native speakers from the USA and the UK. A scale construction task was administered, and data were collected via in person and e-mail according to participants’ judgements on scalar emotion verbs in terms of their relative order on a linear scale. The quantitative data were analysed using SPSS. The results revealed that in terms of constructing consistent scales with previously determined scales in literatutre, pre-service teachers performed poorly for fear-type and disliking emotion verbs, they were partly successful in constructing consistent scales for liking verbs. It was also found that similarly instructors performed poorly in constructing scales for fear-type and disliking verbs, but they were better than pre-service teachers. They were also successful in constructing scales for liking verbs. Native speakers were successful in fear-type and liking verbs; however, like non-native participants, they performed poorly in constructing consistent scales for disliking verbs. This means that there are cross-cultural differences among participants’ judgement of emotion verbs on a linear scale in terms of their intensity. This study provides valuable information for the studies on lexical resources (e.g., VerbNet, WordNet etc.) Previous studies (e.g. Fellbaum & Mathieu,2014; Sheinman, & Tokunaga, 2009) show a way to represent the scalar properties of emotion verbs in WordNet, and other possible extensions to additional verb families can cause a more subtle semantic analysis of emotion verbs in lexical databases with potential benefits for automatic inferencing, language pedagogy and translation. This study contributes to semantic analysis of emotion verbs in lexical databases. It also provides some implications for students, language teachers, and policy makers in terms of vocabulary learning and teaching.

This study investigated EFL learners' comprehension of scalar properties of three types of emotion verbs, namely, fear type, liking and disliking emotion verbs and compare their performance with instructors and native speakers of English. The participants were 38 nonnative pre-service teachers from ELT department at a state university in Turkey, 11 ELT instructors at different universities and 10 native speakers from the USA and the UK. A scale construction task was administered, and data were collected via in person and e-mail according to participants' judgements on scalar emotion verbs in terms of their relative order on a linear scale. The quantitative data were analysed using SPSS. The results revealed that in terms of constructing consistent scales with previously determined scales in literatutre, pre-service teachers performed poorly for fear-type and disliking emotion verbs, they were partly successful in constructing consistent scales for liking verbs. It was also found that similarly instructors performed poorly in constructing scales for fear-type and disliking verbs, but they were better than pre-service teachers. They were also successful in constructing scales for liking verbs. Native speakers were successful in fear-type and liking verbs; however, like non-native participants, they performed poorly in constructing consistent scales for disliking verbs. This means that there are cross-cultural differences among participants' judgement of emotion verbs on a linear scale in terms of their intensity. This study provides valuable information for the studies on lexical resources (e.g., VerbNet, WordNet etc.) Previous studies (e.g. Fellbaum & Mathieu,2014;Sheinman, & Tokunaga, 2009) show a way to represent the scalar properties of emotion verbs in WordNet, and other possible extensions to additional verb families can cause a more subtle semantic analysis of emotion verbs in lexical databases with potential benefits for automatic inferencing, language pedagogy and translation. This study contributes to semantic analysis of emotion verbs in lexical databases. It also provides some implications for students, language teachers, and policy makers in terms of vocabulary learning and teaching.

INTRODUCTION
Language learners often come across situations in which they are required to decide on an appropriate word to use among a few near-synonymous words in the vocabulary learning process. There may be subtle differences between the near-synonymous words, and the meaning of near-synonyms may be different in the target language and native language, so differentiating these words becomes more challenging for language learners (Sheinman & Tokunaga, 2009). For instance, when we consider the following sentences: "This film is good", "This film is great" and "This film is superb", we see that all of these sentences include positive evaluation of a film; however, under what conditions or in which of these sentences will the film be perceived as the best by a native speaker of English?
Which of the sentences will denote the most intense emotion for speakers and listeners?
How can the language learner know it? (Sheinman & Tokunaga, 2009). Are computational lexicons or online lexical databases useful for language learners to recognize subtle differences between words? These are some driving questions that encourage us to conduct such a study.
As this study draws attention to the online lexical resources and their possible benefits, it is plausible to start with recognizing the importance of lexical classification.
Lexical classes are defined with regard to similar morpho-syntactic behaviour of words and common meaning components, and these lexical classes are beneficial for capturing some generalizations about a number of cross-linguistic properties (Kipper, Korhonen, Ryant, & Palmer, 2008). Kipper et al. (2008) explain the benefits of lexical classes and they state that natural language processing (NLP) systems can draw on lexical classes in a few ways. They encapsulate the benefits of lexical classes as follows: • Lexical classes describe "the mapping from surface realization of arguments to predicate-argument structure", thus a crucial component of a system that calls for the latter.
• The classes may be utilized as a principled tool to abstract away from specific words when needed because they can grab high level of abstractions (e.g., semantic or syntactic features).
• They are beneficial for numerous operational contexts in which lexical information is required to be comprehended from small application-specific corpora. They can compensate for unsufficient data by completely behaving as a very typical example of relevant words.
• They are also useful in terms of supporting many multilingual tasks such as language generation, computational lexicography, machine translation, semantic role labelling, word sense disambiguation, parsing and subcategorization acquisition (Kipper et al., 2008, p. 22).
Among these lexical classes, "verbs are a locus of information for overall sentence structure and selectional restrictions on arguments", thus their organization or representation is important for natural language processing (Swift, 2005, p. 115). They are the most important syntactic and lexical category in a language (Fellbaum, 1990). There are various views on verb classification. For instance, Levin (1993) characterizes semantic verb classes which pattern in terms of syntactic alternations. Levin's classification is the basis of VerbNet that is an online lexical database. On the other hand, verbs are classified through semantic concepts in FrameNet (Baker, Fillmore, & Lowe, 1998) instead of syntactic alternations. The classification of verbs has varied syntactically and semantically for different languages; (Ortony, Clore, & Collins, 1988;Belletti, & Rizzi, 1988;Levin, 1993;Mathieu, 2006;Kipper et al., 2008;Mathieu, & Fellbaum, 2010;Tamm, 2012) however, there is little consensus among these various analyses (Fellbaum, & Mathieu, 2012).
As language teachers, we see that Turkish EFL learners usually have difficulty in deciding on an appropriate word to use among a few near-synonymous words. It may be beneficial for students to know verb classification and scalar properties of verbs or adjectives, or other linguistic properties of words which can be done using online lexical resources such as VerbNet (Kipper-Schuler, 2005); WordNet (Miller, 1995;Fellbaum, 1998); FrameNet (Baker, Fillmore, & Lowe, 1998); PropBank (Kingsbury, & Palmer, 2002); VerbOcean (Chklovski, & Pantel, 2004); and TRIPS [The Rochester Interactive Planning System] (Ferguson, & Allen, 1998). It is especially important for pre-service teachers as they will teach vocabulary in their classes in the future. The teachers of future will probably use technology more intensively than ever before. Being aware of these lexical resources, they can be encouraged to conduct linguistic studies by using these resources, such as speech understanding, dialog processing, and semantic parsing through task planning, natural language generation and intention recognition (Swift, 2005). Thus, they can teach English words more effectively in their classes. When we review the literature, there are no empirical studies conducted with Turkish EFL learners on scalar emotion verbs and there is little or no interest in online lexical resources such as VerbNet, WordNet, FrameNet, PropBank, VerbOcean and TRIPS which may provide language learners and teachers with valuable information for language learning, especillay in terms of vocabulary learning. Therefore, the current study aims to investigate whether Turkish pre-service teachers can construct consistent scales with the scales of previous studies in literature in terms of the order of emotion verbs based on their intensity. That is why we aim to explore whether participants from various cultures similarly order emotion verbs on a linear scale in terms of their intensity. For this purpose, we selected 3 groups of emotion verbs. The first group includes 5 fear emotion verbs: Intimidate > alarm > scare > frighten > terrify which have been scaled in Fellbaum and Mathieu's (2012) study; ; the second group includes 5 liking emotion verbs: like > love > adore > worship > deify ; and the third group includes 5 disliking emotion verbs: dislike > hate > abhor > detest > loathe whose intensity scales have been demonstrated in Faber and Usón's (1997) study. The study further aims to compare the scales constructed by native speakers, instructors and pre-service teachers. It will be discussed whether the scale constructed by the participants in this study will yield similar scales with the scales of previous studies in terms of the order of emotion verbs based on their intensity. In this direction, the current study attempts to answer the following research questions:

1.
To what extent can pre-service teachers, instructors and native speakers construct consistent scales with the scales of previous studies in literature in terms of the order of emotion verbs based on their intensity?

2.
Are there significant differences among pre-service teachers', instructors' and native-speakers' scales in terms of the order of emotion verbs based on their intensity?

LITERATURE REVIEW
As the current study focuses on scalar properties of emotion verbs, our main discussion will be about emotion verbs (or psych verbs). In this part, we aim to present some prominent studies on scalar properties of words, and then several studies on the efficiency of vocabulary learning. It will be beneficial to start with the notions "gradation" and "scalarity" as some emotion verbs will be analyzed in terms of their gradation on a scale in this study.
Gradation is generally considered as a prototypical feature of adjectives, but it is not restricted to adjectives and gradation can be articulated even though a language does not possess a particular class of adjectives (Fleischhauer, 2016). A degree gradable verb is explained as follows: "a verb admits degree gradation if it either lexicalizes a suitable gradation scale or if the activation of a suitable gradation scale is licensed by the conceptual knowledge associated with a meaning component lexically specied in the verb" (Fleischhauer, 2016, p. 176). Gradation is often specified synonymously with intensification and it is "the linguistic process of comparing two (or possibly more) degrees on a scale" (Fleischhauer, 2016, p. 16). A scale is constructed by "a linearly ordered set of degrees" (Fleischhauer, 2016, p. 16 (Horn, 1989, p. 232).
"Quantitative scales are defined by entailment; Pj outranks Pi, on a given scale iff a statement containing an instance of the former unilaterally entails the corresponding statement containing the latter" (Horn, 1989, p. 231). Horn (1989) gives some examples of such scales, "where <. . ., Pj, Pi,, . . .> indicates that Pj > Pi, that is, that Pj outranks (is stronger than) Pi on the relevant scale" as shown in Figure 1. Figure 1 demostrates that some emotion verbs like liking verbs such as adore, love, like; and disliking verbs such as loathe, hate, dislike really possess scalar qualities and emotions differ in terms of their intensity.
Similarly, Levinson (1983) states that "a linguistic scale consists of a set of linguistic alternates, or contrastive expressions of the same grammatical category, which can be arranged in a linear order by degree of informativeness or semantic strength" (p. 133).
The lexicon of emotions poses considerable challenges for lexical encoding and systematic investigation (Fellbaum, & Mathieu, 2014). Some emotion verbs may be related to each other as they possess the same basic eliciting condition; however, they differ with regard to their intensity and weights which are assigned to diverse manifestations or components (Ortony, Clore, & Collins, 1988). Ortony, Clore and Collins (1988) remark that one of the most salient characteristics of emotions is that they differ, to a great extent, in their intensity between people. They maintain that the intensity of emotions is affected by several variables such as praiseworthiness, desirability and appealingness which "correspond to three foci of valenced reactions, namely, agents, events, and objects" (Ortony, Clore, & Collins, 1988, p. 34). Desirability that is computed regarding goals is related to reactions to events; praiseworthiness that is computed regarding the standards is related to reactions to the actions of agents, and appealingness that is determined in respect to attitudes is related to reactions to objects. It means that among other variables which can affect the intensity of event-based emotions, desirability necessarily affects all of them (Ortony, Clore, & Collins, 1988). Thus, if the event seems to be more desirable or undesirable, the experience of emotion will be more intense. Similarly, praiseworthiness affects the intensity of all attribution emotions, and appealingness affects the intensity of attraction emotions (Ortony, Clore, & Collins, 1988, p. 48). Accordingly, they explain that the intensity of fear emotions such as nervous, worried, timid, petrified, fear, scared, terrified, frighten, etc. is affected by "the likelihood of the event or the degree to which the event is undesirable" (Ortony, Clore, & Collins, 1988, p. 112); the intensity of liking emotions such as like, love, adore, attracted-to, affection, etc. is affected by "the degree of familiarity with the object or the degree to which the object is appealing"; and the intensity of disliking emotions such as loathe, hate, dislike, disgust, detest, aversion, etc. is affected by "the degree of familiarity with the object or the degree to which the object is unappealing" (Ortony, Clore, & Collins, 1988, p. 157).
The intensity of liking and disliking verb classes and their linear scales were examined in another study. Faber and Usón (1997) have analyzed "the internal structure of the lexical field of feeling, structured in hierarchies of meaning and subcategorization patterns as the codification of knowledge representation" (p. 36). They utilize "the hierarchical nature of dimension-level schemata" which specify that lexical field to demonstrate "how hierarchical grading reflects conceptual saliency" (p. 36). Faber and Usón (1997) explain these dimension-level schemata and state that the categorization models demonstrated in "the argument structure of feel" are substantiated in the following aspects of lexical category of feeling. They enumerate them as follows by classifying them with regard to their focus on particular constituents of event structure: "(i) to feel a physical/mental sensation; (ii) to feel an emotion (iii) to feel an emotion (focus on entity/event triggering it); (iv) to feel an emotion (focus on the reaction of the experiencer)" (Faber & Usón, 1997, p. 42). In this study, we are solely interested in the third categorizing parameter which focuses the way of speaker's manifestation of his/her feeling towards an entity as our study investigates scalar properties of liking and disliking verbs whose intensity scales were demonstrated in Faber and Usón's (1997) study. Faber and Usón (1997) clarify this categorizing parameter and specify that the dimension-level schemata that are "sensitive to categorization parameter are to feel/experience aversion/dislike/ and to feel/experience attraction/interest" (p. 47). They add that in both cases, there is a clear intensity scale in the side of the hierarchy. They exemplify this situation and state that in the positive side, the loved entity increasingly elevates in position until it is discerned as a god and for negative dimesion, it is also true, but it works in the opposite direction (Faber & Usón, 1997, p. 47). They illustrate the scales of these two verb classes according to intensity of emotion as follows: "To feel attraction/interest: love -> adore -> worship -> idolize -> deify To feel hatred/dislike: dislike -> hate -> abhor -> detest -> execrate -> loathe" (Faber &Usón, 1997, p. 47).
They explain that as a result, these verbs subcategorize the following symbolic formula, or predicate schema which embodies the different semantic and syntactic realizations of the predicates at the bottom of these dimensions: Figure 2. Dimension level predicate schema (Faber & Usón, 1997, p. 47).
They add that "the axiological weight of the second argument (X2) depends on the positive or negative nature of the dimension-level schema that subsumes that predicate" (Faber &Usón, 1997, p. 47). As clearly seen in this dimension-level schema, an obvious scale of intensity exists in parts of the hierarchy for both liking and disliking verb classes.
The studies hitherto have demonstrated that emotion verbs may have different classes and near-synonymous emotion verbs may differ in terms of their intensity of emotion.
Before mentioning the core studies (Fellbaum, & Mathieu, 2012, 2014 which form the basis of our investigation of scalar emotion verbs, it is worthwhile to Levin's verb classification which is the basis of VerbNet as our study lays emphasis on such online lexical databases. Levin (1993) categorizes semantic verb classes that pattern according to syntactic alternations. Levin's verb classes are presumed to share both a set of syntactic alternations and a common semantics. She describes 78 diathesis alternations. In her study, 3,104 English verbs are classified into 49 verb families, and partly divided into 191 sub-classes in terms of alternations the respective verbs undergo. She states that psychological-state verbs typically take two arguments which are frequently characterized as the stimulus (or cause, theme, target of emotion, and object of emotion) and the experiencer. Levin states that it is possible to differentiate 4 classes of psychological verbs in English with regard to expression of the arguments (the experiencer and stimulus): two of these classes are transitive verbs and two of other classes are intransitive verbs that take prepositional complements. The most numerous are transitive psych-verbs which are divided into two classes according to experiencer of the emotion: subject experiencer verbs (the admire verbs) or object experiencer verbs (the amuse verbs). On the other hand, the intransitive psych-verbs are divided into two dintinct classes according to whether the experiencer is articulated "as the subject (the marvel verbs) or as the object of the preposition heading a prepositional phrase complement (the appeal verbs)" (Levin, 1993, p. 189). Now, we turn our focus to the empirical studies which found that emotion verbs have scalar qualities. In two studies (Fellbaum, & Mathieu, 2012, 2014, English emotion verbs such as astonish, fear and surprise were investigated in terms of their subclasses. As Fellbaum and Mathieu (2014) state, "the gradation is richly lexicalized by verbs that denote different degrees of intensity of the same emotion" (p. 99). They examined manually constructed classes of verbs that express different intensity degrees through corpus data.
They found that the chosen emotion verbs indeed have scalar qualities, and the Web data enabled them to construct consistent scales with verbs which were ordered in terms of their emotional intensity. An example of these scales related to fear type emotion verbs is as follows: "Intimidate > alarm > scare > frighten > panic > terrify" (Fellbaum, & Mathieu, 2014, p. 107). Their study shows a way to represent the scalar properties of emotion verbs or other verb classes in WordNet. They suggest that other possible extensions to additional verb families can cause a more subtle semantic analysis of emotion verbs and their developed representation in lexical databases with potential benefits for automatic inferencing, language pedagogy and translation. "WordNet (Miller, 1995;Fellbaum, 1998) is a large lexical database of English" (WordNet, 2010). Verbs, nouns, adverbs and adjectives are grouped into synsets (sets of cognitive synonyms), each indicating a distinct concept.
Synsets are interlinked through conceptual lexical and semantic relations. Verb synsets are grouped into hierarchies in WordNet; verbs which take place at the bottom of the trees show increasingly particular manners describing an event (troponyms). The verbs "{communicate}-{talk}-{whisper}" can be a good example for this. The particular manner expressed depends on the semantic field like intensity of emotion (e.g., like-love-idolize) or speed (e.g., move-jog-run) (WordNet, 2010).
In their another study, Fellbaum and Mathieu (2012) purposed to develop a semantic classification of English emotion verbs empirically and they stated that this classification • "provides a subtle, novel and empirically grounded analysis of a crucial component of the English verb lexicon; • serves as the basis for appropriate representations in lexical resources serving natural language processing, like WordNet; • has the potential to improve automatic text understanding by facilitating the detection of lexically based cohesion and inferencing; • may be represented in WordNet in a way that is consistent with WordNet's structure" (Fellbaum & Mathieu, 2012, p. 105).
They investigated three English verb groups which express the causation of emotions (anger, surprise and frighten). In order to arrive at a placement of the verbs on their respective scales, they used Sheinman and Tokunaga's (2009) patterns ((perhabs) even; not to say; if not …. then) which are exemplified below. Like Sheinman and Tokunaga, they utilized the Web as a corpus. They chose three patterns: (P1) (perhaps) even (e.g., She looked alarmed, even scared by our festive look.) (P2) not to say (This information is good, not to say superb).
(P3) If not …. Then (The idea of going down into the caves scares her, if not scares then intimidates her.) Based on these patterns they constructed a linear scale with fear type emotion verbs which express the causation of emotions as follows: "Intimidate > alarm > scare > frighten > terrify" (Fellbaum & Mathieu, 2012, p. 109).
In order to make a comparison between their automatically derived scales and human judgments, they gathered data from ten native-speaker students studying at Princeton University (Fellbaum & Mathieu, 2012). They gave the students three groups of verbs that express causation of anger, surprise and fear in random order and asked them to construct a scale according to those verbs' intensity of emotion. Native speakers' scales were congruent with Fellbaum and Mathieu's (2012) Web data. They state that a crosslinguistic investigation of emotion verbs' scalar properties may yield interesting variations in terms of lexicalization. They also maintain that it is not likely that all languages will encode the same degrees of a given emotion lexically.
In another empirical study, Sheinman and Tokunaga (2009) have also investigated the scalar properties of adjectives. Sheinman and Tokunaga (2009) introduced AdjScales which is a way of scaling similar adjectives by their strength. In order to automatically make a distinction between similar adjectives which characterize the same quality by strength, their method combined Web-based computational linguistic techniques. They chose some adjective pairs, first selecting one of WordNet's direct antonyms, such as good-bad, and then one adjective that is semantically similar to it from the same side of the scale (e.g., great). They utilized 10 seed word pairs chosen from the adjective scale samples proposed by Fellbaum, Gross and Miller (1993). Figure 3 illustrates an example of their unified scale through AdjScales method. Figure 3. The illustration of unified scale (Sheinman & Tokunaga, 2009, p. 1545 They evaluated their method by comparing with annotation on a subset of adjectives from Wordnet by 4 native speakers of English. The method was also compared with annotations of 2 non-native speakers. It was found that the scales constructed by natives were mostly consistent with AdjScales, but there was less agreement between AdjScales and the scales of non-natives. They suggest that it is important to have a method for grading lexicalized adjectives and especially useful for learners struggling with similar adjectives.
They also state that this sort of information can be extracted more easily than before by means of the Web available as corpora. They list some possible contributions of such scales as follows: • They can be used in the field of language learning tools, • They help learners distinguish between similar words, • Automatic acquisition of adjective scales in construction of ratings for questionnaires for interface design, • Textbook authoring, • Lexical resource enhancement (Sheinman & Tokunaga, 2009).
In a further empirical study, Sheinman, Fellbaum, Julien, Schulam and Tokunaga (2013) proposed "a new semantic relation for gradable adjectives in WordNet" (p. 797). They used several lexical patterns such as "is / are x but not y", "x even / perhaps y" and "if not y at least x" and utilized the Web as corpus for validation of the relative strength of adjectives such as "huge", "large" and "gigantic" in terms of their attribute (size). Their analysis yielded the following adjective scales: tiny > small > smallish; large < huge < gigantic= monstrous. Then they suggested several applications of such scales in WordNet and their benefits for linguists and language learners. These are graphically representation of scales through software programs like WordNet to facilitate lexical acquisition, cross-lingual encoding through mapping scales across languages for machine and human translation, cross-scale relations to extract additional information about words, identifying spam product reviews which are online reviews of products for unhelpful and deceptive purposes, comparing nouns with AdjScales in terms of their shared attributes (Sheinman et al., 2013).
It is obvious that like adjectives, verbs have also scalar qualities, and we can construct consistent scales with verbs which are ordered in terms of their emotional intensity. It may also be useful for language learners to order word groups in terms of degree of intensity in order to learn the words more quickly. Lindstromberg (1985) indicates that it is possible to learn numerous words, and in order to achieve it: people usually acquire ordered groups better and faster than random groups; English words are, to a large extent, grouped into families (or arrays) small and big; we can see this kind of grouping in regard to meaning areas that are internally ordered via sense relations, like antonymy of different sorts (e.g. hot-cold, dead-alive), binary oppositions (e.g. continuous vs. non-continuous, as in tow vs. tug), degree of intensity and inclusion of one meaning by another (p. 235). For example, look at the sentence frame below and the word pairs illustrated in Figure 4, 'I wasn't just__________ , I was positively_________.  (Lindstromberg, 1985, p. 236) (Note: ! stands for a word expressing causation surprise; -stands for a word expressing causation fear or excitement; ? stands for a word expressing causation confusion. The number of the symbols shows the intensity of the given word) If we fill in the blanks first with the words of each pair in the given order in Figure 4, and then in the reverse order, the reverse order clearly yields peculiar sentences. Lindstromberg (1985) claims that this condition provides an evidence for a simple, two-step scaling in terms of intensity, as illustrated by the Figure 4 above-a stair step-type schema-for this kind of ordering. Figure 5. Example of a stair-step schema (Lindstromberg, 1985, p. 240) He also claims that students may not learn emotive words properly if they do not learn this kind of ranking and grouping as shown in Figure 5. He suggests that in a lowintermediate class, the tutor should make sure that students have learnt surprise and excite well, which means, among other things, that the students should be aware that these words are not synonyms as from each word, in terms of English usage, a different 'stair step' will be derived. Unless the students are taught accurately at this level, these words will probably throw them into confusion for later on (Lindstromberg, 1985). Similarly, Sökmen (1997) asserts that classifying or ordering words is a technique that allows students to distinguish differences between words in meaning and organize them to improve retention. When learners are required to make a list of words in a specific order, organizing these words will associate new information with the old and, in turn, establish memory links (Sökmen, 1997).
The literature shows that like adjectives, we can also construct linear scales with emotion verbs based on their intensity of emotion. We have reviewed some empirical studies and we aim to to investigate whether pre-service teachers can construct consistent scales with the scales of previous studies in literature (Fellbaum & Mathieu, 2012;Faber & Usón, 1997) in terms of the order of emotion verbs based on their intensity.
It is seen that here are no studies conducted with Turkish EFL learners on scalar emotion verbs and there is little or no interest in online lexical resources, such as as VerbNet, WordNet, FrameNet, PropBank, VerbOcean and TRIPS that can provide language teachers and learners with useful information for language learning, especially with regard to vocabulary learning. Thus, we aim to explore whether participants from various cultures similarly judge emotion verbs on a linear scale in terms of their intensity, or there are crosscultural differences. In addition, this study may raise awareness of aforementioned online lexical resources among Turkish language learners and teachers.

Participants
The present study was conducted with 38 non-native pre-service English teachers (23 females, 15 males) studying at Yozgat Bozok University, 11 English instructors working at 4 different state universities in Turkey, and 10 native speakers of English (4 British, 6 American). Pre-service English teachers' proficiency level was mostly upper-intermediate.
They were deliberately selected from ELT department due to the nature of the task which would be otherwise too difficult for students at lower proficiency levels. In addition, as preservice English teachers will teach vocabulary as teachers of future, it is assumed that such a vocabulary-based study will be more meaningful and useful for raising their awareness of scalar properties of words and online lexical resources. In order to compare the pre-service teachers' scores with more competent samples, English instructors and native speakers were involved in the study. A convenience sampling method was preferred because of the convenient proximity and accessibility of the participants to the researcher. One of the native speakers is an instructor at Bozok University, so with her help, we could reach 5 more native speakers living in the USA and the UK. We reached other 4 native speakers with the help of colleagues at the same university.

Research Design and Data Collection Tools
A quantitative research design was adopted as the success rates of the participants regarding their scale construction were computed and a comparison was made among the scales of pre-service teachers, instructors and native speakers. A scale construction task was administered in order to gather data from participants' judgements on scalar emotion verbs in terms of their relative order on a linear scale. It is presented in Appendix A. The task is composed of two parts. Part I included three groups of emotion verbs. In the first group, there were fear type verbs: intimidate < alarm < scare < frighten < terrify which were supposed to be placed in the given order. These fear type emotion verbs were scaled in Fellbaum and Mathieu's (2012) study. The second group included 5 liking emotion verbs: like < love < adore < worship < deify; and the third group included 5 disliking emotion verbs: dislike < hate < abhor < detest < loathe whose intensity scales have been demostrated in Faber and Usón's (1997) study. In the second part of the task, there were a total of 18 sentences including three groups of emotion verbs with a multiple-choice test. The first 6 sentences included fear type emotion verbs; the second 6 sentences included liking emotion verbs and the last 6 sentences included disliking emotion verbs. In order to collect data from participants, the Ethics Committee Approval numbered 2020-05 and dated 26.11.2020 was granted. The permission of the university where the research was done was ensured. All the participants were informed about the nature of the study and they all agreed to participate in this study.

Data Analysis
In order to compute the participants' scores regarding the relative order of the verbs on the scale, descriptive statistics were used. In order to compare group means, one-way ANOVA test was used. All the tests were carried out through SPSS Statistics 23 (Statistical Package for the Social Sciences). In the first part of the task, each correct scale was given the value of 4, and each incorrect scale was given the value of 2. In the second part of the task, each correct answer was given the value of 1, so the highest score in this part was 18 as there were a total of 18 items testing three groups of emotion verbs (6 fear type verbs, 6 liking verbs and 6 disliking verbs). For each type of verb class, the highest score was 6.

Procedure
The data were collected from pre-service teachers via direct communication with them during their course hours and it took two days to collect data. The data were obtained from instructors through office visits during a week. The data were collected from native speakers mostly via e-mail and it took 2 weeks to collect data. One of the native speakers was at the same university where the researcher works, so her data were gathered via direct communication and it took an hour to collect data. In the first part of the task, the participants were asked to place the emotion verbs on a scale regarding their intensity of emotion. They were presented in random order in the instruction of the task. In the second part of the task, a total of 18 sentences including three groups of emotion verbs were given to the participants with a multiple-choice test. In each item, participants were asked to choose an appropriate verb that may complete the sentence appropriately among 4 options.
It was stated that more than one option could be possible, and all 4 options could also be possible in some cases. For example, when we consider the example below, all the options can complete the sentences as the pattern "if not… then" allows us to place less intense verbs alarm, scare, frighten and intimidate on the right of more intense verb terrify.
Example: The prospect of change and evolution terrifies many people around the world, if not terrifies then ______________ them.
The first and second patterns were adopted from the patterns used in Sheinman and Tokunaga's (2009) and Fellbaum and Mathieu's (2012)  What he had said was all through rage, he didn't really detest Cooper… just disliked her.
The prospect of change and evolution terrifies many people around the world, if not terrifies then scares them.
After all data were collected from the participants, the researcher entered data into SPSS Statistics and then analysed the data by running statistical tests and interpreted them.

RESULTS
In     in the fourth position (90 %); and terrify in the last position (90 %), which is consistent with the actual scale. The results of these study are congruent with Sheinman and Tokunaga's (2009) study as native speakers' adjscales were more consistent with the actual scale than the scale of non-natives in their study. The results are also in line with Fellbaum and Mathieu's (2012) study as native speakers in their study constructed the same scale regarding fear-type verbs as in the current study.
As for liking verbs, the relative order of the verbs on the scale is: like < love < adore < worship < deify. According to pre-service teachers' judgements, as shown in Table 2  The last group of verbs includes disliking verbs, and their relative order is as follows: dislike < hate < abhor < detest < loathe. Table 3 below shows that most of the pre-service teachers placed dislike in the first position (84.2 %), half of them placed hate in the second position (50 %), but they had difficulty in deciding on which verb to place in the 3 rd , 4 th and 5 th position as the percentages of their ordering the verbs in these positions were close to each other. Instructors also faced the same difficulty in terms of placing the verbs abhor, detest and loathe. Although all of them placed dislike in the first position (100%) and the majority of them placed hate in the second position (90.9 %) as expected, 36.3 % of them placed abhor either in the third or in the fifth position, more than half of them placed detest in the third position and loathe in the fifth position. It seems that they were unsure about these 3 disliking verbs on the scale. Native speakers judged the verb dislike to be the weakest (100%) and loathe to be the strongest (70%) in this scale. They placed hate in the second position (70%) and detest in the fourth position (60%) as in the actual scale; however, they were also unsure about the position of the verb abhor as nearly half of them (40%) placed it either in the third or in the fifth position.
It can be clearly seen that the pre-service teachers and instructors had difficulty in placing the verbs abhor, detest, loathe in the correct position; and native speakers had difficulty in judging the correct position of abhor. This may partly be due to the decrease in the frequency of using abhor over time as seen in Figure 6 below and thus, especially nonnative speakers may not be familiar with this verb. In addition, differentiating the subtle differences between the meanings of the verbs is difficult, and thus such tasks may be perceived as difficult by the participants (Fellbaum, Grabowski, & Landes, 1997).  As shown in Figure 7, the frequency of using loathe is also low and non-native speakers may not be much familiar with its usage, so they had difficulty in deciding on its correct position on the scale. Therefore, it is possible to state that frequency of verb usage plays a role in human judgements. When the participants' overall scores in terms of their correct scale construction were examined, as shown in Table 4, only 7 pre-service teachers could construct a correct scale for fear-type emotion verbs (18.4 %) and 1 pre-service teacher could construct a correct scale for disliking verbs (2.6%). However, more than half of them could construct a consistent scale for liking verbs. Similarly, instructors performed poorly in terms of their correct scale construction for fear-type (36.4%) and disliking verbs (27.3 %) but performed better for liking verbs (63.6%). Native speakers could construct better scales for fear-type (70%) and liking verbs (90%). However, 60% of them could not also construct correct scales for disliking verbs. It was seen that the worst performance was observed in scale construction for disliking verbs among all participants.  In order to find out whether there are significant differences among pre-service teachers', instructors' and native-speakers' success rates in terms of placing emotion verbs on the scale correctly, one-way ANOVA test was conducted. Table 5 above shows that there was a statistically significant difference across pre-service teachers, instructors and native speakers in terms of their correct scales for fear-type verbs (F(2,56)=5,820, p< .05), disliking verbs (F(2,56)= 6,853, p< .05), and their total success rates (F(2,56)= 2,817, p< .05). In order to determine where the difference was among groups, descriptive statistics were examined.
According to Table 6, the most successful group was native speakers, and the least successful group was pre-service teachers in scale construction for three groups of verbs and in terms of their total success rates. It was also seen that there was not a statistically significant difference among groups in terms of their correct scales for liking verbs (F(2,56)=2,086, p=.134). It was clearly seen that all the participants were more successful in constructing correct scales for liking emotion verbs than they did for the other verb families. When the participants' responses to the second part of the task were examined, oneway ANOVA test, as given in Table 7, the results yielded a statistically significant difference across groups in terms of participants' total correct scores for fear-type verbs (F(2,56)=3.140, p< .05), liking verbs (F(2,56)=7.588, p< .05), and disliking verbs (F(2,56)=20.903, p< .05).
Descriptive statistics in Table 8 below further yielded consistent results with participants' success rates in scale construction in the first part of the task. The results similarly revealed that the most successful group was native speakers for all type of emotion verbs; and the least successful group was pre-service teachers.  Our findings imply that it is not so easy to differentiate the scalar emotion verbs as they have subtle meaning differences and the meanings of near-synonyms may be interpreted differently by people from different cultures and the meaning of these words may be different in target language and native language as Sheinman and Tokunaga (2009) also emphasized. For example, Jackson et al. (2019) examined "the meaning of emotion concepts in 2474 languages from 20 major language families" (p. 1517). They discussed whether emotion terms possess the same meaning across cultures, and they determined "the degree of similarity in linguistic networks of 24 emotion terms across cultures" (Jackson et al., 2019(Jackson et al., , p. 1522. They found low similarities and high variability; and "similarity of emotions can be predicted depending on the geographic proximity of the languages, the physiological arousal they evoke and their hedonic valence" (Jackson et al., 2019(Jackson et al., , p. 1522.
Their findings show that interpretation of near-synonymous emotion verbs like in this study may differ across different cultures.

CONCLUSION
This study examined scalar properties of three types of emotion verbs, namely, feartype emotion verbs, liking emotion verbs and disliking emotion verbs. We aimed to find out whether pre-service teachers, instructors and native speakers can construct consistent scales with previously determined scales in Fellbaum and Mathieu's (2012) and Faber and Usón's (1997) studies; and whether the participants' scores regarding scale construction differ significantly. The results revealed that pre-service teachers performed poorly in constructing scales for fear-type and disliking emotion verbs, they were partly successful in constructing consistent scales with previously determined scales for liking verbs. Instructors similarly performed poorly in constructing scales for fear-type and disliking verbs, but they were better than pre-service teachers. They were also successful in constructing scales for liking verbs. Native speakers were successful in fear-type and liking verbs, but they also performed poorly in constructing consistent scales for disliking verbs. We attributed it to the decrease in the frequency of using abhor and loathe, which makes it difficult to decide the position of these verbs on the scale in terms of their intensity of emotion. When the groups were compared in terms of their success rate in scale construction as in the actual scales, a significant difference was found among groups regarding fear-type emotion verbs, disliking verbs and their total success rates. A significant difference was also seen among groups in terms of their responses to multiple choice test. Native speakers were found to be the most successful group whereas pre-service teachers were the least successful group. We may conclude that the participants from various cultures may judge emotion verbs differently on a linear scale in terms of their intensity and there may be cross-cultural differences.
Although the current study provides some valuable implications, it has some limitations too. The second part of the task was found to be quite difficult by the participants as some of them stated that differentiating the verbs with subtle meaning differences was very difficult without knowing the context and they needed more context to decide on the correct verb. Another limitation is that the number of the native speakers and instructors were limited. Further studies with more native speakers and language teachers may be conducted to get more accurate results. Further studies on scalar verbs can be carried out with the help of corpus and then the data can be compared with the human judgements.
Nevertheless, this study provides useful information for the studies on lexical resources such as VerbNet, WordNet, FrameNet etc. This study also provides some implications for students, language teachers, and policy makers. Turkish EFL learners often have difficulty in choosing an appropriate word among near-synonymous words. If they become aware of scalar properties of words in English and verb classification and lexical resources mentioned above, they may improve their vocabulary skills and develop strategies for efficient vocabulary learning. Pre-service teachers can also benefit from such information as they will teach vocabulary in their future classes. When they are aware of lexical resources, they can carry out linguistic studies such as dialog processing, semantic parsing, speech understanding, textbook authoring, and lexical resource enhancement. As "people generally learn ordered groups quicker and better" (Lindstromberg, 1985, p. 235), we may state that ordering the words in terms of their degree of intensity in linear scales or stair-step schemata will help language learners acquire English words quicker and more efficiently. Check (√) the best option(s) to complete the sentences below. More than one option is possible. All the options may be possible in some cases.
(There is a total of 18 sentences including three groups of emotion verbs given above. The first 6 sentences include the verbs: scare / alarm / terrify / intimidate / frighten; the second 6 sentences include the verbs: adore / worship / like / deify / love; and the last 6 sentences include the verbs: loathe / dislike / detest / hate / abhor.) A: 1-The idea of going down into the caves scares her, if not scares then ___________ her.
___ alarms ___ terrifies ___ frightens ___ scares 5-I am still dealing with some level of doubt and fear, the assignment scares and even ___________ me a little. (Fellbaum, & Mathieu, 2012).