Exercise:Reliability And Validity

Exercise 2: Reliability and Validity

For this exercise, your task is to estimate the reliability and validity of a measure of Need for Cognition (nCog; Cacioppo & Petty, 1982; Cacioppo, Petty, & Kao, 1984).

An SPSS data file is included in the assignment folder, with responses from 294 college students. Within the data file you will find raw scores for nCog items, as well as composite scores for several other variables (see page 2 for descriptions).

Your tasks for this assignment are described below. To answer each of these questions, run the analyses using SPSS and report relevant information in your write-up (i.e., the reliability or validity coefficient). Write a report explaining your findings for each of the questions above. Provide an interpretation of what the results mean. In other words, explain your findings in sufficient detail so that another person could understand what you examined and why.

Reliability

1. Estimate Coefficient Alpha for nCog using the items in the data set.
2. Estimate test-retest reliability for nCog – data were collected 2 weeks after the original nCog data (NCOG_T2).

Criterion-Related Validity 

1. Report the criterion-related validity estimate for nCog using College GPA (CollegeGPA).
2. Report the criterion-related validity estimate for nCog using Student Satisfaction (StudentSat).
3. Report the incremental validity estimate of nCog above and beyond High School GPA (HS_GPA) and ACT Scores (ACT) in predicting College GPA (CollegeGPA).
4. Report the incremental validity estimate of nCog above and beyond High School GPA (HS_GPA) and ACT Scores (ACT) in predicting Student Satisfaction (StudentSat).

Construct-Related Validity 

1. Report convergent validity evidence using Intrinsic Motivation (IntMot) and Mastery Goal Orientation (MGO).
2. Report discriminant validity evidence using Extrinsic Motivation (ExtMot), Performance Approach Goal Orientation (AppGO), Performance Avoid Goal Orientation (AvdGO), and Social Desirability (SocDes).

Variables in Data Set/Assignment

Need for Cognition –described as a drive to understand situations and the world around us (Cacioppo & Petty, 1982; Cacioppo et al., 1984). People with a high need for cognition tend to enjoy problem solving, abstract reasoning, and enjoy complex situations/problems. Items were rated on a 5-point scale (1 = Strongly Disagree to 5 = Strongly Agree), and higher scores indicate higher levels of the construct. Items were taken from Cacioppo et al. (1984), in the attached article.  Note that several of these items are reverse-scored. You will need to re-code them prior to running the analyses.

High School GPA – is cumulative, high school grade point average, on a 4.0 scale. Scores were taken from admissions records.

ACT Scores – ACT scores were gathered from university records.

College GPA – is cumulative, college grade point average on a 4.0 scale. This was taken from university records. In this study College GPA was collected one year after Need for Cognition was measured.

Student Satisfaction – is the extent that students are satisfied with their experiences at the university. Six items were written for the purpose of conducting this study. Cronbach’s alpha for this scale was .90. In this study student satisfaction was collected one year after Need for Cognition was measured.

Academic Motivation focuses on the reasons that students go to college. This is based on Deci and Ryan’s (1985) self-determination theory (Ryan & Deci, 2000). For the purposes of this data collection ten items were written to measure intrinsic motivation and ten items were written to measure extrinsic motivation.

• Intrinsic Motivation – is based on a drive to learn and accumulate knowledge. The process of learning itself is internally motivating for individuals with a high level of intrinsic motivation. Coefficient alpha for this scale was .92.
• Extrinsic Motivation – is based on the anticipation of some other, external reward, Individuals with a high level of intrinsic motivation are less interested in learning itself, but motivated because with a college degree they are more likely to obtain a high-paying job or promotion at work. Coefficient alpha for this scale was .87.

Goal Orientation – reflects the purpose, or type of goal that a person aims to achieve when pursuing a particular task (Elliot et al., 1999). This can be composed of mastery goal orientation, performance approach goal orientation, and performance avoid goal orientation. Seven items were written to measure each of the three dimensions below.

• Mastery Goal Orientation – has a focus on improving oneself as a function of task pursuit, and is often associated with deep-level processing and drive to learn and understand material (e.g., expanding one’s knowledge). Coefficient alpha for this scale was .90.
• Performance Approach Goal Orientation – is associated with achieving positive outcomes and successful task accomplishment. However, this is more often associated with surface-level processing, with a focus on simply obtaining task outcomes (e.g., getting a good grade). Coefficient alpha for this scale was .83.
• Performance Avoid Goal Orientation – is also focused on achieving outcomes, but specifically on avoiding failure or negative outcomes. Instead of focusing on learning material or getting a favorable grade or expanding one’s knowledge, the focus remains on not failing a class, for example.   Coefficient alpha for this scale was .89.

Social Desirability – involves providing what would be perceived as favorable, or socially desirable, responses (Crowne & Marlowe, 1960). The KR20 internal consistency estimate for this scale was .87.
References

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131.

Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 3, 306-307.

Crowne, D.P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349–354.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.

Elliott, A. J., McGregor, H. A., & Gable, S. (1999). Achievement goals, study strategies, and exam performance: A mediational analysis. Journal of Educational Psychology, 91, 549-563.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78.

ADDITIONAL INSTRUCTIONS FOR THE CLASS

Discussion Questions (DQ)

• Initial responses to the DQ should address all components of the questions asked, include a minimum of one scholarly source, and be at least 250 words.
• Successful responses are substantive (i.e., add something new to the discussion, engage others in the discussion, well-developed idea) and include at least one scholarly source.
• One or two sentence responses, simple statements of agreement or “good post,” and responses that are off-topic will not count as substantive. Substantive responses should be at least 150 words.
• I encourage you to incorporate the readings from the week (as applicable) into your responses.

Weekly Participation

• Your initial responses to the mandatory DQ do not count toward participation and are graded separately.
• In addition to the DQ responses, you must post at least one reply to peers (or me) on three separate days, for a total of three replies.
• Participation posts do not require a scholarly source/citation (unless you cite someone else’s work).
• Part of your weekly participation includes viewing the weekly announcement and attesting to watching it in the comments. These announcements are made to ensure you understand everything that is due during the week.

APA Format and Writing Quality

• Familiarize yourself with APA format and practice using it correctly. It is used for most writing assignments for your degree. Visit the Writing Center in the Student Success Center, under the Resources tab in LoudCloud for APA paper templates, citation examples, tips, etc. Points will be deducted for poor use of APA format or absence of APA format (if required).
• Cite all sources of information! When in doubt, cite the source. Paraphrasing also requires a citation.
• I highly recommend using the APA Publication Manual, 6th edition.

Use of Direct Quotes

• I discourage overutilization of direct quotes in DQs and assignments at the Masters’ level and deduct points accordingly.
• As Masters’ level students, it is important that you be able to critically analyze and interpret information from journal articles and other resources. Simply restating someone else’s words does not demonstrate an understanding of the content or critical analysis of the content.
• It is best to paraphrase content and cite your source.

LopesWrite Policy

• For assignments that need to be submitted to LopesWrite, please be sure you have received your report and Similarity Index (SI) percentage BEFORE you do a “final submit” to me.
• Once you have received your report, please review it. This report will show you grammatical, punctuation, and spelling errors that can easily be fixed. Take the extra few minutes to review instead of getting counted off for these mistakes.
• Review your similarities. Did you forget to cite something? Did you not paraphrase well enough? Is your paper made up of someone else’s thoughts more than your own?
• Visit the Writing Center in the Student Success Center, under the Resources tab in LoudCloud for tips on improving your paper and SI score.

Late Policy

• The university’s policy on late assignments is 10% penalty PER DAY LATE. This also applies to late DQ replies.
• Please communicate with me if you anticipate having to submit an assignment late. I am happy to be flexible, with advance notice. We may be able to work out an extension based on extenuating circumstances.
• If you do not communicate with me before submitting an assignment late, the GCU late policy will be in effect.
• I do not accept assignments that are two or more weeks late unless we have worked out an extension.
• As per policy, no assignments are accepted after the last day of class. Any assignment submitted after midnight on the last day of class will not be accepted for grading.

Communication

• Communication is so very important. There are multiple ways to communicate with me: 
  • Questions to Instructor Forum: This is a great place to ask course content or assignment questions. If you have a question, there is a good chance one of your peers does as well. This is a public forum for the class.
  • Individual Forum: This is a private forum to ask me questions or send me messages. This will be checked at least once every 24 hours.

Exercise:Reliability And Validity

Exercise:Reliability And Validity

Exercise:Reliability And Validity