CHR 8604.010 – Research & Evaluation

Monday and Wednesday

5:15 p.m. to 8:45 p.m.

Bartley Hall Room 024

Credit Hours: 3

Professor: Dr. Edward Garcia Fierros

Phone: 610 519-6969

Email:  Edward.fierros@villanova.edu.

Office: St. Augustine Center for the Liberal Arts Room 359

Office Hours: by appointment – please email me at Edward.fierros@villanova.edu or leave me a phone message

URL: http://www.homepage.villanova.edu/Edward.Fierros

“no natural history can be interpreted in the absence of at least some implicit body of intertwined theoretical and methodological belief that permits selection, evaluation, and criticism.” Kuhn

Required Texts:

o   Readings listed below or assigned in course.

Course Description: Analysis of diverse types of research and evaluation; familiarization with research resources and literature survey procedures; evaluation of counseling research; its implications for, and application to, educational and counseling practice; emphasis on the scientific methods of investigation; development of competency to conduct a research study and write a research paper.

PRE-COURSE ASSIGNMENT:

1.    Write a personal response to the following questions. Responses will be collected on the second day of class, July 2, 2008

A. Where do you get most of your information about the latest “best practices” in education? In other words, what kind of information affects your own practice?

B. If you were faced with a situation in which a research study had shown that one counseling technique was effective, yet you found that the technique did not work for you last year, what would you do? Would you try the technique again? Consider the research faulty?

C. What does it take to be a great counselor? How much of counseling is about content? How much about relationships?

COURSE REQUIREMENTS:

1. Discussion of responses submission of responses to the pre-course questions.
2. Successful completion and submission of all course activities.
3. Attendance and participation at all classes. 

4. Successful completion of a midterm. The exam will be based on information from course readings and class work. The midterm will be held during class on Monday, July 14, 2008.

5. Successful completion of a research proposal. The research proposal is due on the last day of class, Monday July 28, 2008.

COURSE OBJECTIVES

Students will be able to

1. Know and describe the theoretical and practical principles of research;
2. Explain and put to use the principles and phases of research;
3. Develop skills to be good consumers of published research studies;

4.    Demonstrate how the research process can be effectively used to answer educationally relevant questions.

5.    Know and describe the requirements of completing a research study.

COURSE TOPICS

1. Defining the research problem and understanding research terminology

In this session we will discuss basic research vocabulary (e.g., operational definitions, different kinds of variables), and discuss how to develop researchable questions and testable hypotheses.

2. Qualitative v. Quantitative research methods

In this session we will discuss the epistemological and methodological differences between qualitative and quantitative approaches to research. We will discuss the idea that the appropriate approach depends upon the nature of the research question under investigation.

3. Reviewing the literature

We will discuss the advantages and disadvantages to using established databases (e.g., ERIC, PsychInfo) as well as online resources for locating related research, as well as different strategies for locating relevant literature. In addition, we will discuss the difference between empirical research and research reviews, as well as primary and secondary sources.

4. Instrument design, reliability, and validity

We will discuss how to write good questions for surveys and achievement tests, and how to avoid common item writing pitfalls that lead to increased measurement error. In addition, we will discuss various kinds of reliability (e.g., internal consistency, inter-rater reliability) and validity (content, criterion, and construct), paying particular attention to they way in which these concepts can help to distinguish good research from poor research.

5. Sampling

We will discuss various approaches to and requirements for sampling within the context of both quantitative and qualitative approaches to research.

6. Ethics

We will discuss key studies through the years that gave rise to the development of a code of ethical conduct for research. We will discuss the importance of obtaining informed consent from participants as well as for maintaining participants’ confidentiality. 

7. Experimental Design

We will discuss the ways in which true experimental and quasi-experimental designs for research can be used to rule out various threats to the internal validity of a study (e.g., maturation, selection.

8. Analysis - Descriptive statistics

We will discuss the importance of coming to know the data by first exploring the results on a descriptive level. The utility of measures of central tendency and measures of variability will be discussed along with a discussion of the properties of the normal distribution and its utility.

9. Analysis - Inferential statistics

We will discuss various methods for comparing groups on outcomes of interest. In addition, we will discuss the importance of statistical power and the use of effect size estimates to describe the importance of group differences.

COURSE EVALUATION

Pre-course written work = 10 %

Class Attendance and Cooperativeness = 15%  

Participation in class work and submission of written work for class activities = 20%

Midterm = 25%

Final examination (i.e., Research Project) = 30%

Total Possible Points = 500 Possible Points 

Letter Grades:

                 A   = 93 - 100%

A-   = 90 - 92.5%

B+ = 87 – 89.5%

B   = 83 – 86.5%

B-   = 80 – 82.5%

C+ = 77 – 79.5%

C   = 73 – 76.5%

C-  = 70 – 72.5%

Writing Guidelines: The course reflection papers will be graded using the following guidelines:

Late Assignments: If you must turn in an assignment late, you are expected to notify me in advance or to be able to provide some proof of your trouble. If you do not satisfy these criteria, submit your assignment! Assignments/papers that are late will lose 10% of their value for each day they are late.

 

Academic Integrity: As a community committed to the Augustinian ideals of truth, unity and love, Villanova University prides itself on maintaining the highest standards of academic integrity and does not tolerate any form of academic dishonesty or misconduct. Dishonesty (including plagiarism) in any assignment, test or examination is punishable by the grade of F and is to be reported, through the deans, to the Vice President for Academic Affairs. A second offense will result in the dismissal of the student from Villanova University.

Students with Special Needs

The Office of Learning Support Services, in conjunction with faculty, provides reasonable accommodations for students with various special needs. Often students are hesitant to request support since they have done so well through high school and are now part of a competitive university. However, disabilities are not something that you outgrow - it is something that you learn to master.

In order to obtain accommodations, students must register with the Learning Support Office by submitting current documentation. To ensure confidentiality, students must complete a Request for Accommodation Form each semester at Villanova in order to receive accommodations during that semester. Nothing can be shared with your professors without your permission. Call or email the office for an appointment. Phone: 610-519-5636 E-mail: nancy.mott@villanova.edu

COURSE FORMAT

The course will include multiple instructional approaches including: lecture, discussion, small group activities, presentation, and critical reflection activities.

Students will be asked to conduct thoughtful discussion of the material assigned for the day’s reading. The course experience will be cooperative in nature with students working in small groups toward resolution of issues and debates raised in class and in the readings. These are not intended to exhaust the potential themes in the course, only to suggest some possible directions for exploration.

COURSE REQUIREMENTS:

 Mid-Term Examination – July 14, 2008

Your mid-term examination will include multiple-choice, short-answer constructed response, and extended constructed response items (i.e., questions).

Final Project - Research Proposal – Due July 28, 2008

Based on your readings and experiences, propose a research project that could be conducted at your school. The project must include:

1)            a statement of purpose

2)            a rationale for why it should be studied
3) review of the related research literature description of the data collection methods which includes design, selection of subjects, instruments, and procedures

4)            description of proposed data analysis

5)            proposals for possible implementation plans and how they would be evaluated.

Your Action Research Proposal should be in American Psychological Association (APA) style, 9-12 pages long, typed and double-spaced. 

 

Tentative Calendar – The professor reserves the right to change the syllabus at any time during the semester. It is your responsibility to note these changes.

Dates

Topic

Required Readings (to be read before class) and available on my website at http://www.homepage.villanova.edu/Edward.fierros

What is Research and Why Should We Do It?
The term “research” is often used rater loosely in our society. But what exactly do we mean when we talk about “research”? What is that separates research from other ways of knowing? How can research be used to improve our practice as counselors?

6/30

Syllabus Review - Questions?

Complete Course Pre-Test

Conducting Library Searches

Literature Reviews

Review of Research in Education (EJournal available via Falvey Library Website)

U.S. Regional Educational Laboratories

http://www.relnetwork.org/

Allison, P. & Pomeroy, E. (2000, Fall). How shall we “know?” Epistemological concerns in experiential education. The Journal of Experiential Education, 23. (2). p. 91-98.

 “Allison Pomeroy Know” What is Research? Activity

McMillan, J.H. (2004). Educational research: Fundamentals for the consumer. 4^th Edition. Boston, MA: Pearson Allyn& Bacon. p.49-76.

“McMillanReviewLiterature”

7/2
FULL MOON

Ethics in Research

Ethics Presentation

APA Guidelines for Ethical Research (http://www.apa.org/ethics/code.html)

Stanford Prison Study (http://www.prisonexp.org/)

Participant Commentary- Do the Ends Justify the Means: The Ethics of Deception in Social Science Research

http://www.onlineethics.org/CMS/research/rescases/gradres/gradresv1/justify/justify-c1.aspx

Ethics and Access
Professional societies serve and important function for researchers and society in general. These societies are bound together by a shared devotion to a particular code of conduct, known as ethics. Within the field of Counseling Research, there is an established code of conduct that is shared by all legitimate researchers. This section will discuss how the necessity of ethical codes, and whether or not they are truly necessary.

Group Assignment. You will be assigned to one of 3 groups. Each group will read and focus on one controversial experiment. Each group member will then write a brief reaction to the study they read. Then each group will share a short summary of their controversial experiment and present the ways the experiment supports or violates ethical standards. The audience will expected to play Devil’s advocate if everyone seems to be on the same side. In other words, what were the benefits to the study? What were the consequences? Was it worth it?

7/7

Qualitative vs. Quantitative Research

Thomas, R.M. (2005). Research Methods, Qualitative & Research Methods, Quantitative. Teachers doing research: An introductory guidebook. Boston, MA: Pearson. p. 45-90.

• Thomas QualQuan1a
• Thomas QualQuan1b

 “Thomas QualQuan1a and 1b”

An Overview of the Methodological Approach of Action Research http://www.web.net/~robrien/papers/arfinal.html

Defining the Research Problem
The first step in any research project involves brainstorming about the topic area you wish to investigate. All of us have had experience in school, community, or clinic settings, and we all have ideas about how things might have been done better. What are some ideas that you have about how to improve education?

Consider a potential research problem that you might want to explore this semester. Submit your list along with methodological approaches – Due 7/9/08.  Note the possible pros and cons to your question.

7/9

Selecting and Refining a Topic

Data Collection

Studer, J., Oberman, A., & Womack, R. (2006). Producing evidence to show counseling effectiveness in schools. Professional School Counseling, 9 (5) 385-392.

Millman, J. (1998, Summer). Strategies for identifying a research topic in educational measurement. Educational Researcher. Washington, DC: American Educational Research Association. p. 37-39.

Data Collection

• http://www.nmsca.org/cpg/use_of_data.htm
• http://www.schoolcounselor.org/files/execsumm.pdf
• http://www.vermontinstitutes.org/equity/schools_striving/criteria8.html#vhs

7/14

Content Analysis; Action Research

MIDTERM

 McAllister, G. & Irvine, J.J. (2002, Nov/Dec). The role of empathy in teaching culturally diverse students: A qualitative study of teacher’s beliefs. Journal of Teacher Education, 53. (5) p. 433-443.   “McAllister Content”

Rowell, L. (2006). Action research and School counseling: Closing the gap between research and practice. Professional School Counseling, 9 (5) 376-84.

7/16

Article Critique #1 Due

Qualitative Design

Marie C. Hoepfl, M.C. (1997). Choosing qualitative research: A primer for technology education researchers. Journal of Technology Education, 9. (1).  

http://scholar.lib.vt.edu/ejournals/JTE/v9n1/hoepfl.html

Farber, N.K. (2006). Conducting qualitative research: A practical guide for school counselors. Professional School Counseling, 9 (5) 367-375.

Part of good planning is anticipating potential problems and then doing what you can to prevent them.

7/21

Quantitative Design – Descriptive, Comparative, Correlational, Causal-Comparative Research

Porter, A.C. (1997). Comparative Experiments in Education Research in Methods for research in Education 2^nd Edition. Washington DC: American Educational Research Association. p. 523-585.

1.  Porter 1

2.  Porter 2

Ware, W. & Galassi, J. (2006). Using correlational and predication data to enhance student achievement in K-12 schools: A practical application for school counselors.

7/23

Article Critique #2 Due

Data Collection

Instrumentation; Survey Research

McMillan, Ch. 7

Data Collection http://gse.gmu.edu/research/tr/TRcollection.shtml

Williamson, J.B., Karp, D.A., Dalphin, J.R., & Gray, P.S. (1982). The research craft: An introduction to social research methods. 2^nd Edition. p. 125-162.

  “Williamson Survey”

7/28

Final Project Due

Now that you have decided what topic area you wish to explore and you have a few research questions of interest, it is time to get up to speed on the latest thinking in your field. In developing your Action Research Pre-Project (i.e., rough draft, project introduction/rationale, & literature review) you will explore various approaches for finding out what has been done already in your research area. You will also exchange your paper with a fellow classmate so that you can critique their work.