Assignment #5 - Creating a Survey

For Assignment #5, I chose to develop a survey that would be used to gather preliminary data for the program evaluation I will plan for our ECUR 809 final project. The program I have been focusing on is called Recess Guardians. The objective of the Recess Guardian program is to increase positive behavior, attitudes and activity levels of students during recess play. Students receive several instructional sessions where instructors (including Husky athlete volunteers) teach students games, leave equipment for recess use equipment to use, and lead students through some peer teaching opportunities.

The survey created would be given to teachers before the program was to be implemented in their classroom, to develop a sense of teacher perception and observation of student attitude, activity levels and peer interactions during recess breaks. The importance of this data would be especially important in order to note changes in these area based on a post-program survey, student/teacher logs, and observations.

It is important to note that I would include greater space for the short answer response questions than shown here. In fact, after our class on Saturday, I signed up for a fluidsurveys account, and definitely see the ease and merit of using such a tool. However, for this survey, it will be a paper version.

Original Survey:

Recess Guardians – Pre-Program Teacher Survey

Teacher name:                                             School name:

Class Information:

1.     What is the grade level of your class?        _______________

2.     How many students are in the class?         _______________

3.     How many males?     _______________

4.     How many females?  _______________

Please use the rating scale provided to answer the follow questions, based on what you observe from the majority of your students. Circle the most appropriate answer:

1.     My students look forward to and enjoy recess breaks.

                                   

Strongly agree             Agree                          Disagree       Strongly Disagree

2.     My students find appropriate activities to fill their recess times.

Strongly agree             Agree                          Disagree       Strongly Disagree

3.     My students are regularly engaged in physical activity at recess

Strongly agree             Agree                          Disagree       Strongly Disagree

4.     My students demonstrate fair play and sportsmanship in the activities they participate in during recess

Strongly agree             Agree                          Disagree         Strongly Disagree

5.     Students in my class have access to adequate amounts of equipment to use at recess.

Strongly agree             Agree                          Disagree         Strongly Disagree

6.     Students regularly use the equipment that is available to them.

Strongly agree             Agree                          Disagree         Strongly Disagree

7.     My students frequently experience conflict with their peers at recess

Strongly agree             Agree                          Disagree       Strongly Disagree

8.     I spend time working to resolve recess conflicts between students at least weekly.

Strongly agree                         Agree                              Disagree                   Strongly Disagree

Please answer the following questions .

9.     Describe some of the activities that students in your class choose to participate in and seem to enjoy at recess throughout the school year.

10. What do you hope your students will gain from participating in Recess Guardians?

After receiving feedback from four colleagues who completed the survey, I was able to see some clarity errors within the first survey, and a few missed points that needed to be added. The following changes were made:

-      -  I added a short explanation at the top of the survey to let teachers know the purpose for the survey

-      -  Roman Numerals were added to separate each of the three areas (class information, rating scale questions, and short answers)

-      -  I had several colleagues comment that while I asked for the breakdown of males and females in the class, there were no other questions that would allow teachers to express or explain if they see a big difference between the recess play of their male students and female students. To account for this, I added an additional short answer question related to the nature of recess conflicts

-       - I changed the rating scale from four indicators to five. I struggled with this a bit, as I did not like the word “neutral” for the middle indicator, as was suggested by one colleague. I did agree, however, that many teachers might struggle to make such a definitive choice between agree (2) and disagree (3). Therefore, I only named the highest (Strongly agree) and lowest (Strongly disagree) ratings, but used numbers for the entire scale.

-      -  I added some criteria I question II. 3 about how teachers might observe or know if students have been physically active

-       - Two additional questions were added to the short answer section regarding types of conflict that occur during recess, at the recommendation of two colleagues

-      -  A “thank you” was added to complete the survey

Revised Survey:

Recess Guardians – Pre-Program Teacher Survey

Teacher name:                                             School name:

Please answer the following questions based on your current classroom grouping. The purpose of this survey is to learn more about your class and how the Recess Guardians program might best impact your students’ activity level, interactions with peers, and attitudes during recess breaks.

I. Class Information:

1.     What is the grade level of your class?        _______________

2.     How many students are in the class?         _______________

3.     How many males?     _______________

4.     How many females?  _______________

II. Please use the rating scale provided to answer the follow questions, based on what you observe from the majority of your students. Circle the most appropriate number from one to five:

1.     My students look forward to and typically enjoy recess breaks.

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

2.     My students find appropriate and productive activities to fill their recess times.

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

3.     My students are regularly engaged in physical activity at recess (they come in looking like they have been breathing harder, they need a drink of water, etc.)

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

4.     My students demonstrate fair play and sportsmanship in the activities they participate in during recess

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

5.     Students in my class have access to adequate amounts of equipment to use at recess.

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

6.     Students regularly use the equipment that is available to them.

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

7.     My students frequently experience conflict with their peers at recess

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

8.     I spend time working to resolve recess conflicts between students at least weekly.

1                              2                           3                       4                           5              

                       

Strongly                                                                                              Strongly

            Agree                                                                                                  Disagree

III. Please answer the following questions .

9.     List some of the activities that students in your class choose to participate in and seem to enjoy at recess throughout the school year.

10.  Describe the types of conflicts that arise during recess play.

11. Explain  how (or if) recess conflicts differ based on gender, social groupings, etc.

12. What do you hope your students will gain from participating in Recess Guardians?

Thank you for taking the time to complete this survey Your input is valued!

Assignment #3 - Planning a PE

Planning document for the evaluation of a 

Saskatchewan based program called 

Recess Guardians 

1.           Engage Stakeholders

Who should be involved?

Stakeholders include Board of Directors, Executive Director (M. McDonald), Volunteer Coordinator, volunteer leaders, teachers, students, administration

How might they be engaged?

Stakeholders will participate in the process of determining the program’s effectiveness through the various data collected (surveys, interviews, anecdotal logs, etc.). They will also play an important role in offering suggestions for improvement

2.           Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

Effectiveness of the  Recess Guardians Program in increasing physical activity, positive behavior and attitudes and leadership of students during recess periods.

What is the purpose of the evaluation?

The purpose of the evaluation will be to determine effectiveness of current programming, and make recommendations for program improvement in goals with regards to student activity, attitudes and behavior during recess periods

Who will use the evaluation? How will they use it?

Who/users	How will they use the information?
Board of Directors	Evaluation will be used to determine modifications that should be made to program and/or use of available funding
Executive Director	Findings will assist in future planning of program operation and delivery. Also, to determine changes that should occur in programming itself, volunteer training, collaboration needed with schools, and decisions to expand or modify program.
Volunteer Coordinator	Data will help in future planning with regards to numbers and types of volunteers required, organization of volunteers and training needed
Volunteer Leaders	Evaluation can affirm role and importance of volunteers, and/or make suggestions for greater effectiveness
Teachers/Administrators (schools)	Evaluation can help determine whether schools will continue to utilize program, and see (and hopefully give input into) recommended changes for improvement
Students	To see data showing the changes (if any) to their activity levels, attitude and behaviors during recess periods

What questions will the evaluation seek to answer?

How does the program affect student engagement in activity during recess periods?

How is student attitude towards activity and sportsmanship impacted by the program?

How does the program foster the development of leadership skills in students?

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
Does the program increase student activity during recess periods?	- Observation of students pre-program and post-program - Increased amount of play on the playground; games taught through program and others - student journals (record of activities played and attitude/feelings towards activity)
Do students exhibit more positive attitudes and sportsmanship when utilizing games taught through the program?	- students demonstrating ability to work cooperatively and with positive sportsmanship - logs (teachers) – are students returning from recess with less incidents of dispute due to lack of fair play, or “getting into trouble” due to lack of productive activity?
Does the program improve student leadership (skills and attitudes)?	- Increased confidence in students working with other students to play together and teach each other - ability to problem solve and work together to play appropriately (less arguments, more working together) -students taking roles to facilitate play, rather than waiting for teacher direction

What evaluation design will you use?

The evaluation will be both formative and summative, utilizing a mix of Stufflebeam’s CIPP model along with Scriven’s purpose driven approach.

 3.         Collect the information 

What sources of information will you use?

Existing information:	Surveys of previous participants (volunteers, teachers, students), previous news articles
People:	Executive director, Volunteer coordinator, students, teachers, volunteers
Pictorial records and observations:	Lessons taught (in class instruction), playground carry-over (games being played, climate of sportsmanship,etc.)

X	Survey		Document review
X	Interview		Testimonials
X	Observation		Expert panel
	Group techniques		Simulated problems or situations
	Case study	X	Journal, log, diary
	Tests		Unobtrusive measures
X	Photos, videos		Other (list)

Instrumentation: What is needed to record the information?

Surveys, anecdotal observation records, video camera, Student journal/teacher logs

When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Survey	X		X
Interview	X		X	X
Observation	X	X
Photos/Video		X
Student journal/teacher log		X	X

Will a sample be used? 

No
Yes	X	If yes, describe the procedure you will use.
Program delivery will be observed, and data collected (pre-program and post-program surveys, interviews, anecdotal student and teacher logs) from three Saskatoon schools. One similarly sized and grade grouped class will be used for the sample (example, 25 grade 4 students from each of three schools). The teacher for each classroom will also be asked to participate in a survey, interview, and in collecting anecdotal observations/tracking log

           Analyze and Interpret

4.  

How will the data be analyzed?

Data analysis methods:	All data (quantitative and qualitative) will be tabulated and sorted, with common themes and findings being noted. Also, unanticipated outcomes will be noted, to include in evaluation reporting
Who responsible:	Evaluator

How will the information be interpreted—by whom?

The evaluator will be responsible for creating surveys, interview questions, observations and analyzing the data

What did you learn? What are the limitations?

Ultimately, the hope is to learn if the program is being successful in meeting its objectives and offering suggestions that can further enhance the program.

Some limitations I foresee include that since the program runs for only two weeks within each school, the evaluator would need to ensure that all pre-program surveys, observations, teacher and student logs and post-program surveys were conducted in a timely fashion, recognizing the demands on teacher instructional time. Also, given the nature of our weather in Saskatchewan, observations may vary greatly if performed within different seasons (a factor to consider when comparing sample results).

	5.

 

    Use the Information

How will the evaluation be communicated and shared?

To whom	When/where/how to present
Board of Directors, Executive director, Volunteer Coordinator	Detailed written report of findings (are objectives being met) and recommendations for change
Teachers, administrators	Short summary statements of evaluation, including summarized suggestions for improvement acquired through student and teacher surveys
Classrooms of participating Students	Recognition and thank you for participation, evidence that their ideas have been heard (from student survey) – Pictures/video presentation

Next steps?

Recommendations given for modification and/or improvement of program

           Manage the evaluation                   Standards

X	Human subject’s protection	X	Utility
	Management chart	X	Feasibility
X	Timeline	X	Propriety
X	Responsibilities	X	Accuracy
X	Budget