Summative & Formative Evaluation Planning

Assignment 1: General Overview of Evaluation Criteria (p. 132)

Evaluation Criteria Explanation Data Source
Appeal: create a connection for user between the formula (daily activities + time management tools = more efficiency.) Review ability to hook the audience by pin pointing real life challenges
  • Teacher KLW survey
  • Application of technologies in the classroom
  • Time on task survey
Usability: ease of user access and use Determine if users are able to navigate the course without any tech support
  • Ability to turn assignments in
  • User interest in taking another similar course
 Effectiveness: ability to utilize the tools as they were taught in the WBI course Determine if participants are able to apply new skills
  • Frequency with which the technologies are applied the classroom

General Evaluation Orientation (p.132)

This course model is oriented toward the participant.  The both formative and summative assessments will be designed to evaluate how well the course has served the participants needs.

Evaluation Matrix: Formative Evaluation Questions. (p140)

Evaluation Criteria & Categories Questions
Appeal
Goals
  • Are the goals relevant to the learner?
  • Is there a clear connection between technology use and time management?
  • Are the goals manageable for learners at varying levels technology proficiencies?
Content
  • Does the information directly address teachers’ daily duties?
  • Is the information applicable immediately?
  • How many people will application of the content impact?
Technology
  • Is the navigation intuitive?
  • Are the materials easy to access?
  • Is the course free of typos?
Message Design
  • Is the course organization easy to understand?
  • Do the videos, graphics, fonts, and colors have universal appeal?
  • Is the course pitched at adult learners of varying age and skill level?
Efficiency
Goals
  • Are the goals stated clearly and concisely?
Content
  • Is the content appropriate for all teaching disciplines?
Technology
  • Is the LMS/web structured properly?
  • Is access to other learners or an instructor an option?
  • Do technology applications outside the course design function easily and efficiently (Google & Weebly)?
Message Design
  • Does the information scaffold on prior skills?
  • Is there a clear organizational pattern?
Effectiveness
Goals
  • How do the goals support best practice instructional strategies?
  • How to the goals reprioritize teachers time?
Content
  • Will the information enable teachers to apply skills immediately or will additional practice be needed?
  • Does the information reflect current instructional practices?
Technology
  • Will the required WBI platform (Edmodo) support learners?
  • Will there be access to an instructor or other learners?
Message Design
  • Is the message design consistent with district instructional practices?
  • Are the directions short and concise?
  • Have screen shots been used according to district standards?

Stakeholders (p.143)

Primary Stakeholders

  • Designer: Responsible for course quality and revisions
  • Instructor: Responsible for informing the designer of snags or challenges in the course
  • Learners: will spend time trying to expunge information from the course
  • Learner’s students: successfully application will directly impact how the students are trained to use these tools.

Secondary Stakeholders

  • ITC department: The course reflects the department’s productivity level
  • School Administration: teachers working smarter reallocates time teachers’ time
  • Parents: Transparency and efficiency = more effective instructional time

What is Being Evaluated? (p.145)

   
  • Motivation strategies
  • Objectives
Design Plans
  • Grouping of objectives
  • Instructions
  • Activities
  • Example Items
Prototype & Website
  • Assessment items
  • Navigation
  • Interface
  • Collaborative Modeling

Who Are The Evaluators and Reviewers (p. 147)

Evaluator/EDTECH 512 Online Course Design Professor:

Youngkyn Baek has his Ph.D., Educational Foundations and Computer Based Instruction. He is a professor at Boise State University and teaches Online Course Design among other online classes.  Areas of Interest and Expertise: Game-based Learning, Instructional Mobile Game Design, Teaching and Learning in Virtual Worlds.

Evaluator/Designer/Instructor: Gillian Riley is an Instructional Technology Coach for SCCPSS. She holds an Education degree and has 7 years of classroom teaching experience. She is also a former Instructional Designer and is ¾ through the M.E.T program at Boise State University.

Expert Reviewer (instructional design, subject matter expert): Wendy Marshall is the Program Manager of the Instructional Technology Department of SCCPSS. She is the former director of the Educational Technology Training Center at Armstrong Atlantic State University.

Expert Reviewer (instructional design): Cris Higginbotham has her Ed.S. in Education and is an Instructional Technology Coach SCCPSS high school teachers.  She headed the adoption of Edmodo as a LMS for professional development and has developed the districts first online courses for teachers.

Expert Reviewer (subject matter): Stephen Routh is Science teacher in SCCPSS. He is partnering with the designer to test implementation logistics. Stephen is an avid user of Google Apps for Educators and is a leader of technology integration at his school and throughout the district.

Expert Reviewer (instructional design): Melissa Milligan is and Instructional Technology Coach at the middle and elementary level for SCCPSS. She has her Ed.S. in Education and is an online instructor.

Expert Reviewer (instructor): Caronia Shafer has 12 years teaching experience and is a professional development coach for SCCPSS. She has her Ed.S. in education and works with the designer as an instructional partner.

End-User Reviewer: Teachers who participate in both the face-2-face course and online version will be asked to give feedback on the course after each module. Learners from of the face-2-face version will have required attendance. Those accessing the content online will have chose to take the course. This should be noted as the responses from participants will be impacted based on how they came to take the course.

Evaluation Matrix Revisions (p.152)

Evaluation Criteria & Categories Questions Methods & Tools

 

Appeal

Goals
  • Are the goals relevant to the learner?
  • Is there a clear connection between technology use and time management?
  • Are the goals manageable for learners at varying levels technology proficiencies?
  • Observation
  • Pre-assessment
  • Pre-requisite survey
Content
  • Does the information directly address teachers’ daily duties?
  • Is the information applicable immediately?
  • How many people will application of the content impact?
  • End-user and expert reviews (SME)
  • Observation
  • Survey
Technology
  • Is the navigation intuitive?
  • Are the materials easy to access?
  • Is the course free of typos?
  • End-user and expert reviews (ID)
  • Survey
Message Design
  • Is the course organization easy to understand?
  • Do the videos, graphics, fonts, and colors have universal appeal?
  • Is the course pitched at adult learners of varying age and skill level?
  • End-user and expert reviews (SME, ID)
  • Survey

 

Efficiency

Goals
  • Are the goals stated clearly and concisely?
  • Expert review (SME, ID)
Content
  • Is the content appropriate for all teaching disciplines?
  • Expert review (SME, ID)
  • End-user review
  • Observation
Technology
  • Is the LMS/web structured properly?
  • Is access to other learners or an instructor an option?
  • Do technology applications outside the course design function easily and efficiently (Google & Weebly)?
  • Expert review (SME, ID)
  • End-user review
Message Design
  • Does the information scaffold on prior skills?
  • Is there a clear organizational pattern?
  • Expert review (SME, ID)

Effectiveness

Goals
  • How do the goals support best practice instructional strategies?
  • How to the goals reprioritize teachers time?
Expert review (SME, ID)
Content
  • Will the information enable teachers to apply skills immediately or will additional practice be needed?
  • Does the information reflect current instructional practices?
End-User and expert (SME, ID) review
Technology
  • Will the required WBI platform (Edmodo) support learners?
  • Will there be access to an instructor or other learners?
Expert review (ID, instructor)
Message Design
  • Is the message design consistent with district instructional practices?
  • Are the directions short and concise?
  • Have screen shots been used according to district standards?
  • Expert review  (ID)
  • Instructor review

Assignment 2:  Summative Assessment (p.163)

 Guiding Questions—Evaluating Appeal

  • Is there a change in appeal of face-2-face professional development vs. WBI?
  • What benefits do participants sight as preference for WBI over face-2-face instruction?
  • What challenges do participants sight a detractor for WBI vs. face-2-face instruction?
Data Source  Rationale & communication method Time-frame
Survey participant perceptions Evaluate the appeal of online professional development to present report to department head and board of education. Face-2-face data collected Spring 2013. Online data collected Summer and Winter 2013.
 Guiding Questions—Evaluating Efficiency

  • Do learners find WBI more efficient than face-2-face instruction?
  • Did learners feel their time was spent more efficiently in WBI or face-2-face course?
Data Source Rationale & communication method Time-frame
Survey participant perceptions Evaluate the efficiency of online professional development to present report to department head and board of education. After a face-2-face and online version have been offered (Winter 2013)
 Guiding Questions—Evaluating Effectiveness

  • Was there a difference in implementation rates between face-2-face participants vs. WBI participants?
Data Source Rationale & communication method Time-frame
ObservationInterview participants Evaluate the efficiency of online professional development to present report to department head and board of education. Conducted Spring 2014
About these ads

One thought on “Summative & Formative Evaluation Planning

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s