Evaluation Strategy & Rationale
The primary goal of our evaluation strategy is to assess the efficacy of the course at achieving its learning and engagement goals.
Curriculum Goals
The Introduction to Social Robotics curriculum is built around four core goals:
- Hands-on Experience: Provide practical, direct experience with building and styling social robots.
- Attitude and Awareness: Increase learners’ interest, confidence, and awareness of technology and social robots.
- Conceptual Distinction: Help learners recognize the core distinction between simple automation and interactive robotics.
- Interdisciplinary Nature: Encourage learners to recognize how social robotics blends technology with creative writing, storytelling, and interaction design.
Relevant Methodologies
We draw on established academic and educational frameworks to structure our assessments:
1. The ARCS Model of Motivation
Developed by John Keller, the ARCS model helps us monitor and optimize learner motivation along four dimensions:
- Attention: Keeping learners engaged and curious.
- Relevance: Making sure the content links to learners’ personal interests and real-world experiences.
- Confidence: Empowering learners to build skills and succeed independently.
- Satisfaction: Celebrating completed projects and showcasing achievements.
2. ELIPSS Framework
We use principles from the ELIPSS (Evaluation of Learning in Physics/Science Spaces) rubrics to qualitatively assess learners’ open-ended text reflections, tracking conceptual progress and problem-solving growth.
Evaluation Strategy
Our multi-faceted assessment strategy tracks progress across three key data channels:
1. Pre and Post Camp Surveys
- What: Learners rate their interest and confidence levels and answer open-ended questions about technology.
- Goal: Compare responses before and after the program to measure shifts in attitudes and conceptual understanding.
2. Module Reflections
- What: Learners answer quick module-specific prompts in their portal textboxes regarding engagement, design challenges, and future plans.
- Goal: Track learner growth over time and gather immediate feedback on specific module activities.
3. Learner Work Products
- What: Completed designer sheets, storyboards, physical robot bodies, and their live presentation demos.
- Goal: Qualitatively analyze learners’ skill acquisition and conceptual understanding.
Facilitator Evaluation
Facilitators also participate in self-reflection and tracking:
- Preparation check: A pre-camp checklist and preparation survey.
- Module reflections: Quick reflections after running each module to note what went well and what challenges arose.
- Wrap-up review: A post-camp survey summarizing overall class outcomes.
Survey Schedule & Order
To ensure clean data collection, please administer evaluations in this order:
| Timing |
Participant |
Survey |
| Day Before Camp |
Facilitator |
Facilitator Preparation Survey |
| Beginning of Camp (Day 1) |
Learners |
Learner Pre-Survey |
| End of Each Day/Module |
Learners & Facilitators |
Learner Module Reflection & Facilitator Module Reflection |
| Last Day (Post-Presentations) |
Learners & Facilitators |
Learner Post-Survey & Facilitator Camp Survey |
Strategy Limitations
[!NOTE]
Acknowledge Selection Bias: Since this program is elective, learners who sign up for the camp are likely already interested in STEM, coding, or robotics. Evaluators should account for this baseline interest when analyzing pre/post shifts.
Acknowledgements
We would like to thank:
For vetting and recommending much of the metrics and surveying approaches used in this curriculum.