Tutorial 7: Resources
Tutorial 6: Beginning the First Cycle
Tutorial 7: Collecting Data
A. Creating a Data Collection Plan​
Research methods are a specific set of procedures or tools for understanding more about how your actions influence the actions of others. You will need to decide what kind of artifacts you are going to collect. To make your decision, revisit your cycle research question. The first part should describe the action you plan to take, and the second part should describe the outcome you expect to affect. Now, look at your outcome and ask yourself what sort of data would help you see the outcome from the eyes of others. How will you get past your own frames of thinking that shape how you "see" the outcome? In action research, it is good to collect more than one form of data and see if, when together, they help you understand what happened from multiple perspectives. In collaborative action research, you may be collecting data in a team.
Here is a list to help you think about the artifacts you might collect:
Perspectives of others: What do they think or say?
-
Interviews
-
Surveys or questionnaires (students, parents, co-workers)
-
Focus groups/ informal discussions/student reflections,
-
Case Studies or subsamples of participants
Observations of Performances of others: What do they do?
--Assessments
-
Students' blogs or journals
-
Assessments of skills (tests, quizzes, homework, report cards, grades)
-
Rubrics (student-created) and student portfolios or projects (see the Buck Institute for examples)
-
Students' self-assessments
-
portfolios
--Record Keeping
-
Checklists of different actions
-
Number of interruptions, disruptions to learning
-
Logs of the number of meetings
-
Minutes of the meetings
-
Photos/videos or other forms of visual data
-
audio recordings (for analysis of ways of talking)
-
Attendance or online participation, or completing rates
-
diaries or journals, field notes
-
photos
Request for assistance or information
-
Sociograms (who gets help from whom)
-
Number/type of requests for help
-
Number/type of questions asked
It is important to make your stance clear. You are not trying to be an objective bystander beyond the politics of the situation. Instead, you are trying to effect change, and you are likely to have strong desires to have the change result in specified outcomes. Your evidence is not meant to "prove to others" that your actions caused the outcomes. Rather it is for you to understand, at a deeper level, how your actions contributed to the outcome. Your honesty is to yourself first and to others second. You want to know what about a given practice is working. If you lie to yourself by overestimating the outcomes, you will fail to learn as much as you might. You are trying to figure out what actions lead to what reactions. Your care in looking at the outcomes from many different perspectives will be vital to your learning.
With this understanding, you are free to collect the evidence that will help you understand what is happening. You can use traditional social science methods or creatively figure out what evidence will help you understand the change.
Follow this next link for some examples: ​
B. Learning How to Collect Data
Action Researchers collect many forms of evidence: photographs, logs, journals, and structured observations. Some of the data collection strategies overlap with evaluation research. The Online Evaluation Resource Library OERL has created a set of innovative tutorials to help researchers new to data collection learn some of these research strategies. While these efforts are meant to train evaluation researchers, they can be very useful for action researchers who use one or more of these methods. Each of the OERL modules contains (1) step-by-step strategies and tools of evaluators, (2) a scenario that helps you see how to apply the tools to an evaluation problem, and 3) a case study in which there are questions to be answered (and then compared to peers and experts). You do not have to leave your names and if some of the methodological concepts are new to you, don't panic. Take the learning you need from the tutorials.
Here is the list--As you can see, many of these will be useful to action researchers.
These tutorials will help you learn to write a simple and brief survey or develop a protocol for an interview that matches your research question. They will also help you avoid simple mistakes (for example, combining more than one question in a single item).
http://oerl.sri.com/module/modules.html
* Designing an Evaluation:
.......Methodological Approach and Sampling
* Developing Written Questionnaires:
.......Determining if Questionnaires Should be Used
.......Writing Questionnaires
.......Questionnaire Design
.......Administering Questionnaires
* Developing Interviews:
.......Preparing an Interview Protocol
.......Administering Interviews
* Data Collection:
.......Procedures, Schedule, and Monitoring
* Instrument Triangulation and Adaptation
* Developing Observation Instruments
THE LISTENING RESOURCE-- A blog
The Listening Resource blogs by Susan Elliot are very useful for developing strategies in qualitative research.
If you create surveys, ensure you have permission and consent to collect this information. If your work involves distributing surveys at the end of training sessions or student activities that involve reflection, you might not have to take any special provisions. However, if you are collecting information from protected groups that are not a part of their educational process, it is imperative that you have the proper oversight (from your principal, district, and/or university) and that you obtain participant consent, and if children are involved, parental consent.
See the discussion of ethics in action research.
C. Building a Valid and Reliable Data Collection Plan
You will need to think about how to assure that the data you collect is a valid measure of your action and that your method of analysis was reliable or a reasonably accurate representation of the data. This chapter by Richard Sagor in Guiding School Improvement with Action Research will help you think through some of the issues around the validity of evidence to measure what you are exploring or the truthfulness of the selected data. The reliability of the measures address issues of accuracy -- did you summary the data to report the date to others in a way that adequately represents what happened.
D. Building Your Knowledge about Research Methods
Whenever possible, it is better to use an instrument that has been validated by others. Searching online can help you find data instruments, but here are some lists to get you started.
National Institute for Learning Outcomes Assessment (NILOA) has as their mission to "discover and disseminate ways that academic programs and institutions can productively use assessment data internally to inform and strengthen undergraduate education, and externally to communicate with policy makers, families, and other stakeholders." This is another great source for figuring out what data to collect and what you learn is likely to also apply to other learners with some adjustments.
The High School Survey of Student Engagement (HSSSE) might be of use as many teachers are concerned about student engagement.
Here is a list of resources for finding assessment tools (mostly in science, technology, and engineering) have been copied from Purdue University's Assessment Center
The following resources provide further information on other databases, additional assessments, research journals.
ATIS: Assessment Tools in Informal Science - Another database of assessment tools. Certain assessments in the Assessment Center link to ATIS as they can be found on their site
AWE: Assessing Women and Men in Engineering - Provides assessment tools for K-16 formal and informal education outreach activities.
Burros Institute of Mental Instruments - A searchable database of evaluations of tests.
The Buck Institute for Education has many great resources for project-based learning and ways to assess the outcomes with rubrics.
ETS Test Collection Database - A searchable database of tests.
MCAS: Massachusetts Comprehensive Assessment System - A database of assessments and questionnaires that test knowledge on technology, science, and engineering.
NAEP: National Assessment of Educational Progress - Largest nationally representative assessment of math, reading, science, writing, the arts, civics, economics, geography, and US History.
PARE: Practical Assessment Research and Evaluation - An online journal that provides access to refereed articles that can have a positive impact on assessment, research, evaluation, and teaching practice.
Tech Tally: Approaches to Assessing Technical Literacy - This book will be of special interest to individuals and groups promoting technological literacy in the United States, education and government policymakers in federal and state agencies, as well as the education research community.
STEM Education Instruments - The ITEST Learning Resource Center has compiled information on various instruments to help researchers, evaluators, and practitioners, identify and locate instruments used to assess learning and other related outcomes in STEM learning environments.
E. Sharing your Action Research Report- Tech Tools
Blogs and Websites are much easier to construct than with earlier software. Using basic word processing skills, you can share your action research with a larger audience by posting your work on a website. Many universities support their alumni by posting their work during and after their program.
While this list is not exhaustive, it will give you some ideas to get you started:
Blogger , Wordpress, and Dropbox are quick and easy ways to create either a private or public blog to document your process. It is a good practice to blog weekly as you move forward.
You can create a website with basic word processing skills (no programming skills needed). There is some new learning about domain names and hosting vocabulary, but fortunately, there are resources to help with this
​
How to Make a Website - A free, step-by-step guide for making a website in an hour or less
​
Here are some links to places you go to start your websites:
Wix -- Wix is easy to use and it has a free version, and if you want a more professional look, it has a reasonable rate. The downside is that the apps that make it more interactive all have a cost associated with them.
Google Sites—Google Sites is an easy choice for those building their first website.
Weebly -- Many find this a good choice. It shares many of the same features as Wix. Wix appears slightly easier but the ease will be determined by which is more similar to programs you have used in the past.