Frequently Asked Questions (Fall 2023)

MASDER Study

  1. What is this study? 

The MASDER (Motivational Attitudes in Statistics and Data Science Education Research) team is studying attitudes in statistics and data science education. For the past 7 years, we have been developing a family of instruments to measure student attitudes and instructor attitudes in introductory statistics and data science courses at the undergraduate level. We are also collecting data about characteristics of the learning environment. We are using these instruments to collect nationally representative data and link it with measures of achievement. 

  1. What is happening in Fall 2023? Why am I seeing this?

In Fall 2023, we are collecting data from a nationally representative sample of introductory statistics courses: YOU have been selected to participate! This is the last and most critical phase of the project, and we hope you will agree to participate by sending surveys to your students and completing a few surveys about your attitudes and courses.

  1. What is happening in Spring 2024?

In Spring 2024, we will be collecting data from a nationally representative sample of introductory data science courses. This FAQ currently focuses on the statistics data collection because it is happening sooner. (If you teach both statistics and data science, then we may ask you about both in Fall 2023.)

  1. Who is sponsoring this study? Who are the researchers?

The MASDER study is funded by the National Science Foundation under Grant No. DUE-2013392. The study is under the NSF’s Improving Undergraduate STEM Education (IUSE) program. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or our host institutions. 

You can read more about the MASDER research team on our website.

  1. Why was I selected for this study?

You were identified as an instructor of an introductory statistics or data science course at a college or university in the United States. In order to collect data that is widely representative of undergraduate introductory statistics and data science courses in the US, we are conducting a stratified random sample of such courses and instructors. Your institution, course, and you, as an instructor, were randomly selected to participate in this study. The sampling frame was determined based on data from the College Scorecard and the Carnegie Classification systems.

  1. Why should I agree to participate in this study?

Your specific course matters! Our strata have been created to provide information on a variety of educational experiences, structured around highest degree offered, minority-serving institution status, and school selectivity. Without you and your course, your type of institution will be underrepresented in the study. We need your help to reduce non-response bias. Participation in this study will contribute to the improvement of teaching methods and materials in statistics and data science, by contributing to the creation of instruments to measure important aspects of motivation and attitudes of students, instructors, and environments which impact student learning and retention.

After the term ends, you will receive a customized report showing how students’ attitudes in your class(es) compare to the nationally representative data. 

Also… if you participate, you will receive a $125 Amazon gift card! There is an additional $25 gift card available for opting into an additional component of the study (see below). 

  1. What am I being asked to do? What is the time commitment?

This study takes place during one class term, such as a semester or quarter. Prior to the start of your term, you will be asked to complete some surveys about your teaching and your course (I-SOMAS and Pre E-SOMAS). Within the first week of the term, you will give your students a link to the student attitudes survey (S-SOMAS). (Students can take the survey outside of class time.) At the end of your term, you will again administer the S-SOMAS survey to your students and you will complete one more survey about your class (Post E-SOMAS). If you opted in for the additional $25 gift card, then at the end of the semester, you will also administer the content assessment, called BLIS, to your students. 

As an instructor, you will spend about 45 minutes in total completing surveys, and about 30 minutes total sending surveys to your students. Students will spend about 30 minutes total completing attitudes surveys. If your students also complete the BLIS content assessment, it will take them about 50-70 minutes. 

SOMA Surveys

  1. What surveys are being administered? How long do the surveys take to complete?

We are developing a family of six instruments called the SOMA – the Surveys of Motivational Attitudes (SOMA). The focus of the instrument is indicated with a prefix (S for Student, I for Instructor, E for learning Environment), and the audience is indicated with a suffix (S for Statistics and DS for Data Science). The student instruments measure student attitudes about learning statistics or data science, respectively. The instructor instruments measure instructor attitudes about teaching statistics or data science. The environment survey collects information on the course learning environment and is completed separately for each distinct course that is participating in the study. 

Student InstrumentInstructor InstrumentEnvironment Inventory
StatisticsS-SOMASI-SOMASE-SOMAS
Data ScienceS-SOMADSI-SOMADSE-SOMADS
TimelineStudents complete Pre and Post Survey in one course term.Instructors complete one time.Instructors complete Pre and Post surveys in one term. 
Time Commitment15 minutes each time15 minutes10 minutes for Pre20 minutes for Post*
*(Additional time if you have multiple courses participating.)

In Fall 2023, the Statistics instruments are being administered (S-SOMAS, I-SOMAS, and E-SOMAS). In Spring 2024, the Data Science instruments are being administered (S-SOMADS, I-SOMADS, and E-SOMADS). 

  1. What kinds of questions are asked on the surveys?

The S-SOMAS and I-SOMAS gauge respondents’ attitudes about learning statistics and teaching statistics, respectively. These items are answered using a 7-point Likert-type disagree/agree scale and are designed to measure several components of attitudes and motivation aligned with an established psychological framework. They also include demographic-type questions.

  • Some example S-SOMAS items are:
    • Understanding statistics gives me a sense of satisfaction.
    • I need to know statistics because someone important to me wants me to.
    • I value statistics because it makes me an informed citizen.
  • Some example I-SOMAS items are:
    • Teaching statistics is my passion.
    • I am confident in my ability to help students learn statistics.
    • I teach statistics to prepare students for successful careers.

The E-SOMAS includes questions to gather information about the way a statistics course is being taught (pedagogy and learning environment). Some example E-SOMAS items are:

  • What is the mathematical prerequisite for this course?
  • How many students do you expect to enroll in this class?
  • To what extent do you regularly address ethical issues related to statistical practice and data collection?
  • What type of statistical software are students required to use in this course when analyzing data?

The Data Science survey items closely parallel the statistics surveys. 

  1. Why do we need new attitude surveys when some already exist?

While some surveys for measuring student attitudes about statistics exist (such as the SATS-36), the field of statistics education has advanced substantially since these were released. Data science is a young discipline and no surveys exist that have published evidence supporting their use. Surveys of instructors’ attitudes toward statistics and data science also have not been made before: existing surveys focus on pedagogical practices or self-efficacy. We are developing a cohesive set of survey tools that can be used to examine the entire learning environment so we can understand how student attitudes are related to instructor attitudes.

Study Logistics

  1. What are the criteria for receiving a gift card?

There are two levels of participation that will receive gift cards.

In order to receive a $125 Amazon gift card for participating in the study, the following criteria must be met by the end of your course term:

  • Complete the I-SOMAS (instructor attitudes survey)
  • Complete the Pre and Post E-SOMAS (learning environment inventory)
  • Administer the Pre and Post S-SOMAS (student attitudes survey) to your students and receive at least a 75% completion rate each time

If you additionally assign the BLIS student content assessment at the end of the term and receive at least a 75% response rate, you will receive an additional $25 gift card. 

  1. How will I access all the surveys?

We have developed a web portal that will be used to access the surveys. After completing the Google Form, instructors who wish to participate will be sent the URL for the portal. 

  1. How should I assign the surveys to my students? 

Via the web portal, you will receive a link to the survey to share with the students in your course. We will also provide you with a script that you should use in communication with students when soliciting their participation. We strongly recommend making the S-SOMAS survey a class assignment and giving students credit for participation because this helps to reduce non-response bias. Assigning the survey for credit is not required; you could also assign for extra credit or no credit. However, assigning for credit is the most reliable way to receive a high response rate.

  1. Can I get the students’ names for assigning credit?

Yes, you can get the students’ names. We will email you a list of names of students who completed the consent form so that you can give these students credit. Note: students do not have to complete the survey to be included on the list of names you are sent. There should not be any penalty for not giving consent.

  1. Will I be able to see my students’ results?

At the end of the term, you will receive a customized report showing aggregated results from your class with a comparison to national averages on the S-SOMAS. At the end of the grant period, we will make de-identified survey data publicly available, but we are not able to share the raw data from any particular course for privacy reasons.

If you opt in to administering the BLIS content assessment, we will return the student scores to you, but not the full assignment results. 

  1. Does this study have IRB approval? 

This project has been approved by the Institutional Review Board (IRB) at California State University, Monterey Bay. If you would like to connect with your IRB before administering the survey, you may provide them with this folder that includes information they might need to determine your eligibility to administer the survey.

  1. Can any instructor participate in the Fall 2023 study?

Only introductory statistics courses at the undergraduate level that have been selected as part of our random sample can be included in this study. While we intend for the instruments to be useful in a broader context (e.g., with upper-level courses or graduate introductory statistics courses), the validation work for this phase of the project has a carefully defined scope. In future years, the surveys will be publicly available for broader use.

BLIS Content Assessment

  1. What is the BLIS content assessment?

The Basic Literacy In Statistics (BLIS) assessment is a statistics test designed for students enrolled in post-secondary introductory statistics courses. This assessment was developed by Dr. Laura Ziegler (Iowa State University) and can be used in many types of statistics courses (including courses that include simulation-based methods and those that do not include simulation-based methods).

  1. What will I be asked to do if I agree to administer the content assessment?

Administering the content assessment (BLIS) is an optional add-on to this study. If your students do complete BLIS with a 75% completion rate, you are eligible for an additional $25 gift card.

If you agree to administer BLIS, you will access a link to BLIS via our website. We will provide you with a script that you should use in communication with students when soliciting their participation. You will need to assign BLIS for credit during the last week of class in your term. Credit must be based on accuracy rather than completion. BLIS will take students about 50-70 minutes to complete. After students complete BLIS, our team will send you a list of students who completed BLIS, as well as their scores on BLIS, so that you can assign credit. 

  1. Why is BLIS required to be assigned for course credit based on accuracy/correctness?

Because BLIS is a content assessment rather than an attitudinal survey, being able to assess student correct or incorrect answers is essential. This means that students have to have buy-in for doing their best on the assessment. 

Contact

  1. Who should I contact if I have any questions or concerns?

Please contact Dr. April Kerby-Helm at Winona State University (akerby@winona.edu) if you have any questions about the project.