top of page

Quantitative Study 2: Subsequent Course

Data was pulled using .sql query from an institutional database to include: the deidentified student, course name, term, letter grade, and GPA for both the "first" course, that is,  the targeted three courses listed in the above chart, and the "second", or subsequent, course. A "third" course was also populated in the case that a student skipped the immediate subsequent term entirely but re-enrolled in coursework the following term. 

​

   Criteria used to populate raw data for analysis:

 

  • Courses into which students enrolled the immediate next ("second" term)

  • If a student skipped a second term, their subsequent course was identified in the following term ("third" term) 

  • Any "fourth" term data was not identified or considered

  • If a student took multiple subsequent summer courses and withdrew from all of them, and then enrolled in a third subsequent course, the third subsequent was considered as the subsequent course, since it appeared the student withdrew based on something other than the courses being too difficult

  • Required courses that immediately followed at the same or next level (not a lower level course)

  • Subsequent courses had to have ten or more students entering so as to avoid the few students who may have taken a course in an isolated part of college and, therefore, may not have been representative of a trend

  • Excluded courses were: Courses excluded were AGB-101, AGB-331, BUS-161, GLS-110, and SDV-109

  • If students dropped before the refund date (first two weeks of class), they were not included; otherwise, data includes all students past that point

​

When the raw data was populated, it was then processed to eliminate students who had been issued a final grade of W in their "first" course (not the subsequent), as this may have invalidated the data set, since it was not known why or when exactly students stopped attending a course (i.e. early in the semester or right before drop date); Ws ultimately meant they did not finish and were not included in the analysis. 

​

Then, any duplication in the first-subsequent courses for any one student was erased, since some were duplicated if there was a third course. Also, any first-subsequent courses that had no inherent or logical connection referencing language or skill domain was deleted (i.e. it is irrelevant how students in the reading course  did in a subsequent speaking course).  This reduced the data set from approximately 2,000 rows to 550. 

​

Next, all possible "pathways" were identified (see "Pathways" tab in chart, below), labeled using a naming convention, and sorted within the Excel document using a nested if-clause code added to a new column. This allowed specific individual pathways to be displayed for analysis. For example, when a particular pathway is displayed individually, we can more clearly say: I see X number of students who took X course using the X grading structure followed by X course, and here’s what happened. This "pathway" sorting also allowed number grades to be averaged and compared, as well as the compilation of a secondary measure of counting of how many A, A-, B+, B, etc. there were, since numeric averages may not account for range (i.e. all students got a B and some who got all As and Cs still may have the same average of B).

 

The data was then interpreted and reported on the document's "Overall Results" and "Pathways Results" tabs, below.

​

Results Tab Key

  • T = Traditional grading structure (points and weighted categories)

  • R = Competency-based grading structure, revised version (students must demonstrate competency all outcomes before passing the class)

  • Curr= Current course

  • Sub = Subsequent course

  • Cond= Condensed

  • Sts = Students

​

Raw Data Tab Key

  • Code = Deidentified student; this column has redundant number values that represent the same students but with multiple data points

  • First Term = Semester and year of the original language course that used either a competency-based or traditional grading structure

  • First Course = The original language course that used either a competency-based or traditional grading structure

  • TRAD/REV = Traditional grading of points or weighted categories (TRAD); competency-based grading, revised version (REV)

  • Second Term = Semester and year of the subsequent course

  • Second Course = A student's subsequent course from the "first course", whether they enrolled in the immediate subsequent term or skipped it and enrolled in a third term 

Results

Although there are many variables that were not or could not be controlled for, the compiled data suggests the following:

 

A student who took a course using a competency-based grading structure and then completed a subsequent course may...

  • Have an equal or slightly higher probability of being successful with a C- or higher in that subsequent course (80.8% average compared to 80.0%)

  • Have an equal or slightly higher probability of being successful with a D- or higher in that subsequent course (84.53% average compared to 85.14%)

  • Have a lower probability of receiving an A, but a higher probability of receiving a B or C grade

  • Have a slightly lower probability of an average GPA (2.69 compared to 2.91)

  • Have a lower number of students who drop the class (8.7% average compared to 10.57%)

​

Conclusions

The results did not seem to conclusively support nor reject the potential advantages of  competency-based grading. There were multiple variables that this study did not control for (i.e. how students' language exposure changed during any time off, how the grading affected students' perception of education, the natural transition that English language learners experience when entering into required non-ESL college courses, or instructional variation that may have occurred). It may be argued that because the standards within the course are essentially tightening and that less students are passing in a competency-based-based approach to grading, that the model may be negatively affecting student learning.  However, it is hard to deny a critical aspect of competency-based grading: that students left the course having documented, demonstrated abilities in the areas of proficiency, course content, and participation, which translates to between 91.5-99.9% mastery of a course's student learning outcomes (see Quantitative Study 1). Additionally, lower pass rates may be a result of the fact that, in this study, I was the only teacher using this approach to grading; the  perception of this "extremely unique" approach may have influenced a student's ability or performance or decision to drop the course or not, which may have impacted the pass rates. In fact, if such an approach could become the status quo in a program, it is plausible to conclude that future studies could suggest equal to or higher success rates. When coupled with the resulting data from this study,  Quantitative Study 1, and the feedback provided within the student surveys, it is perceivable that competency-based grading may be preferable compared to a traditional grading structure.

 

In any case, this study, considered preliminary, yielded interesting results by allowing insight into student performance both within individual pathways and overall. Using this study as a launchpad, further studies are recommended. 

​

Purpose

This study compared how students, who took the same course, performed under a traditional grading structure versus a competency-based grading structure in their subsequent course(s). Specifically, the study aimed to answer the following questions: What were the success rates? Into which classes did students taking these courses commonly go upon passing? What were the drop rates? If they passed, what percentage of students were successful in their subsequent class(es)? It also, similar to in Quantitative Study 1, measured how student performance within the actual course compared across the two grading structures. 

​

In the end, the purpose of the study was to determine which grading system may most likely positively impact student learning. 

​

Process

The basis of this study was three courses taught between 2007 and 2017 by the same instructor with the targeted controlling variable being the grading structure used: either competency-based grading or traditional grading of total points earned over total points possible or weighted categories. These courses were: a high intermediate level presentation/speaking course (ESI-065 and ESI-069), a high intermediate level reading course (ESI-068), and a low advanced level conversation/speaking course (ESI-090). See chart, below. 

Interested in professional development credit? Click here to learn how to get a certificate of completion.

Copyright © 2018 BY PROFICIENCYBUILDER

© Copyright
bottom of page