Gaining a Foundational Understanding of College Testing [Podcast]

This is a transcription of a recent podcast episode where Former Dean of Admission and Strategic Advisor to Scoir, Peter Van Buskirk, is joined by Rick Clark of Georgia Tech, to discuss standardized testing as it relates to college admission. 

Listen to the podcast "Gaining a Foundational Understanding of College Testing with Rick Clark" here

Introduction & laying the foundation – 0:00

PETER: One of the interesting things about this time of year is that while you're watching carefully to see who among the admitted students will enroll you're dealing with another round of soon to be seniors soon to be college applicants who are trying to get their ducks in order in order to be ready to apply and one of the big issues for a lot of these folks is testing no the last year or so has been a real tough time for anybody wanted to do testing because testing centers have been closed in many parts of the country so that's led to a lot of changes in the way testing as manifest now in the admission process and we'll get to that in a moment but I’m wondering if you could kind of help us establish some foundational understanding of what testing is all about when a selective institution or even non-selective institutions looking at candidates why bother telling a student you need to take an SAT or an Act. What's in the test?

RICK: I think it does vary obviously from school to school and that's our job right is to try to figure out what are those things in an application or about a student's background that are predictive in helping feel confident that students going to do well on your campus. So, certainly, you know, I think there are plenty of colleges and universities out there that can make a pretty good case that they've done their studies and look at the analysis, where they can see some correlation between performance on these tests and ultimately performance on that specific campus. Obviously, there are many who even pre-pandemic, had determined that wasn't true, you know, that it wasn't correlative or it was certainly not absolutely necessary – that there were enough other factors they did pull in to give them that same confidence.

Evolution of testing - 2:40

PETER: Let's take a look at the evolution, if you will, of testing. We don't need to get into the weeds on all of that evolution, but of the tests, the SAT has the longest history and its history is more rooted in a sort of logic and reasoning format. The ACT came along now about 60 years ago. Different kinds of tests, but can you help us understand what the differences are – the fundamental differences are between the two tests?

RICK: It was easier you know pre–New SAT, or I can't remember you know what the marketing term for that was exactly, but you're right. I mean, there was a dichotomy at some point and what we would coach students on is: listen, if you're more content-driven, then the ACT could be a better path for you to go and you know with the science section of the ACT you know there's going to be some other components that could pull in and highlight some of your strengths, versus ¬– and again generalizing – kind of that critical reasoning, a little bit more, let's say logic or deductive reasoning of the SAT…less about necessarily what you've been exposed to an more about some of the skills of deducting to an answer, eliminating bad answers, you know that kind.

I do think though that you know interestingly the year after the ACT had more Americans take it then the SAT the SAT quickly pivoted, and those tests look a lot more similar than they ever had before.

Taking tests - which one is for you? - 4:37

PETER: Now we're talking to, presumably people who are finishing the junior year, maybe some finishing the sophomore, year. They know testing is on the horizon. What is your advice to that young person when they haven't taken the test yet, they haven't taken the PSAT, you know, they're getting some pressure to do something, to practice? What from your experience seems to be a good approach for young people?

RICK: Yeah, it's a great question. I mean, and I do sympathize with how it's gotten confusing, even more, confusing maybe, than it had been in the past right…because the test is something we've said constantly is important – it's one of many factors is certainly how we talked about it. But hey it's always a factor, and now all of a sudden, I can hear, if I'm a 16 or 17-year-old, you know optional means optional…what is that, you know? what does that really mean? is that really true? I think the big thing that I've, you know, as we've talked to students you know the message that we've been given is...listen, you know, it does vary from one school to the next.

PETER: One of the things that I’ve heard as advice presented about testing at the outset is: try one of each the SAT and ACT and see which test suits you best. Most colleges will take either or both, and then focus on one of them not necessarily taking both. Is that squaring with your sense of the process as well?

RICK: Yes, 100%

PETER: Prior to the COVID experience that we've all been part of for the last 14 months, there were just more than 1,000 colleges and universities that publicly acknowledged that they could make good decisions about whom to admit without test results, and that number jumps by about 60% over the last 12 months or so, and I was interested to hear you say that you're not convinced that many of those schools are going to go back. Many of them framed their move as an experiment and I guess the experiment is ongoing now, but you also suggested we’re at an inflection point where the way colleges think about credentialing students in this process and what's really important is taking a lot of different shapes. There are a lot of different types of intelligence that are being measured in some way or another these days too, so testing is there you gotta do it, but it may look different in the not-so-distant future.

RICK: I mean you know especially too, there's some big influencers in this ecosystem. I mean you look at the California system as an example and not only do you have obviously 10s of thousands of kids a year who – for their system – are not going to be required to take the test at least for the foreseeable future and possibly never again, you know, other universities – because California “exports” a lot of students – are going to have to respond to that and there is going to be I think a trickle-down effect as a result.

Score-choice and test-optional - 7:40

PETER: So, now we've got kids saying about the test: "can I afford to take it or can afford not to take it?" Maybe you can help us sort this out a little bit because you know over the last ten years or so we've seen the advent of something called score-choice which means that students need to be reminded that when they've taken a test of any type, they own the result, and that result shouldn't go to any entity without their authorization. Can you allow your credentials to be mined, if you will, by colleges for recruiting purposes, yet withhold them for admission purposes?

RICK: Yes, and I think that's one of the important messages that's really not out there much at all and certainly not out there to students who are less resourced or less you know kind of well-counseled. And I honestly don't know how to convey with you know the gravity of you know authenticity that I think that it's going to take to convince people that you know yes the technology systems that we use or in fact smart enough to allow you to be message to recruited communicated with up until a point and then when your application is submitted an you check the box saying no I don't want you to look at my score it is absolutely possible to predict that not look at that and and never have it into the conversation so that is something that we're actually looking at putting into our presentations to demonstrate to families how you know these systems while they may kind of fall into the same umbrella the what an admission counselor sees in the review of an application is different than maybe how and where information is held to message a student as a prospect right totally different and so that is i think that's a really important point that i'm really hopeful we can kind of proliferate as the as a message to families and convince them that it's genuine

PETER: And your status at Georgia Tech with Test Option. You've gone that route?

BERT: No, not yet, and here’s another important message I think, and something that pandemic has shown a light on, at least to an extent, is local control versus system control. You know, if you're a private school, whether you're a private high school or a private college, you get to set your tuition, you get to make your policy you get to kind of be who you want to be – that's local control, that's on campus control. If you are part of a public system then – like, today, for instance, our Board of Regents set tuition for the entire system for next year – we did not have a say in that. So, they set our tuition. Now, in this case, they left it where it was, thankfully, so there's no increase in tuition next year, but that is made at the system level.

Similarly, with test-optional as just one more example that is going to be the case so we're in wait-and-see mode. We can convey to them the data, we can help them understand, but ultimately, you know, the decision will rest there – and that is where, I would say, the anomaly is going to occur. I do think we're in an inflection point but I think you're going to see some of these southern systems who are often tied together in their lotteries, or in their merit-based awarding of financial aid, very tide to testing and not pivoting away from that.

PETER: Well, it's an interesting point you make well it's an interesting point you make there at the end that even at schools that might be test-optional in the admission process if they have a merit scholarship program of any sort, they'll still want to sneak the test and they will take a look why do you think that is

RICK: I think that there is just a belief you know that that testing is 100% objective standardized and uniform and it is the one and the only level versus you know one school versus another have different grading scales different weighting different quality of teachers you know all the different nuances that occur right from school to school grade inflation and all the other things that we're going to well aware of versus what many would perceive to be the great leveler of testing. Not my personal opinion but I do think that that is the common answer that you hear from the sort of the powers that be you're often adhering to that type of merit-based awarding

How taking a test might assist with merit-based scholarships - 12:20

PETER: So juniors if you're trying to figure out your test strategy right now and you're hearing that a lot of schools are test-optional and you're thinking, well I don't have to take a test for them…then you're hearing that the system-based schools will require them and you want to apply to assuming flagship state universities…OK got to take it there, and you're also maybe looking at exclusively a private school list, but then some of them have a merit program that requires the testing….it probably does make sense then, from the outset, to at least cover yourself…and correct me if I’m wrong, cover yourself with some testing at the outset here just in case you need it.

RICK: I would agree with that. As you as you pointed out once you start to apply you know then it's going to then and this is again where yes your list could include a variety and right an and you may choose to have your tests included at one place even if they are optional based on what you're seeing in their profile right I mean and I think the conventional wisdom has been look if you're if you're looking at a middle 50% range and you're at middle 50 or you know about 50% range for higher send it I'm not I think that that's at least what you're hearing oftentimes said so if you have a good balanced list that has a range of selectivity even if six of your schools are test optional it could be that you're sending it to three 'cause you want that looked at and not sending it to three I mean I've been using the metaphor of a stool where it's like you know look if it's optional then I've got five pegs on this stool you know I've got my GPA I've got my letters recommendation I've got my extracurricular stuff I've got testing you know do I want weight put on that leg or not and in some cases I may and in other cases you know I might not I want to have more weight put on those other factors

Super scoring - 14:15

PETER: What is a super score? And this is something that has again evolved over the last 10 or so years. I think that the premise of a super score has been out there but now we have a name for it. Can you help folks understand what the super score will be?

RICK: So super scoring is the concept of giving the benefit of the doubt, right. That instead of testing is like a football game like one Saturday 4 hours and that's it…we're trying to look over maybe “the season” right where you took the test three times, we're going to pull in your highest possible combined scores, so maybe that – let's just stick for now with the SAT: critical reading on one and then (you know that’s EDRW I guess, technically), and then Math, you know, in September right so I got my May EDRW somehow that goes down in September, but my Math is higher. Well you know I'm pulling those two together. And here's another important message to families: if a school says they super score, they do. You know, I really wish that we could give them some kind of Jedi mind signal here to indicate we're being honest about that, but you know look…our systems, we write the algorithms behind this, it pulls in the ones that we wanted to pull in, it leaves out the ones that we don't. This is why it's so easy to redact scores entirely or to write code so that the math comes in here if it’s higher, and the critical reading comes in there if it's higher, puts it together and then there's your super score.

PETER: So the student who may have taken the SAT 3 times and maybe that I'm going to go with the old school verbal 'cause I the EDRW tongue twisted on that but let's suppose the verbal score was best the first time out, and the math score was best the last time out. The student sends all three to the institution and the institution then presents the reader of the application with only the best score, not the three sets. Is that correct?

RICK: That’s 100% correct and I think that's really important because, you know, even in talking to friends right in my neighborhood – not kids were recruiting and not students in information sessions – just you know, regular people, they doubt that that's true. They really expressed some doubt and what I try to convince them is, think about the thousands and thousands of applicants, right, to these to these schools, and now if each kid took the test two or three times…now that's just order of magnitude we do not have time to be looking through all that. If we tell you we're super scoring, that's exactly what we're doing and you know we're not sitting there trying to hold it against you that you know the one math was a little bit lower or something like that, it's just not the way that it works.

PETER: Now when the student submits an application and I again realized many applications are different, but is it possible for the student to do the super scoring when they actually complete the application and send it in, or do they need to supply all data and then the institution creates super score or is it going to vary?

RICK: It can vary. One thing that Georgia Tech, you know went to, as just one example we're not alone in this a couple years ago was – and partly this has to do with finances and trying to keep students from having to pay to send additional scores – is we said look you can self-report these right and so we provide that opportunity for a student to self-report their testing. That means that they can both say every time they took it and their individual scores, as well as they themselves have a section where they can super score, so they can do it for us, even though obviously we're fully capable of that ourselves but it allows for that.

PETER: Now they will they need to submit, upon acceptance for enrollment, will they need to submit official scores to verify discrepancies between what’s self-reported and the official scores?

RICK: We’ve done this for several years right so we've proven that the students are very accurate in their reporting. We would go away from it honestly if it weren't for our system, but our system requires that actual official information to be submitted to them, and so you know that's the only reason we would say if a kid is ultimately admitted chooses to come then they need to send the official scores.

When to take your tests - 18:35

PETER: What's your experience with the timing of tests? Let's suppose a student and let's suppose a student…I’ll throw hypothetical out here…student takes the PSAT in October, that's junior year, gets a result back in December and immediately signs up for the SAT in January, gets a result that's not quite what he wants…takes it again in March, still not quite…then again in May. Is that a good sequence, do you think?

RICK: You know…I mean obviously there's a lot of data and research out there about, you know, sort of a…diminishing returns right after three takes…varies a little bit, but typically that's the kind of threshold right, is after the third, if there's not been anything different done, right if the preparation, or the you know content received is no different, then there's a diminishing return typically on that…So I would say that taking the test 3 times over a period of, call it a year to a year and some change, that seems like a reasonable cadence to me…However I would say that it's after that second time, you know if they're not seeing, as you said, sort of the results that they wanted to, I would say it's probably time to take some kind of like action to seek change versus just more time in class or whatever else they're going to naturally be learning outside of intentional preparation

PETER: It occurs to me though that one of the variants that can contribute to improvement in testing is time. These tests were really created for kids who reached the start of the junior year in high school and they're continuing to have exposure to things in the classroom and out of the classroom that grow their knowledge base and their ability to reason information, so the more time they allow themselves in those intervals between tests they can perhaps improve their scores as well. I'd like to kind of shift to your former experiences, as well, in places like Cornell and Bucknell and Muhlenberg where sure you're looking at candidates in a slightly different manner, but back to the course selections…when you have a student who says: “I am taking all the language that my school has to offer. Can I just take a study hall, can I get an early release from school, can I take the underwater basket weaving that Bert talked about?” or what's going to serve that student best when the student’s applying to a very selective institution – taking the road of lesser challenge or finding a new challenge?

RICK: I would agree with that, especially as it relates to the math. Especially I think as it relates to the math right I mean in content received so you know I think that there again you know if within a year to slightly over a year you know you're looking at three different takes that again that seems like a very reasonable cadence to me versus that you know for instance like three months in a row or something without any type of quote corrective or different interaction at the same time I mean I I do think if as you say a couple months past he's taken at the second time and you're not pleased or not seeing the results you want I think it's at that point that you should at least assess you know is there other kind of outside preparation that you need to do in addition to maybe what you're being exposed to in the classroom to get those results that you're looking for

PETER: You've mentioned a couple of times, outside preparation. Years ago, The College Board in particular, argued vehemently that there wasn't any kind of outside preparation that would change the results of tests. I think now the college board even agreed that the DTS has agreed that that could be the case. Is it a reasonable thing to expect that students who have access to test prep of some types participate in it? That it's really not sort of corrupting the system?

RICK: Well, I think we've seen enough evidence that test preparation boosts test scores and students have the resources to do that are getting higher scores than they would have gotten otherwise. It doesn't mean that there's a student who never had any test preparation who couldn't make an equivalent score, but that individual students with test preparation are likely to see increased results, and so in that regard it's not the great leveler that again as we were just talking about earlier It's not completely standard 'cause there are outside influencers into the into the testing

New testing developments - 23:17

PETER: I think it’s also established, isn't it, that the student might expect to see a certain increment of improvement from the 1st to the 2nd test even without test prep just so be patient with yourself i guess is an observation.

Before we wrap up, I’d like to take a look at the new developments we talked about, things evolving but The College Board made some announcements in the last several months about changes with regard to the essay, with regard to subject tests, last year the ACT had made some announcements with regard to how it was going to administer testing. Can you in summary form give us a sense of what's going on?

RICK: The essay going away is not a huge shock. I mean obviously, you know, there's resources on The College Board side that are, I guess, not sustainable…and then on the college side, I mean, there wasn't demand and so it just didn't make any sense to keep that. I think obviously subject tests would be the other kind of big piece here…again sunsetting subject tests and what that really points to, in my opinion, is you know The College Board effectively doubling down on APs and I think that's going to be really interesting to watch here in the years ahead.

PETER: Any thoughts around ACT developments where the students taking subsequent tests can just sit for particular element of the ACT rather than the whole thing?

RICK: Well I mean I guess it goes back to what you said earlier about just this just this concept of I think your word was like either spoiling or corrupting you know the pool right I mean not only do you have students who are either financially or timewise able to take it multiple times you have students that are able financially and otherwise to pay for test prep and now you have students that are able time wise and financially to focus just on one section so this again is where I think schools have to be cognizant of how they're using these tests and you know how they're going to evaluate their applicant pool in an equitable way knowing that these type of strategies or options are available to students

PETER: It is indeed a complicated word is it that that at the outset and I'm delighted that we could have this conversation because I I could sense with each topic we approached you're able to kind of tease out greater clarity for those who are trying to figure out what to do or how to make sense of testing in their preparation for the college application process so thank you so much for taking some time away from rather chaotic spring I suspected to have this conversation with this then folks were listening in I’m sure you're going to find some real value in reflections on the testing environment we see right now so thank you again be well everybody rick i hope we can get you back sometime: