Monthly Archives: February 2011

Breaking up with best practices; Hooking up with learning goals

By IRIS Posted on by

Last weekend* I heard two sentences that sparked one of those great “ah hah!” moments. A writing center director said, “We’ve moved away from best practices and toward learning goals. This helps us prioritize and it helps us evaluate whether we’re accomplishing what we wanted to accomplish.”

I’ve talked before about how learning goals keep me focused and keep me from burning out on instruction, but it occurred to me in what felt like new says how the framework of learning goals could solve a lot of problems for me in ways that their less actionable cousins (like “best practices” or “standards” or even phrases like “user centered”) couldn’t.

Here’s what I mean in three examples:

  • In my own teaching, there are usually 15 or 20 Very Important things that I wish I could teach my students in any given session. Using learning goals helps me prioritize from among the very important things, feel less guilty about letting some very important things fall by the wayside, remember to think about what they’re learning rather than what I’m teaching, and feel connected to the broader, more interesting issues of information literacy.
  • In selecting a discovery tool, there are long, long lists of features and functions that user-centered design relies on. No interface has each specific feature, so how do we choose? How do we prioritize the list of very important features? What if we developed learning goals for our discovery system? What if these goals were something like being able to learn the differences between kinds of sources, be able to pick out important terms for the topic and field, and see where to go from here (different searches, different databases, different people). Maybe one system doesn’t have faceting but does have something else that reveals terms and directions. Maybe our usability tests could be more a long the lines of assessment of what the students learned by interacting with the system. Maybe this would all help us prioritize from the long list of important things to choose a system that functions in service of the mission of our library.
  • In first year seminars (the context in which the original phrase came up), focusing on programmatic learning goals could help prioritize from the long list of things it’d be nice if all first year students knew. Maybe it would help guard against creating impossibly long check lists of things students should be exposed to, and therefore guard against treating first year seminars as massive inoculations that transform high school students into college students. Maybe it would also grant the teaching faculty the freedom to explore interesting topics in interesting ways while having similar learning outcomes.

Or maybe I’m just creating my own buzz phrase. Or maybe everyone else already knew this.

But for me, at my institution, expanding this framework beyond my direct teaching or my department’s strategic planning is helping me make those hard decisions that crop up all over the place and to make them with more confidence.

* Last weekend I attended a workshop called Teaching and Maintaining Mulitdisciplinary First-Year Seminar Programs hosted at the gorgeous Pomona College campus. This is the second blog post drawing on my experiences there.

Uncovering Research Practices in Student Writing

By IRIS Posted on by

When I was a baby librarian, I thought Information Literacy was about searching and evaluating. The ACRL standards had some other stuff in there, but it seemed like abstract stuff that I couldn’t do much about. Keywords, operators, relevance, currency, authority — just learn the formula and my work here is done. No wonder librarians were the only people who cared about information literacy, I thought.

In my defense, I was young. In my defense, this is how it had been presented to me all the way up through library school.

In the past three years, I’ve been part of a project that really expanded my thinking and made me fall in love with what information literacy could be and with the ways in which it really is relevant to people and projects on my campus.

But let me back up.

All of our sophomores are required to submit a portfolio of their writing, and passing this assessment is a graduation requirement. When they submit their portfolios, they’re given the choice of designating that their writing can be used for research, which many of them do, and lately the college has been doing three large projects (that I know of) based on these writing portfolios.

  1. Our quantitative initiative (QUiRK) reads a subset of the papers to determine how sophomores use quantitative evidence in their writing.*
  2. The writing program and SERC are pairing up for the Tracer Project, which studies how faculty development (which includes writing portfolio assessment) impacts student learning.
  3. And starting in 2008, we in the library have been reading portfolios to see how information literacy is revealed in academic writing at the sophomore level.

As part of that last one, my department had fascinating hours of discussion about what we could and couldn’t evaluate about information literacy when presented with a finished paper. One of the most interesting and useful of these discussions (for me) was the one which revealed that we could, in fact, assess evaluation of sources even when the paper didn’t use “outside” sources beyond primary sources or sources prescribed by the professor. We could watch students picking primary sources, even from assigned readings, that worked well together and could be used to make a compelling point, or we could see them cramming two such sources together and either treating them entirely separately or in other ways not using them instrumentally toward making a point. We also confirmed what we had always suspected: that implementation of attribution was about more than just mechanics, and that failures in attribution could often signal a fundamental misunderstandings of the sources the student was using or of the purpose of reporting evidence in the first place. And we articulated for ourselves some of the ways in which integrating evidence into a paper can help or hinder the student’s rhetorical goals.

We couldn’t assess much (if anything) about the actual steps in the process that resulted in the writing we had in front of us, but we could look for habits of mind associated with using evidence, and we could look for the ways in which conventions of communicating evidence manifest in sophomore level student writing.

In the end, after much testing and revision, we came up with a rubric for assessing information literacy in writing and sat down to score papers. And yesterday, we finally presented our work and some preliminary findings, handed around a sample of student writing and watched as the faculty and staff attendees pulled interesting and useful insights out of the writing and then all came up with exactly the same score on the rubric (inter-reader reliability!), and had a fun discussion about how this could be used on campus to build shared expectations for information literacy and to help inform our teaching.

For my part, participating in this project has fundamentally changed one of the major ways I think about my work. It was so liberating for me to realize in concrete fashion that “information literacy” does not equal “the research paper.” All of a sudden I discovered that I do have something to contribute to those parts of the curriculum that interest me but that don’t produce traditional research projects. All of a sudden I realized that I don’t have to help faculty squeeze research projects into courses where those projects don’t fit naturally, and that instead we could talk about context-building skills or source interpretation skills for thought-pieces, class discussions, and other non-research assignments.

For me, this project helped me realize that I actually do like the concept of information literacy and that it actually does have meaningfully deep and cross-cutting applications on a liberal arts college campus — that it’s not simply about making mini-librarians out of our students or about searching for searching’s sake. I’m hoping that as we open it up to include faculty readers this year, that same sense seeps through the campus. I hope this is something we can get behind and dig into and find interesting, and that what we learn from analyzing these portfolios will meaningfully inform our practice as teachers.

I’m just so excited about this project, and so glad to be involved in it. It’s probably been the most eye-opening and practice-changing project I’ve participated in.

* Rutz, Carol and Nathan D. Grawe. “Pairing WAC and Quantitative Reasoning through Portfolio Assessment and Faculty Development,” Across the Disciplines, December 2009; Grawe, Nathan D., Neil S. Lutsky, and Christopher J. Tassava. “ A Rubric for Assessing Quantitative Reasoning in Written Arguments,” Numeracy, January 2010.
[back to text]

First Year Seminars: Multi-disciplinarity vs Un-disciplinarity

By IRIS Posted on by

Last weekend I attended a workshop called Teaching and Maintaining Mulitdisciplinary First-Year Seminar Programs hosted at the gorgeous Pomona College campus. I expect this is the first of a couple of blog posts drawing on my experiences there.

It seemed like a lot of the impetus for having multi-discipinary seminars had at least as much to do with a skepticism about the transferable skills within a disciplinary seminar as it did with positive benefits of multi-disciplinary work. (And I think there are a lot of benefits to multi-disciplinary work in these circumstances.) Institutions have a lot at stake with these seminars, helping students make the transition from high school to college and getting them up to speed with college-level writing skills and college-level critical thinking skills and college-level classroom discussion skills. But I think sometimes those goals seem so huge that we end up with the classic problem of not narrowing our research topics. Some parameters usually help us think deeply and carefully — narrow topics are often richer than broad ones.

I kept wanting to protest that fleeing the disciplines in order to isolate these skills from any particular discipline not only made the skills seem that much more insurmountable but also mis-characterized “multi-disciplinary” as “non-disciplinary.” There’s no such thing as “non-disciplinary.” Not only is multi-disciplinary work a discipline unto itself, but there’s just no such thing as a work with all the skills of the disciplines but none of the topical constraints. A college essay is just as much a learned genre as a lab report or a close reading, so teaching students “to write” first, as if it’s a free-floating skill outside of the disciplines, seems like a fallacy to me. All we’re doing is teaching the genre of writing we’ve most internalized as “basic,” first. And that may be a valid approach, but that doesn’t make it generic.

Divorcing skills from content always feels easier to teach, but actually renders each hollow, I think. I remember the language faculty at a recent conference bemoaning the fact that they have interesting topics to think about and teach, but that their students need so much grammar training first. And I remember feeling like this was exactly my problem, too, because I like to teach about epistemology and scholarly communication and critical thinking, but sometimes this means teaching boolean logic and LCSH first.

And then I think about the classes I’ve taught with professors who were willing and eager to interact with the research process in front of their students and build on the day’s discussions or readings in the process, and how marrying content and skills always felt so much richer and more real as a consequence.

And I wonder if fleeing disciplinarity, as such, is really the best way to teach these skills that everyone hopes all first year students will learn. Maybe, it’s equally valid to think about the skills, the learning goals, and build them into classes regardless of disciplinary or interdisciplinary focus.

Or maybe the value of fleeing disciplinarity is that the professors feel like the work is denaturalized and are therefore better able to articulate some of these principles that are otherwise invisible from long acquaintance.