A New Strategy for Open-Ended Long Term Creative Projects with A La Cart Rubrics

Hello, thanks for coming to my talk.

Hi, I’m Hunter Dyar.

I have never given a quiz or a test before in my life! I teach almost exclusively project-based courses.

I cut my teeth working at summer camps, teaching Photography and Photoshop and Minecraft to students aged 7-17. So I have a lot of experience with project-based learning and keeping an active classroom.

I’ve taught 8 summers of tech camp, been an adjunct since 2015 (I think) for about 2 workshops a semester at CMU, and been full-time teaching at Chatham since Fall 2019.

In my decade+ of experience, almost everything I have ever taught has been built around projects.

I work in the Immersive Media Department at Chatham University.

Founded in 2019, we are young! We were modeled after the Interior Architecture program. There’s a heavy focus on project-based studio courses.

We are focused on VR/AR design and development.

We’re the cool intersection-of-art-and-technology kids who project cool art onto buildings.

Although, because we have such a focus on VR/AR technologies, sometimes our showcases just look like this.

This photo just makes me laugh every time I see it. I handed a student worker a camera and wished them luck. Showcase day, we’re all standing around while guests with headsets strapped to their faces try our experiences.

This is what we do. Making Immersive Media is a very interdisciplinary, very broad subject. Doing one project requires learning and executing at 50 different things and 50 different layers of abstraction.

It’s a great fit for project-based courses.

Now, we have these massive studio classes. Doubled-up credits, twice a week, 2.5 hours. They’re great, we can dig into the weeds and do all the work!

Spoilers, one of my solves for assessment without tests is in class work time. Watching all the students work. It’s hard to sacrifice class time for this… so we schedule long classes.

Well, while great for us; terrible for students who want to take, say, any other course in any other department. So to be less of a scheduling nuisance, we were asked if we could split the courses up.

So, if I had to anyway, I had a chance to sit down and design my perfect project-focused course. I made a list of all the things that are frustrating about project courses, and tried to tackle as many as I could.

This talk is the result of that effort. Welcome!

Project courses are hard! Really hard!

It’s hard to deal with experienced students skating by and not learning much, while other students struggle but learn more?

It’s hard to get the students to pay attention to all their learning objectives, and not just focusing on the next biggest challenge in front of them?

It’s hard to prevent a snowball from building for the students when they get behind.

It’s hard, and time-consuming, to grade! An A and an F might be clear, but fairly deciding between B and B- for disparate projects! Oof.

It’s hard to direct students to branch out into unknowns.

It’s hard on students when there are fewer grades in the course! I’ve had classes where the grades are all, basically, just 2 large projects. Whiff on one of those, and it’s hard to get caught up.

I’m done. Just kidding, no I’m not.

Students sometimes whiff it! Or they get stuck on some random technical software update for a week and, for no good reason, are behind and struggling. That’s the reality of what we’re learning, but it still sucks!

Bottom right example is a student fighting against Tobii eye-tracking terms of service.

Top right is a student who spent a week on research, and didn’t get far. They felt bad! Had a hard time!

Finally, I love kicking my students into the deep end. All the IMM majors, they’ve learned how to drown effectively. Overwhelmed with infinite technology, infinite cool projects, infinite scope. I put em through the ringer so they come out capable of the work. But I get a non-major who hasn’t had the trial-by-fire of our project-heavy program, and they are suddenly extremely low-confidence.

No matter what their experience level is, it takes work to pull students who are used to being told all the checklists and get them used to this different course structure.

Project based courses have a lot of down-sides. So let’s get to how I tackled them.

Banter with audience about pain points they have felt in their classrooms

Here’s the background context on the new course design, which I’m going to call “Production”.

You can’t steal this stuff, because it’s probably out of your control.

I split our new class up into sections, with a major project for each section. It’s basically determined by the calendar. 2-5 weeks per section.

The content cadence is weekly because we meet once a week. When I say “week”, feel free to substitute whatever makes sense.

I also have all of the students provide their own learning objectives. We provide some from a co-requisite theory course, and from the course; but the students all should have a clear goal of what extra directions they want to go on.

For example, when I teach game design, some students want to get more into programming, or level design, or 3D modeling. This course has the flexibility to do that while still hitting all the important objectives.

I make the students right it down. This is important for later.

But I’m getting ahead of myself.

The most important question of a project course.

What are the projects?

No Clue!

We pitch them during the class. I set constraints that have to do with learning objectives, like using a new technology or engaging in design research, but they otherwise have total freedom!

Well, not total freedom. I’ll come back to this.

An important part of my personal teaching philosophy is that students are smart, and their ideas are great. They’re cooler and more awesome than any of my ideas anyway.

I am not here to just assert ideas into their skulls (although I also do that). I think of teaching as plucking obstacles out from in front of them.

Alright, finally. I managed to get to the point.

So the students have project ideas. They’ve got a section to try to make some version of this idea. What do they do?

Each week is split up. For me, that’s one assignment per week. There is no cumulative or large project grade. This helps a lot with a single whiffed project destroying a students grade.

Each student then chooses what assignment they are going to do. I have a buffet of rubrics that they pick from! Every project, and every student, is different, so they choose the right tool for the job.

On groups,

So we got a lot of rubrics to pick from!

It’s decomposition and isolation.

The right is a students mood-board for a VR heist game that took a thousand pivots

Each of these are “a part of a project”, although some of them are “other, see instructor”.

The final showcase project is a prototype, but it must be demonstrated in person during a showcase. They don’t get to pick that one, that’s what they all do for the last week, which we call “install week”.

Ever given a student great feedback… and then that was it? They had a cool idea, and there are some cool next steps, but it’s time to move on? I hate that! So almost all of our projects can be continuations of previous projects. Since the rubrics are process not product, there is always something to evalutate – students aren’t ahead or behind no matter where they are on the project.

The “iterative pass” is a rubric for, specifically, taking noted feedback and without adding new features, just hacking away on it.s

Six Sketches is another good one. 2 paper sketches, 3 sketches in the final engine/tool, and 1 sketch made with real-life things like clay or legos. Plus, only 1 sketch can revise a previous one.

Now, you can steal my rubrics. But there’s no need. They’re all basically the same, in this abstract way.

Each rubric is a student decomposing their project to identify a piece of it, identify a plan to tackle that piece, executing, and documenting.

Sometimes a student isn’t sure which rubric they picked (was it a one-week-prototype or a-technical-proof-of-concept?). The secret is that it doesn’t really matter, so long as they were able to complete these steps.

The rubrics are guiderails. They force you to go slow, have intentionality with your work (“a plan”), and document what you’ve done.

Lastly, it’s not fair to just ask students for ideas.

We have to spend time talking about interesting projects, why they are interesting. Show them cool things, take them to field trips to museums, so on. You don’t get great ideas for free, but by building up a collaborative studio atmosphere.

I mentioned the students don’t actually get much freedom. Project decisions are a collaborative process, aided by in-class brainstorming exercises. In those sessions, it’s pretty easy to say “Cool Idea, but not enough x. What about y?”, and they’ll start thinking.

The actual classes?

Well, we all know how long writing up feedback that students never read takes. So, I don’t! Instead, that’s most of the classes. Discussions about what the student should do next and why. What went well, what’s missing, etc.

I’ll talk more about feedback on the next slide.

A form in the LMS lets students submit suggestions for topics, and I observe things they are stuck on.

I’ve given mini-lessons on project organization, on 3D modeling workflows, on game AI, on how the internet works, on HTML/CSS basics, and more. I would never design a class like this, but the lessons come from the students; and it’s what they need. I also bring in guests to help out, covering other topics. Us Professors are still basically the best resource the students have, so I make sure to enable them to take full advantage of this resource.

Finally, work time. I have to mention it because it’s one of the assessment strategies. No better way to quickly grok a student’s mental model then watch them work.

Feedback is the heart of the class sessions.

I’m also teaching students how to give solid and effective feedback. This is an important objective for us, and really propels the students as designers.

That enables the hack for large class sizes, or days when everyone did a prototype rubric. Split them into related groups, and have them give each other project feedback. Take quicker laps in order to evaluate everyone, without the entire burden of feedback.

Outside of feedback, the other part of class is the project selection time, that I call brainstorming sessions, and mini-lectures!

So, here’s my Grading Scale.

All the way from A to F. Two options! Satisfactory and Not Satisfactory. That’s it!

It’s good enough, or it isn’t.

When it isn’t good enough, they just take it back and fix it up. In my experience, most NS’s can be turned into S’s with under an hour of work. Frankly, it’s usually just that they forgot some documentation when trying to get the project presentable.

There’s a secret third grade item, incomplete, but that’s only in the grade-book and only for me to know which students to chase down later. It’s all NS.

At the 200 level, S is C- or better. 300 level, it’s B- or better. 400 level, A- or better.

When I give an NS, I always list concrete and actionable reasons that it isn’t good enough. This is easy because of the rubrics, it would be hard when talking about the project wholesale.

Their percentage of S’s, plus section reflection papers and a participation grade, get turned into their final grade. I end up giving out lower grades than I normally would, and the students were happy with them. Grades shouldn’t be surprises, and they aren’t.

This was the same experience as my old colleague, Professor North Cook, had when he first used the S/NS system for a Porfolio course. I took it from him.

I’ve never had a major conflict between a students desire and their reality. This, I feel, is because I never play antagonist or assigner. I don’t, grammatically, give them something to conflict against.

Instead, we just have a discussion. Remember those personal learning objectives the students wrote down?

Those are my ammunition throughout the rest of the class. Talk about their project… and to redirect a student, I’ll just casually mention their learning objectives and a hypothetical way they might get there… and they tend to agree!

If you are going to steal one thing, it’s asking students this question: “What does success look like?”

That’s it. That’s the secret to the course. Have a student describe specifics of what their goals are, then give them the resources to get there.

Resubmissions are another huge problem with project grades.

The problem is we can’t let students get too behind (and learn good habits), balanced against giving them opportunities to fail, learn, and recover.

My personal opinion is that I never want a situation where it wouldn’t be better for a student to do the work.

I don’t have any late penalties, which I find feel arbitrary and punitive. I also don’t have a deadline for late submissions. Instead, I simply limit the rate at which students can resubmit.

Only one resubmission per week. Two during midterms and finals. This has the same effect at forcing the

This keeps me from having to grade too much during finals, and doesn’t let students build up a backlog of work for future them to handle. It’s also simple to implement from a logistics perspective.

Rate-limited resubmissions! Give it a try.

I also need to talk about groupwork. Groupwork is a traditional massive pain for project courses. Because, they become courses in mangaging interpersonal conflict.

Not so much with this strategy. Grades are separated, but project feedback is singular. Each student has their own rubric. So the group identity and shared-evaluation anxiety is lessened without changing the fact that they have to do group work.

I’m also spoiled. I can combine cohorts. The 200,300, and 400 level classes are in the same room at the same time. I have second-years and graduating seniors working together. Let the leaders lead by example.

One of my favorite rubrics is “Asset Creation for Others”, where one student works on another project for just one week. That one models a client relationship with an initial meeting and a feedback round meeting, with required notes from them.

I also had a group that wanted to go in 2 different directions. One students wanted horror-comedy VR, the other wanted horror-horror. They split up. That was just… allowed? If the learning objectives were for learning group-work, like it is at the 200 level, I would have intervened. But for the Seniors, they were fine. Yeah, split up. Sure! Now I get to compare similar projects and pull out design lessons!

That concludes my talk!

I have a number of examples, but let’s jump to questions and I’ll weave the examples in if it’s appropriate.

• Student’s constraint was “design for a single person”
• They chose ‘Benjamin Neyinyahou’
• It’s not a project I am the best person to advise!
• The solution is in the structure of the course - the group did research. One did research on games for change, referencing Brenda Romero’s ‘Train’ and others, for example.
• They first had to be able to answer the question “How is this kind of thing done well” before they started answering “how do to this kind of thing”

What I can do is facilitate resources. In this case, connections to other faculty and peers.

The “settlement” project(s) during showcase. A TiltFive project and a multi-surface projection video/audio instillation.

The same idea of “let the students learn what you can’t teach anyway” applies to other topics, like students character rigging/weight painting that I am only a novice in.

Again: What I can do is facilitate resources. In this case, virtual learning content.

This final example image is to show the “low-fi speculative prototype”, or for VR projects “brownboxing”. Really useful for VR design!