Wednesday, April 10, 2013

Reflections on Teaching GPU Programming and Architecture: Take III

I've taught CIS 565: GPU Programming and Architecture at the University of Pennsylvania three times now, and every time I like to write a reflections post on what went well and what didn't.

This semester I made some pretty unconventional changes...

No Written Homework. No Tests. No Final.

CUDA path tracer - Aparajith Sairam
My course is about writing code that runs on the GPU so I've tuned it as such. I replaced written homework with longer and harder coding projects. The first three projects were a ray tracer, a path tracer, and a rasterizer, all written in CUDA.

The previous semester a few students did a path tracer for their final project, but this semester we raised the bar and made it a regular project. In general, we are always looking for outstanding final projects that we can turn into a regular project the next semester. We also added a rasterizer - a stripped down OpenGL ES 2.0-like pipeline. Quite frankly, I wasn't sure how feasible it would be, but the results were awesome.

Open Source Projects

Tessellation shaders - Tiju Thomas
The second time I taught, I required students use private GitHub repos to help them learn source control. The feedback was great; students started using GitHub for everything: other classes, personal projects, etc.

This semester we took it a step further; we encouraged them to make all their projects open source (previously we only did this for the final project). The motivation is to help them build a code portfolio for job hunting as inspired by an internship posting by Christophe Riccio:

"The profile for the candidates: C++, OpenGL, OpenCL. Applications with a source code and for the principle of it, a resume."

To discourage cheating, the academic integrity policy was strict and projects had large open-ended parts. No two projects were exactly alike. Students also gave random random presentations in class on their work. I'm sure this discouraged cheating, but the real motivation was to give them experience talking on their feet and explaining their work.

I wrote a full post about our open source experience: GitHub and Open Source in the Classroom.

CUDA path tracer with a kd-tree - Zakiuddin Shehzan Mohammed

Also along the lines of building a portfolio, we had students keep semester-long blogs showcasing each of their projects. Previously, we only required a blog for the final project.

No Final? Really?

OK, I actually told the students that there was a final, but it wasn't graded and they couldn't study for it. On the first day of class, I had students draw a graphics pipeline, first individually then as a group. The result was:

At the end of the semester, we did the same exercise as the "final" (with no prior warning as to what the final was), and the result was much better:

This was almost like a final for me! I wrote a full post on this exercise: Graphics Pipeline: Before and After My GPU Course.

No Grades

CUDA Rasterizer - Himanshu Masand
Last semester, I decided grades just got in the way. They measure rigor, and rigor isn't everything. I want to create a culture of passion for graphics, and to facilitate the quest for knowledge, not the quest for grades. So I let students grade themselves. Well, we reserved 40% of the grade as a sanity check and we, of course, still provided detailed feedback on their work.

The results were great - grades were never a concern as far as I could tell, and students tended to grade themselves harder than I would have, especially the top students.

Instructor/student culture matters and removing grades helps. It makes instructors more approachable; making them less like a boss and more like a mentor.

This also helped create a culture of doing projects for the right reasons, not for grades. For example, several students continued working on their projects after the due-date just because they wanted to make them better.

Course != Lectures

CUDA path tracer - William Boone
Traditionally, we look at a course as covering a topic using lectures and reading, and with homework, projects, and exams to drive home the material.

One minor thing I did was re-brand "homework" as "projects." No one wants to do homework, but everyone loves projects. In my course, everything is projects anyway. This is a minor wording change, but it helps with course culture.

I also fundamentally revisited how I look at a course. It is not a series of lectures; it is a series of projects with supporting lectures. I'm not an instructor; I'm more like a coach helping the team prepare for events (projects). One problem I ran into is that I want to do more projects, but it's hard for lectures to keep pace.


A theme this semester was that we can't teach passion, but we can create an environment that fosters it. One of the ways this manifested itself is with a hackathon. I canceled class one day and replaced it with a six-hour hackathon using the GLSL Sandbox and GPU ray marching. The results were great, the energy was awesome, and the students were focused.

This wasn't graded - that would take the fun out of it. Instead I let the students judge each other for prizes. Strictly speaking, the students didn't even have to be there, but everyone was there and they pushed hard. I can't reproduce that with an exam.

Yuanhui Chen presents her hack

I wrote a full post about our hackathon: Hackathons in the Classroom.

Guest Lectures

NPR - Kong Ma
We were fortunate to have several great guest lectures this semester. Dave Shreiner from ARM talked about mobile graphics, OpenGL, and the future of graphics programming; Graham Sellers from AMD talked about OpenGL from the implementer's perspective; and Tim Kaldewey joined us twice to talk about GPU search and data warehousing.

I like guest lectures because they bring in an outside expert perspective, and overall add to the excitement of the course. I will always cancel one of my lectures for an outstanding guest lecture, but don't be fooled, setting up a guest lecture is more time consuming that preparing a lecture, but it is worth it.

Flexible Schedule

Globe Rendering - Ashima Gupta
In previous semesters, I planned out dates for the projects and lectures well in advance and tried hard to stick to them so students could plan their time in advance.

When introducing potentially last-minute guest lectures and trying projects that we've never done before, keeping a rigid schedule was too hard. Instead, we only planned one project out and adjusted the amount of time as needed.

Not For Everyone

These off-the-wall teaching methods won't work for everyone or for every course. I am lucky in that my course usually attracts second year graduate students. I would change things up for, say, an introduction to computer science course.

What Didn't Go Well

Although I think this was my most successful semester - in that the students enjoyed it and several took GPU-related jobs or jobs thanks, in part, to this course - a few things did not go well.
  • Lack of performance focus - This is embarrassing, but we put so much effort into just seeing if we could get students to be successful with the new projects, that I ignored a lot of performance details in the projects. This really hit home when one student told me they went the entire semester without using shared memory in CUDA. I think I got performance right the first semester I taught so I need to improve this.
  • Lack of tools - I'm a big fan of developer tools, but I barely touched them. We need detailed walkthroughs of tools like the WebGL Inspector and Parallel NSight.
  • Not enough OpenGL - I made the false assumption that the students already knew OpenGL pretty well, and when material needed to be cut, I cut OpenGL. Big mistake.
  • Lectures were OK - The best lectures I give are the ones I just created from scratch. I had some tough time commitments, namely NORAD Tracks Santa, which lead me to giving a few lectures reusing slides and barely looking at them beforehand. Those lectures were OK, but I'm striving for much better than OK.

Ideas for Next Semester

CUDA path tracer - Seunghoon Park
For this upcoming fall, we're going to try some new ideas:

  • More hackathons - the students liked the hackathon so much that they suggested doing more of them. In particular, doing one strictly on optimization. Students come in with their basic GPU ray tracer or path tracer and then compete to make it as fast as possible during the hackathon.
  • Compete against another university - I wanted to compete against a similar course at another university for a while. This could be a joint hackathon or a joint project. There could be prizes for best performance, best visual quality, etc. This would really build some excitement and exposure for the students.
  • Alternative final projects - most students do a flashy final project, usually a new rendering technique. I want to encourage an alternative route that lacks the flash, but impacts the graphics community: write or contribute to an open-source tool that the graphics community needs, e.g., think texture compression libraries, shader translation, etc.
  • Edge Detection - Gundeep Singh Bharma
  • Focus more on WebGL, OpenGL ES, and mobile architecture.

Finally, much of this semester's success was due to my teaching assistant, Karl Li, who implemented the starter code for the projects and supported the students. I simply said things like "let's do a rasterizer" and Karl made them happen.

Also, here's my reflections on previous semesters:

Monday, April 8, 2013

Graphics Pipeline: Before and After My GPU Course

A big part of my GPU course is the modern graphics pipeline. Before students take my class, they usually have had one or two graphics classes, but not necessarily much experience with real-time rendering.

To see how well we did this semester, on the first day of class, I asked each student to draw the graphics pipeline on a sheet of paper. Then I asked the class as a whole to collaborative and draw a graphics pipeline on the board. We came up with something like this:

Then I handed out the pipeline map from OpenGL Insights, and said by the end of the semester they would have a pretty good understanding of it.

The semester went on, and we did a bunch of cool projects, including a graphics pipeline implemented in CUDA.

At the end of the semester, I told the class that there would be a final, but they didn't need to study for it and that it wouldn't be graded. For the "final", we repeated the first day of class: students individually drew pipelines, followed by a pipeline by the entire class:

Wow - what a difference! I also handed back individual pipelines so students could compare theirs to the first day of class.

Sunday, April 7, 2013

How developers can get into teaching

A friend recently asked:

I'm considering teaching a course either at the college or high school level in programming similar to what you do. Do you have any tips on how I can learn more about the opportunities? Should I get a masters if I plan to teach college level courses? How did you reach out to schools about teaching?

I really like the idea of developers from industry teaching computer science courses. Practitioners bring a real-world perspective and coolness to the classroom that is hard for full-time faculty to mimic. The academic rigor also rubs off on practitioners by exploring a subject at a new depth. "You don't know something until you can teach it."

Here's a cleaned up version of the answer I gave my friend:

Should I get a masters if I plan to teach college level courses?

Major universities require that professors have a PhD. Generally their PhD is from a more prestigious university and it is rarely from the same university. However, there are part-time and full-time teaching positions, called lecturers (lecturer has a different meaning in the UK), that do not require a PhD. For example, Penn pretty much always has openings for lecturers.

A research-focused university may prefer lecturers have a PhD, but only require a master's degree. Without a PhD, you may find yourself running into glass ceilings, like not being able to introduce a new required undergraduate course, but if your ideas are good, things can happen one way or another. Many universities, especially those with only undergrad programs and/or where most students go into industry, allow lecturers to have just a bachelor's degree, especially if they have good industry experience.

So, no you don't need a masters degree, but it increases the number of places you can teach at - and helps you build an academic network.

How did you reach out to schools about teaching?

I didn't. I was lucky. I knew a lot of folks at Penn after going there, Joe Kider in particular, and they encouraged me to apply to teach the GPU course.

It's not always that simple, but there's plenty you can do...

Give a talk at a local university

There's two ways to do this.
  • Reach out to the instructor of a relevant course and offer to give a guest lecture. Pretty much all computer science departments list their current courses. Likewise, faculty have websites with their email addresses. For example, here's Penn's faculty. Write a concise email to the instructor saying where you work, what you'd like to give a talk on, and how it maps to their course. Having slides already made could help, but I've personally never seen it.

    Generally, an instructor really likes guest lectures, especially for graduate-level and seminar-style courses. In fact, I spend a good bit of time trying to find guest lectures. Given that I teach a GPU course, I'm thrilled to have folks from GPU vendors, game studios, research labs, etc. come in and talk. Even though I am from industry, guests bring another real-world perspective. In fact, I am willing to cancel one of my lectures for the right guest lecturer. They are one of the highlights of the semester.

    Make sure to get your timing right. For example, don't email the instructor with two weeks left in the semester to see if you can come in. It's hard to say when the best time is because preparation varies by instructor. I generally firm up guest lectures at the start of the semester or a month or so before. Just a warning - the first week of classes is generally crazy.

  • Reach out to a faculty in your area of interest and offer give a talk to the entire department. This is nice because you'll meet more people and sometimes instructors don't want to give up class time. Most departments have a colloquium series where visitors from academia or industry present their work. For example, here's Penn's colloquium. A lot of time this is research-focused, but it doesn't have to be, especially at a smaller university.

    When I defended my master's thesis, I invited graphics faculty from the local universities. None of them actually came, but it did result in me being invited to speak at the Villanova colloquium. My work wasn't full research. It had a minor contribution or two, but it was really more like "here's a survey of a field and a cool project with a careful write-up." I imagine most practitioners could prepare a talk like this.

There's a few motivations for giving a talk.
  • You are passionate about a topic that you want to share with others. If this isn't the case, teaching isn't for you.
  • Giving a talk isn't the same as teaching, but it is good practice and relevant experience.
  • It is good for networking even if there aren't any current openings at that particular university.

Give a talk at a conference

Giving a talk at a conference can have a lot of the same benefits of giving a talk at a university. It's good for sharing knowledge, building experience, and networking, especially if a lot of academics attend the conference.

Depending on the conference, getting a talk accepted can be hard. I've been rejected a couple of times. No worries, everyone gets rejected at least sometimes.

Big conferences like SIGGRAPH are competitive, but it's possible to join an established course or partner to create a new course or panel, talk at a Birds of a Feather (BOF), or even hold your own BOF. Something like a BOF talk won't hold as much credibility in the eyes of an academic but it's still fun, and good experience and networking.

Big conferences aren't the only option. Your company's user's conference is an easy option. So are local meetups. Smaller and industry-focused conferences are also a good option.

You can also give a talk at company to build experience. Most places have brownbags and invited speakers. It will help to know someone there who can host you. For example, I gave a talk on our WebGL engine, Cesium, at MIT Lincoln Lab through a friend I previously worked with at IBM.

Give a talk at a local high school

Most high schools have computer science and even AP computer science classes now. The teachers love having someone from industry talk to their students. It helps them bridge the gap between what students see in the classroom and what students can do in the real-world.

I've been back to my high school a few times. One time I wrote a small Java framework to play tic-tac-toe. As a lesson in polymorphism, students could implement a class to write their AI, and then their AIs could play each other. I even had some of my friends implement AIs so the students could play the "pros."

Teachers would also be thrilled for you to come in and talk about how you got into development, what you do everyday, and what advice you have for students. A talk like this is pretty quick to prepare too.

Other ideas

Although I don't have any experience with them other than serving as a reviewer, teaching an online course like those offered by Udacity, is another option. The student interaction is quite different, but it could still be rewarding and you could reach a lot of people.

Wednesday, March 27, 2013

How I got involved in glTF and Khronos

glTF is the GL Transmission Format - for WebGL, OpenGL ES, and OpenGL. It's currently a work-in-progress.

I've thought about adding 3D models to Cesium, our WebGL virtual globe engine, for a while. My plan was to convert COLLADA, a widely-used XML-based interchange format, to something directly consumable by a WebGL engine. For example, polygon primitives would be triangulated into indexed triangle lists, shaders for materials would be pregenerated, and geometry would be stored in binary to be directly feed to WebGL, and so on. This is a very common practice in the 3D engine world - assets are stored in a fairly verbose format suited to modeling tools, and then processed into a format more palatable to the engine as part of a content pipeline.

I was designing yet another asset format, but mine was different, of course! I was tailoring it to WebGL and carefully designing the content pipeline so assets would be fast and trivial to render. I was exposing the content pipeline through an SDK, command-line tool, and REST API. Yes, mine was different.

Well, I wasn't alone with these ideas - they're not exactly original anyway. In fact, something very similar, called WebGLTF at the time, was demoed at SIGGRAPH last August by Fabrice Robinet and Brandon Jones. I took note.

For a while I declared that I could write a model renderer in our engine in one sitting. In a Sunday hackathon, I did just that. Using the COLLADA2JSON converter and the then WebGLTF JavaScript loader, and with help from Matt Amato, we put the famous COLLADA duck model in the orbit of the International Space Station around Earth. Here and there, Tom Fili and I improved on this mostly by COLLADA optimizations to the content pipeline, which allowed VT MAK to build a simulation app to demo at ITSEC.

I quickly declared that the folks working on WebGLTF were creating exactly what I envisioned. Even better, they were doing it as part of Khronos, the group that publishes my favorite open-standards including OpenGL, WebGL, and COLLADA. So instead of me designing yet another asset format, I could just use theirs and reap the benefits of their code and broader content support. Win. Then it got better; they asked me to help design the spec. Win. Then it got better again. In the spirit of transparency and early community feedback, the kind folks at Khronos allowed us to make the spec available well before we are actually done. Win. Even better, all the spec discussion now happens in the open as GitHub issues. Win.

As you probably guessed, we renamed WebGLTF to glTF, the GL Transmission Format - for WebGL, OpenGL ES, and OpenGL. Admittedly, it's very WebGL biased at the moment. Now I'm just trying to balance spec work with actual implementation work...

Thursday, December 27, 2012

GitHub and Open Source in the Classroom

I started using source control in high school, and I believe every computer science student should be comfortable with at least one source control package. Last semester in my GPU Programming and Architecture course at the University of Pennsylvania, we had students use GitHub for their code. This semester, we took it a step further.

Last semester, we used a GitHub educational account so each student could access a free private repo. To release a project, we added a new directory to each student's repo (which was slighly less work than creating a new repo; forking was not an option since each student wasn't required to create an educational account). Students submitted their work by creating a tag. This all worked OK. Students learned basic git features. Rolling out fixes to our starter code was reasonable enough.

But we could do much better.

This semester we said goodbye to private repos, and went completely open source (well, students still had the option to use private repos, but no one did). To release a project, we simply added the new repo to our GitHub organization, and students forked it. Students submitted their work with pull requests. Besides better exposing students to git, this had some major benefits.

Code portfolios

Each student built a code portfolio, which will help them start their careers. This is inspired by an internship posting by Christophe Riccio:

"The profile for the candidates: C++, OpenGL, OpenCL. Applications with a source code and for the principle of it, a resume."

I whole-heartily agree with the spirit of this. Students with public code portfolios are more marketable than students without. Public code portfolios give students more exposure and give employers a lower barrier-to-entry. By diffing repos, employers can also see what exactly the student did vs. what was provided.

Combine a public code portfolio with cool projects (our students implement GPU ray tracers, path tracers, and rasterizers to name a few), and quite frankly, I don't know how students without a code portfolio will compete.

In addition to helping our students, keeping everything public has the potential to help anyone.

Help the larger community

Interested in writing a GPU path tracer? Fork ours. Don't stop there, read our slides and listen to the audio. They're public for everyone's benefit. Ultimately if the course gets attention, it also benefits those who took the course.

Academic integrity

The most common question I get is if everyone's work is public, can't students cheat?

Yes, but students don't need public repos to cheat. They can talk among themselves and there's already lots of code online.

There's also lots of things that discourage cheating:
  • No two projects are alike - we never provide rigid requirements. For example, the image processing project gives a selection of filters to implement, and requires students to come up with new ones. The rasterizer gives a set of required stages, and a selection of extra ones. In practice, projects turned out to be remarkably different.
  • Random presentations follow projects - three or four students present their projects in class without any notice. The main motivation is for students to get use to talking on their feet giving an elevator pitch. Discouraging cheating for fear of not presenting well is perhaps a side-effect.
  • Social pressure - the code trail is public and commits have time stamps.

Cheating is, of course, still possible, especially copying one-liners. However, the benefits of public code repos outweigh the risk. We strive for a course culture where students don't care to cheat. They want to implement the projects - actually, even students outside of our course want to implement them. If a student wants to chase after grades, they need to take another course. In our course, we stay up to 2am coding for fun, not grades.

Saturday, December 1, 2012

Hackathons in the Classroom

I teach a graduate-level GPU Programming and Architecture course at the University of Pennsylvania. I do lots of atypical things like open source projects using github (there is no "homework" - who wants to do homework?) and I've even let the students make their own final - of course, they didn't know they were doing it at the time. This year I got rid of the final altogether to allow us to focus more on building portfilios of projects like, for example, the work of Zakiuddin Shehzan Mohammed.

Inspired by Facebook and PennApps, and based on feedback from last semester, I decided to bring a hackathon into the classroom. The goal being to build a culture of excitement and intensity, and to code for the fun of it. No grades, just bragging rights and prizes: a copy of OpenGL Insights and a few Cesium t-shirts.

Tiju Thomas presents his hack

I canceled class and replaced it with a six-hour hackathon, 6pm-12am, in the graphics lab using the GLSL Sandbox, an online editor for writing GLSL fragment shaders. The theme was two triangles (or even one) ought to be enough for anybody. The GLSL Sandbox renders a full-screen quad providing one fragment per pixel for students to ray trace, ray march, or do anything else they liked. They were allowed to fork any existing example in the gallery as long as they demoed the original along with their work.

I even encouraged students to fork each other's work as a fun way to compete, e.g., a student might focus on soft shadows, procedural textures, and other shading techniques, and then fork another student's work doing CSG to create something really awesome. This arms race didn't pan out in practice, but with more preparation - perhaps a project on ray marching leading up to the hacakthon - it could be really interesting.

Ashima Gupta and Yue Hue present their hack

Once the hackathon got rolling, there was a sense of focus and intensity. By the end, all students got something working, and many surprised themselves. People outside of my course even joined in, like Nop Jiarathanakul's procedural smoke and Gabriel Leung's Conway's Game of Life

We stayed until about 2am doing demos and reflecting on the experience. It was very cool to see students motivated and working hard for the sake of the project; there were no grades involved. A student could have walked out or not even showed up with nothing to lose, but no one did.

Yuanhui Chen presents her hack

Moving forward, we may compete with a class at another university and do a longer event open to all our graphics students. I want to thank our TA, Karl Li, for helping with the event, Nop Jiarathanakul for demoing his work at the kick-off, and Aline Normoyle for letting us use the lab.

Check out the results:

WebGL, Virtual Globes, and Santa

I've been quiet on this blog recently; in addition to teaching GPU Programming and Architecture again, I have been working on a fun project using WebGL and Cesium, our virtual globe, to help NORAD track Santa this Christmas Eve.

NORAD has a long history of tracking Santa. The NORAD Tracks Santa site is very popular, last year it received 19 million unique visitors over the holiday season. NORAD has provided a 3D globe with Santa's real-time location on Christmas Eve before. However, thanks to WebGL, this is the first time the globe does not require a plugin, so I expect 3D will reach even more people.

I wrote a blog post on the Cesium website with many of the tech details: rendering global terrain, imagery, stars, etc. However, some of the biggest challenges weren't strictly technical, for example, how do you design a UI for 5-10 year-olds and their parents?

Of course I am excited about this opportunity for Cesium, but I also see this as a huge opportunity for WebGL. APIs need killer apps to drive adoption. Although there are lots of amazing WebGL demos, there are few WebGL apps that have reached the masses with a compelling better experience. This needs to change, and I hope NORAD Tracks Santa is one step in that direction.

Our globe will be live Christmas Eve at In the meantime, there is the tech preview I wrote and instructions for getting WebGL, but if you're reading this blog, I'm pretty sure you already know how to do that.

Tuesday, July 24, 2012

OpenGL Insights

Our new book, OpenGL Insights, will be available at SIGGRAPH, and shipping from Amazon and elsewhere early August.

Christophe Riccio and I served as editors, and worked with 52 authors and 17 technical reviewers from the OpenGL, OpenGL ES, and WebGL communities to create 44 chapters and 712 pages on rendering techniques, performance, debugging, profiling, software design, teaching, and other topics.

The OpenGL Insights website is now up.  It contains:
  • Five sample chapters
    • Octree-Based Sparse Voxelization Using the GPU Hardware Rasterizer by Cyril Crassin and Simon Green
    • Performance Tuning for Tile-Based Architectures by Bruce Merry
    • Asynchronous Buffer Transfers by Ladislav Hrabcak and Arnaud Masserann
    • ARB_debug_output: A Helping Hand for Desperate Developers by António Ramires Fernandes and Bruno Oliveira
    • The ANGLE Project: Implementing OpenGL ES 2.0 on Direct3D by Daniel Koch and Nicolas Capens
  • OpenGL 4.2 and OpenGL ES 2.0 Pipeline Map
    • This was Christophe's idea, and his amazing work created the most detailed pipeline diagram that I've ever seen.  I love it.  I have a copy on my cube wall and plan to hand it out the first day of class in my GPU course.  A two-sided 14x18 inch detachable color version is also included with the book.
  • Tips
    • The book includes short OpenGL tips, for example, Depth writes only occur if GL_DEPTH_TEST is enabled.  The website includes all the tips in the book, and we plan to continuously update it, so please send your tips to

Big thanks to Omar Rodriguez, coauthor of the Browser Graphics Analysis and Optimizations chapter, for his help with the website.

When I look back at the original call for authors, I am delighted that the final book lives up to the high expectations and broad scope that we initially set.  We received an overwhelming number of chapter proposals, especially considering OpenGL Insights was an unestablished, new series.  Authors put in significant effort, sometimes reworking parts several times in feedback loops with reviewers.  All chapters received feedback from multiple reviewers; some chapters had as many as six.  The reviewer's technical knowledge was humbling.

A huge part of this project's success is due to Christophe.  When I initially emailed Christophe in March, 2011 to ask him to join me in this effort, I said: this project needs your enthusiasm; it will not be the same quality without you.  Quite frankly, that was an understatement.  Christophe's passion, drive, vision, belief in community, and OpenGL knowledge are amazing.  Working so closely with him was a rewarding experience.