Criterion-based grading, agile goals, and course (un)completion strategies: a visualization of a student’s learning path on TUT Programming 1 course. The Y-axis shows different automatically assessed programming tasks and the X-axisshows the rounds of the course (mapping to weeks on a semester). Colored balls are completed (the student’s solution passed the tests). Different colors represent different categories/levels of the tasks.

While working on a book chapter called Criterion-based grading, agile goals, and course (un)completion strategies (with Ihantola, Isohanni & Mikkonen) for a new Springer book that concentrates on agile learning strategies etc. we needed to generate lots of visualizations. The visualizations were supposed to show the completion paths of different students for the first programming course at Tampere University of Technology.

I took on the task and chose Python for scripting to digest all the data and transform it to a suitable format, D3.js for generating the actual visualizations, and Puppeteer to handle the generation and PNG saving of ~1000 visualizations.

The visualizations turned out pretty nice and the workflow was good. After nailing down the Python data digestion and what we even wanted from it, I played a ton with D3 and came up with a nice looking visualization. After we all agreed on how the thing should look like I used a simple Puppeteer script to turn everything into a nice directory hierarchy of lots of PNGs. I was happy that each of the steps were self-contained and didn’t depend on any of the other steps. My initial data was in CSV format and I used Python to go through it and dig out the needed bits and pieces. I made a simple visualization.html that contained all HTML, CSS, and JavaScript to display a single visualization, nothing more. I ran Python’s SimpleHTTPServer to serve the page on my localhost and to play with it until it looked nice. In the end I had a very simple Node.JS script that started Puppeteer which uses Headless Chrome browser in the background to do all sorts of webpage manipulation; Puppeteer went over every student’s data and loaded the visualization.html for all of them and turned to page into a PNG file.

I could have gone for all Python solution but I hadn’t had any reason to use Puppeteer before so this was a nice thing to tryout. Also, I had some experience from D3.js so it was much easier to come up with the visualization code using the library and the web instead of some Python plotting library or whatnot. We were happy with the results.

The post Using D3 and Puppeteer to generate visualizations appeared first on Pietari Heino's personal website.