CREASE Project Blog

CREASE in Class

One of the goals of the CREASE project is to improve the usability of research testbeds for network experiments. Part of our recent work has focused specifically on developing new tools to make debugging experiments easier and faster, as debugging can be a significant time sink when designing and running workloads on testbeds.

In order to evaluate if our methods are an improvement, we want to see how outside users are able to use them and, ideally, gain efficiency from them. But we also want to directly use these tools in our own work on FABRIC and for teaching. (For more about how we use FABRIC for teaching, see our SIGCSE'25 paper.) Therefore, once the tooling we're building started to get mature, we gave an early glimpse of the system to students taking CS542—a graduate computer networking course at IIT—to help the students learn network debugging and to show them current research in the area.

Fig 1. Image of the experimental topology students worked on. Monitored links were part of the custom tool being evaluated.

This first contact between students and CREASE required careful handling to ensure that the students were sufficiently prepared to understand how to use it, and that we provided suitable on-boarding and support materials for students. Compared to experienced network professionals, on average we expect students to be less likely to be able to recognize and troubleshoot errors in their tooling, since they're still shaping their understanding of the subject. We didn’t want CREASE to cause more problems than it solved for students!

Before giving students this new tool, we first ran the students through debugging exercises with traditional techniques. This both gave students a deeper foundation on network debugging and continued to familiarize them with the environment on FABRIC, which was the key platform that was being used in the course. When students were given the new tool, we also provided a tutorial video to supplement written instructions and provide further practical guidance to students. This video also provided an example of correct output, so that when some students ran into issues that caused outputs significantly different from the video, they were able to reach out and we could help them figure out how to problem-solve. Finally, we made sure the user-facing side of the debugging exercises we gave students was very clean, with most code hidden in separate script files. This helped to make it obvious where students had to do work, to help students focus on the network debugging problem and avoid possible distraction from extraneous code.

Fig 2. Image of one of the exercises students had to solve, inside a Jupyter notebook. All the working code was in a separate file, so students only saw what was necessary.

Using CREASE in class proved to be highly symbiotic for both the class and the CREASE project. Students learned more about network debugging and saw what newly-imagined tools can look like. Some highly-motivated students provided very clear feedback on ways to improve the tool's usability, and the whole process helped us think more deeply about how to present the tool to a wider audience, and helped us identify several bugs. One improvement we developed afterwards is automatic telemetry, so that when users run into errors, we automatically receive the errors and context useful for solving them. This is being used in future versions of CREASE and helps us reduce the back-and-forth troubleshooting communication with users.