Description of the Problem

The legacy nbgrader interface was built with an older version of the Jupyter Notebook backend. Newer versions of the Jupyter Notebook have decoupled the backend from the frontend. As such, the frontend sends and receives data using the Jupyter Server REST API instead of "directly" communicating with the Jupyter Notebook backend code.

This leads to some inconsistencies when using the user interface (UI) to trigger actions such as releasing feedback, auto-grading, among others.

NOTE: the IllumiDesk team is currently testing a new version of the Grader Console to help mitigate these issues.

Solution

Running Commands - Tips and Tricks

Fortunately, the nbgrader package includes a command line interface (CLI) that allows users to run the commands from the Terminal included with the Jupyter Notebook instead of having to rely solely on the user interface. But thanks to Jupyter Notebook magics (also known as IPython magics) users have the option to run these same commands from the Jupyter Notebook interface directly!

Running Commands with Jupyter Magics

Jupyter Notebook magics allow users to emulate a terminal environment within the Jupyter Notebook interface. Magics are similar to aliases or shortcuts since they allow users to emulate a terminal environment without having to leave the familiar Jupyter Notebook interface.

To run commands with Jupyter Notebook magics simply prepend the commands below with an exclamation point. For example, running the command below within the Terminal would look like so:

$ nbgrader --help

Running that same command from the Jupyter Notebook interface is exactly the same but notice how the exclamation point prepends the command within the Notebook interface:

Using the Debug Flag

Running commands with the --debug flag provides more information when troubleshooting operations but is not a requirement. Add the --debug flag at the end of any of the commands below to obtain additional output about the operation's results. For example:

nbgrader generate_assignment "Problem Set 1" --debug

Opening the Terminal Window

To open the Terminal window in Jupyter Notebook Classic click on New --> Terminal from the Jupyter File Manager page (identified with the /tree path in the address bar). However as mentioned above, all of the commands listed below (or any terminal command for that matter) run with Jupyter magics as well.

Operations for Instructors

Produce the version of an assignment to be released to students

The generate assignment command takes one argument (the name of the assignment) and looks for notebooks in the 'source' directory. For example:

nbgrader generate_assignment "Problem Set 1"

Release and Assignment to Students

Release an assignment so students can fetch the assignment from their Notebook environment.

To release an assignment named `assignment1` run:

nbgrader release_assignment assignment1

If the assignment has already been released, you will have to add the `--force` flag to overwrite the released assignment:

nbgrader release_assignment --force assignment1

List the released assignments (used mostly for verification purposes):

nbgrader list

Collect Submitted Assignments

Collect assignments submitted by students. To collect assignment1 for all students:

nbgrader collect assignment1

To collect assignment1 for only student1:

nbgrader collect --student=student1 assignment1

All submissions are timestamped and students can turn an assignment in multiple times. The collect command will always get the most recent submission from each student, but it will never overwrite an existing submission unless you provide the --update flag:

nbgrader collect --update assignment1

Autograde a notebook by running it

To grade all submissions for "Problem Set 1":

nbgrader autograde "Problem Set 1"

To grade only the submission by a student with ID 'Foo':

nbgrader autograde "Problem Set 1" --student Foo

To grade only the notebooks that start with '1':

nbgrader autograde "Problem Set 1" --notebook "1*"

Generate Feedback for Students after Manual Grading

Running the generate feedback command requires that the auto-grading operations have been run and that all grading is complete.

To generate feedback for all submissions for "Problem Set 1":

nbgrader generate_feedback "Problem Set 1"

To generate feedback only for the student with ID 'Foo':

nbgrader generate_feedback "Problem Set 1" --student Foo

To feedback for only the notebooks that start with '1':

nbgrader generate_feedback "Problem Set 1" --notebook "1*"

Release Feedback for Students

Once feedback has been generated, Instructors may release feedback so Students can view in-line comments within their Notebook assignments.

To release the feedback for an assignment named `assignment1` run:

nbgrader release_feedback assignment1

NOTE: Release feedback overrides existing files but is a safe operation since the feedback is associated with the Notebook's unique identifier, not the file itself. Therefore the only way an overwrite would occur is if the Instructor re-runs the grading and re-submits comments with manual grading.