This chapter covers results in the paper Axiomatic Attribution for Deep Networks available at this Arxiv link.
The lecture slides are available here.
A minimal working example for creating an attribution similar to the integrated gradient of an input given a class label is presented in the Colab document. Here are a few illustrations generated by this minimal example:
Using any input benchmark of your choice such as ImageNet and any model of your choice such as ResNet50, write a Jupyter notebook to identify components of an input that cause it to be predicted as a given label (20 points). Do not copy code from the minimal working example above. The computed attribution should be based on the idea of integrated gradients.
The minimal working example uses a blank image as a baseline. In your implementation, change the baseline to another image, such as random noise, an adversarial example (if you know about adversarial methods in machine learning), or an image of your choice. Then, compare the subjective visual quality of the computed attribution for the two different baselines in at least 10 images of your choice. (40 points)
The minimal working example uses 200 steps (NSTEPS=200) to approximate the integral. Vary the number of steps in your implementation between 5 and 200, and compare the subjective visual quality of the computed attributions for at least 10 images of your choice. (40 points)
Subjective estimates of the visual quality of the computed attributions are clearly not satisfactory. One objective approach to measure attribution quality may be to hide all those pixels in the image with relatively low attribution values, and check if the neural network still correctly classifies the resulting image. There may be other ideas for measuring the quality of the computed attributions objectively. Please suggest an objective or algorithmic method for evaluating the quality of computed attributions and repeat (1) and (2) above using this method for comparing the quality of computed attributions. (Extra Credit: 50 + 50 points)
These assignments will be evaluated as graduate assignments, and there is no single correct answer that is expected. A variety of answers can equally satisfy the requirements in the above assignments.
Q1. I was wondering if you want the notebook to look like the example one, such as including load model, load input, transformation, and visualization.
A1. The code does not have to look like the example notebook. You can arrange the workflow in the code as you deem fit.
Q2. Do we have to use the ImageNet benchmark?
A2. You can use MNIST, CIFAR-10, or any benchmark data set of your choice.