Integrated Gradients

This chapter covers results in the paper Axiomatic Attribution for Deep Networks available at this Arxiv link.

Lecture

The lecture slides are available here.

Introduction to Attributions and Baselines

--- primaryColor: steelblue shuffleQuestions: false shuffleAnswers: true --- ### A natural baseline is an input where the DNN is “neutral”. In case of object recognition, the paper suggests that a neutral input is - [ ] an image where the colors have been inverted. - [x] a black image, i.e., an image with all 0s. > Correct. - [ ] a red image, i.e., an image where all the non-red channels are 0s.

Prior Work: Lack of Sensitivity and Invariance

--- primaryColor: steelblue shuffleQuestions: false shuffleAnswers: true --- ### An earlier approach solved the visualization problem by performing gradient search - [ ] in the output space to maximize the input. - [x] in the input space to maximize the output. > Correct. - [ ] in the input space to maximize the loss function.

Integrated Gradients: Completeness, Sensitivity, Implementation Invariance, Dummy and Linearity

--- primaryColor: steelblue shuffleQuestions: false shuffleAnswers: true --- ### Completeness - [ ] requires that attribution for an input must be zero if the function changes values when the input changes. - [x] requires that attribution for an input must be non-zero if the function changes values when the input changes. > Correct. - [ ] requires that attribution for an input must be non-zero if the function does not change values when the input changes. ### Consider an attribution method that respects linearity. If we linearly compose 2 DNNs f1 and f2 with attributions A1 and A2 with respect to an input to form a third DNN c f1 + d f2, then the attribution of the new DNN with respect to the same input - [ ] A1 + A2. - [x] c A1 + d A2. > Correct. - [ ] c + d.

Integrated Gradients is the Symmetry Preserving Path Integral

Experimental Results and Conclusions

--- primaryColor: steelblue shuffleQuestions: false shuffleAnswers: true --- ### Integrated Gradients, as an attribution method, is only applicable to images. The statement is: - [ ] True. - [x] False. > Correct. - [ ] Integrated Gradients is not an attribution method.

Code and Assignment: Attributions

A minimal working example for creating an attribution similar to the integrated gradient of an input given a class label is presented in the Colab document. Here are a few illustrations generated by this minimal example:

Image	Attribution

Assignment 3: Creating Integrated Gradients for Models (Due March 13, 2023)

Using any input benchmark of your choice such as ImageNet and any model of your choice such as ResNet50, write a Jupyter notebook to identify components of an input that cause it to be predicted as a given label (20 points). Do not copy code from the minimal working example above. The computed attribution should be based on the idea of integrated gradients.

The minimal working example uses a blank image as a baseline. In your implementation, change the baseline to another image, such as random noise, an adversarial example (if you know about adversarial methods in machine learning), or an image of your choice. Then, compare the subjective visual quality of the computed attribution for the two different baselines in at least 10 images of your choice. (40 points)
The minimal working example uses 200 steps (NSTEPS=200) to approximate the integral. Vary the number of steps in your implementation between 5 and 200, and compare the subjective visual quality of the computed attributions for at least 10 images of your choice. (40 points)
Subjective estimates of the visual quality of the computed attributions are clearly not satisfactory. One objective approach to measure attribution quality may be to hide all those pixels in the image with relatively low attribution values, and check if the neural network still correctly classifies the resulting image. There may be other ideas for measuring the quality of the computed attributions objectively. Please suggest an objective or algorithmic method for evaluating the quality of computed attributions and repeat (1) and (2) above using this method for comparing the quality of computed attributions. (Extra Credit: 50 + 50 points)

FAQs

These assignments will be evaluated as graduate assignments, and there is no single correct answer that is expected. A variety of answers can equally satisfy the requirements in the above assignments.

Q1. I was wondering if you want the notebook to look like the example one, such as including load model, load input, transformation, and visualization.

A1. The code does not have to look like the example notebook. You can arrange the workflow in the code as you deem fit.

Q2. Do we have to use the ImageNet benchmark?

A2. You can use MNIST, CIFAR-10, or any benchmark data set of your choice.