dianna
Deep Insight And Neural Network Analysis
DIANNA is a Python package that brings explainable AI (XAI) to your research project. It wraps carefully selected XAI methods in a simple, uniform interface. It’s built by, with and for (academic) researchers and research software engineers working on machine learning projects.
Why DIANNA?
DIANNA software is addressing needs of both (X)AI researchers and mostly the various domains scientists who are using or will use AI models for their research without being experts in (X)AI. DIANNA is future-proof: one of the very few XAI library supporting the Open Neural Network Exchange (ONNX) format.
After studying the vast XAI landscape we have made choices in the parts of the XAI Taxonomy on which methods, data modalities and problems types to focus. Our choices, based on the largest usage in scientific literature, are shown graphically in the XAI taxonomy below:
The key points of DIANNA:
- Provides an easy-to-use interface for non (X)AI experts
- Implements well-known XAI methods LIME, RISE and KernelSHAP, chosen by systematic and objective evaluation criteria
- Comes with a dashboard where results of different explainers can be compared for all data types
- Supports the de-facto standard of neural network models - ONNX
- Supports images, text, time series, tabular data modalities and embeddings (in a related package)
- Comes with simple intuitive image, text, time series, and tabular benchmarks, so can help you with your XAI research
- Includes scientific use-cases tutorials
- Easily extendable to other XAI methods
For more information on the unique strengths of DIANNA with comparison to other tools, please see the context landscape.
Installation
DIANNA can be installed from PyPI using pip on any of the supported Python versions (see badge):
python3 -m pip install dianna
To install the most recent development version directly from the GitHub repository run:
python3 -m pip install git+https://github.com/dianna-ai/dianna.git
If you get an error related to OpenMP when importing dianna, have a look at this issue for possible workarounds.
Pre-requisites only for Macbook Pro with M1 Pro chip users
- To install TensorFlow you can follow this [tutorial](https://betterdatascience.com/install-tensorflow-2-7-on-macbook-pro-m1-pro/).
- To install TensorFlow Addons you can follow these [steps](https://github.com/tensorflow/addons/pull/2504). For further reading see this [issue](https://github.com/tensorflow/addons/issues/2503). Note that this temporary solution works only for macOS versions >= 12.0. Note that this step may have changed already, see https://github.com/dianna-ai/dianna/issues/245.
- Before installing DIANNA, comment `tensorflow` requirement in `setup.cfg` file (tensorflow package for M1 is called `tensorflow-macos`).
</details>
## Getting started
You need:
- your trained ONNX model ([convert my pytorch/tensorflow/keras/scikit-learn model to ONNX](https://github.com/dianna-ai/dianna#onnx-models))
- a data item to be explained
You get:
- a relevance map overlayed over the data item
### Template example for any data modality and explainer
1. Provide your *trained model* and *data item* ( *text, image, time series or tabular* )
```python
model_path = 'your_model.onnx' # model trained on your data modality
data_item =
Lets illustrate the template above with *textual* data. The data item of interest is a sentence being (a part of) a movie review and the model has been trained to classify reviews into positive and negative sentiment classes.
We are intersted which words are contributing positively (red) and which - negatively (blue) towards the model's desicion to classify the review as positive and we would like to use the *LIME* explainer:
```python
model_path = 'your_text_model.onnx'
# also define a model runner here (details in dedicated notebook)
review = 'The movie started great but the ending is boring and unoriginal.'
labels = ["negative", "positive"]
explained_class_index = labels.index("positive")
explanation = dianna.explain_text(model_path, text, 'LIME')
dianna.visualization.highlight_text(explanation[explained_class_index], model_runner.tokenizer.tokenize(review))
```
Here is another illustration on how to use dianna to explain which parts of a bee *image* contributied positively (red) or negativey (blue) towards a classifying the image as a *'bee'* using *RISE*.
The Imagenet model has been trained to distinguish between 1000 classes (specified in ```labels```).
For images, which are data of higher dimention compared to text, there are also some specifics to consider:
```python
model_path = 'your_image_model.onnx'
image = PIL.Image.open('your_bee_image.jpeg')
axis_labels = {2: 'channels'}
explained_class_index = labels.index('bee')
explanation = dianna.explain_image(model_path, image, 'RISE', axis_labels=axis_labels, labels=labels)
dianna.visualization.plot_image(explanation[explained_class_index], utils.img_to_array(image)/255., heatmap_cmap='bwr')
plt.show()
```
| Dataset | Description | Examples | Generation |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------- | :--------------------------------------------------------------- | :------------------------------------------------------------------ |
| [Stanford sentiment treebank](https://nlp.stanford.edu/sentiment/index.html)
| Dataset | Description | Examples | Generation |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ |
| [Coffee dataset](https://www.timeseriesclassification.com/description.php?Dataset=Coffee)
| Dataset | Description | Examples | Generation |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ |
| [Pengiun dataset](https://www.kaggle.com/code/parulpandey/penguin-dataset-the-new-iris) Examples
And why would Imagenet think the same image would be a *garden spider*?
```python
explained_class_index = labels.index('garden_spider') # interested in the image being classified as a garden spider
explanation = dianna.explain_image(model_path, image, 'RISE', axis_labels=axis_labels, labels=labels)
dianna.visualization.plot_image(explanation[explained_class_index], utils.img_to_array(image)/255., heatmap_cmap='bwr')
plt.show()
```
</details>
### Overview tutorial
There are **full working examples** on how to use the supported explainers and how to use dianna for **all supported data modalities** in our [overview tutorial](/dianna/tutorials/overview.ipynb).
### IMPORTANT: Sensitivity to hyperparameters
The explainers are sensitive to the choice of their hyperparameters! In this [work](https://staff.fnwi.uva.nl/a.s.z.belloum/MSctheses/MScthesis_Willem_van_der_Spec.pdf), this sensitivity to hyperparameters is researched and useful conclusions are drawn.
The default hyperparameters used in DIANNA for each explainer as well as the values for our tutorial examples are given in the Tutorials [README](/dianna/tutorials/#important-hyperparameters).
### Introductory video
This video shows the main functionality of DIANNA and shows you how to use DIANNA also from its dashboard.
[Watch the video on YouTube](https://youtu.be/9VM5acip2s8)
## Dashboard
Explore the explanations of your trained model using the DIANNA dashboard.
[Click here](https://github.com/dianna-ai/dianna/tree/main/dianna/dashboard) for more information.
## Datasets
DIANNA comes with simple datasets. Their main goal is to provide intuitive insight into the working of the XAI methods. They can be used as benchmarks for evaluation and comparison of existing and new XAI methods.
Images
| Dataset | Description | Examples | Generation |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Binary MNIST 
| Greyscale images of the digits "1" and "0" - a 2-class subset from the famous[MNIST dataset](http://yann.lecun.com/exdb/mnist/) for handwritten digit classification. | 
| [Binary MNIST dataset generation](https://github.com/dianna-ai/dianna-exploration/tree/main/example_data/dataset_preparation/MNIST) |
| [Simple Geometric (circles and triangles)](https://doi.org/10.5281/zenodo.5012824) 
| Images of circles and triangles for 2-class geometric shape classificaiton. The shapes of varying size and orientation and the background have varying uniform gray levels. | 
| [Simple geometric shapes dataset generation](https://github.com/dianna-ai/dianna-exploration/tree/main/example_data/dataset_preparation/geometric_shapes) |
| [Simple Scientific (LeafSnap30)](https://zenodo.org/record/5061353/) 
| Color images of tree leaves - a 30-class post-processed subset from the LeafSnap dataset for automatic identification of North American tree species. | 
| [LeafSnap30 dataset generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/dataset_preparation/LeafSnap/) |
Text
| Dataset for predicting the sentiment, positive or negative, of movie reviews. | _This movie was actually neither that funny, nor super witty._ | [Sentiment treebank](https://nlp.stanford.edu/sentiment/treebank.html) |
| [EU-law statements](https://zenodo.org/records/8200000) 
Time series
| Food spectographs time series dataset for a two class problem to distinguish between Robusta and Arabica coffee beans. |
| [data source](https://github.com/QIBChemometrics/Benchtop-NMR-Coffee-Survey) |
| [Weather dataset](https://zenodo.org/record/7525955)
| The light version of the weather prediciton dataset, which contains daily observations (89 features) for 11 European locations through the years 2000 to 2010. |
| [data source](https://github.com/florian-huber/weather_prediction_dataset) |
</details>
Tabular
| Palmer Archipelago (Antarctica) penguin dataset is a great intro dataset for data exploration & visualization similar to the famous Iris dataset. |
| [data source](https://github.com/allisonhorst/palmerpenguins) |
| [Weather dataset](https://zenodo.org/record/7525955) | The light version of the weather prediciton dataset, which contains daily observations (89 features) for 11 European locations through the years 2000 to 2010. |
| [data source](https://github.com/florian-huber/weather_prediction_dataset) |
| [Land atmosphere dataset](https://zenodo.org/records/12623257)
| It contains land-atmosphere variables and latent heat flux (LEtot) simulated by STEMMUS-SCOPE (soil-plant model), version 1.5.0, over 19 Fluxnet sites and for the year 2014 with hourly intervals. |
| [data source](https://zenodo.org/records/12623257) |
</details>
## Models
**We work with ONNX!** ONNX is a great unified neural network standard which can be used to boost reproducible science. Using ONNX for your model also gives you a boost in performance! In case your models are still in another popular DNN (deep neural network) format, here are some simple recipes to convert them:
* [pytorch and pytorch-lightning](https://github.com/dianna-ai/dianna/blob/main/tutorials/conversion_onnx/pytorch2onnx.ipynb) - use the built-in [`torch.onnx.export`](https://pytorch.org/docs/stable/onnx.html) function to convert pytorch models to onnx, or call the built-in [`to_onnx`](https://pytorch-lightning.readthedocs.io/en/latest/deploy/production_advanced.html) function on your [`LightningModule`](https://lightning.ai/docs/pytorch/latest/api/lightning.pytorch.core.LightningModule.html#lightning.pytorch.core.LightningModule) to export pytorch-lightning models to onnx.
* [tensorflow](https://github.com/dianna-ai/dianna/blob/main/tutorials/conversion_onnx/tensorflow2onnx.ipynb) - use the [`tf2onnx`](https://github.com/onnx/tensorflow-onnx) package to convert tensorflow models to onnx.
* [keras](https://github.com/dianna-ai/dianna/blob/main/tutorials/conversion_onnx/keras2onnx.ipynb) - same as the conversion from tensorflow to onnx, the [`tf2onnx`](https://github.com/onnx/tensorflow-onnx) package also supports keras.
* [scikit-learn](https://github.com/dianna-ai/dianna/blob/main/tutorials/conversion_onnx/skl2onnx.ipynb) - use the [`skl2onnx`](https://github.com/onnx/sklearn-onnx) package to scikit-learn models to onnx.
More converters with examples and tutorials can be found on the [ONNX tutorial page](https://github.com/onnx/tutorials).
And here are links to notebooks showing how we created our models on the benchmark datasets:
Images
| Models | Generation |
| :-------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Binary MNIST model](https://zenodo.org/record/5907177) | [Binary MNIST model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/MNIST/generate_model_binary.ipynb) |
| [Simple Geometric model](https://zenodo.org/deposit/5907059) | [Simple geometric shapes model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/geometric_shapes/generate_model.ipynb) |
| [Simple Scientific model](https://zenodo.org/record/5907196) | [LeafSnap30 model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/LeafSnap/generate_model.ipynb) |
Text
| Models | Generation |
|:---------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Movie reviews model](https://zenodo.org/record/5910598) | [Stanford sentiment treebank model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/movie_reviews/generate_model.ipynb) |
| [Regalatory statement classifier](https://zenodo.org/record/8200001) | [EU-law regulatory-statement-classification](https://github.com/nature-of-eu-rules/regulatory-statement-classification) |
Time series
| Models | Generation |
| :-------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Coffee model](https://zenodo.org/records/10579458) | [Coffee model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/coffee/generate_model.ipynb) |
| [Season prediction model](https://zenodo.org/record/7543883) | [Season prediction model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/season_prediction/generate_model.ipynb) |
| [Fast Radio Burst classification model](https://zenodo.org/records/10656614) | [Fast Radio Burst classification model generation](https://doi.org/10.3847/1538-3881/aae649) |
Tabular
| Models | Generation |
| :-------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Penguin model (classification)](https://zenodo.org/records/10580743) | [Penguin model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/penguin_species/generate_model.ipynb) |
| [Sunshine hours prediction model (regression)](https://zenodo.org/records/10580833) | [Sunshine hours prediction model generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/sunshine_prediction/generate_model.ipynb) |
| [Latent heat flux prediction model (regression)](https://zenodo.org/records/12623257) | [Latent heat flux prediction model](doi:10.5281/zenodo.12623256/stemmus_scope_emulator_model_LEtot.onnx) |