Look Inside Language Models

Ecco is a python library that creates interactive visualizations allowing you to explore what your NLP Language Model is thinking.


Explaining Transformers Article
(visualizations created using Ecco)

Spill the beans!

Give GPT2 some input text, and see it generate the output token by token.

Why do you say that?

Which input word contributed the most to generating each output word?

Zoom, enhance

See more precise input saliency values using the detailed view.

Amateur Brain Surgery

Ecco captures the model's neuron activations. Can you make sense of them?

Locate Intelligence

Examine underlying patterns in neuron activations using non-negative matrix factorization.

Use it now.

Use Ecco now in your browser using these colab notebooks:

Neuron Activation Colab

Input Saliency Colab

Ecco is an alpha release of a research project, with a focus on GPT-based Transformer language models. We'd love to hear your feedback!