Why 1-Bit Transformers Will Change the World

First published on HackerNoon

Welcome to the New World!

Transformers, once the domain of companies and research organizations with 8-figure budgets, are undergoing the biggest disruptive change in their (short) history.

Once upon a time, incredibly expensive clusters of A100s and V100s Graphical Processing Units (GPUs) that cost millions of dollars to buy, run,and maintain were necessary for state-of-the-art research work.

However, things have changed.

Welcome to the age of 1-bit transformers, that run on desktop CPUs and low-end GPUs without sacrificing performance or capabilities and inference capacities!

The Scenario

The 1-bit Transformer was first introduced by Kim et al. (2020) as a way to reduce the memory footprint and computational complexity of the original Transformer architecture.

The key idea behind 1-bit Transformers is to quantize the weights and activations of the model to 1-bit values, i.e., -1 or 1.

This quantization process not only reduces the memory requirements of the model but also enables the use of binary operations, which are significantly faster than floating-point operations.

And that’s putting it mildly because binary operations like AND, NOT, and OR run on an extreme order of magnitude of 100,000 faster than FP-32 operations on GPUs when optimized.

However, the exact speedup depends upon the optimization used and the context.

The main advantage of 1-bit Transformers is their ability to achieve comparable performance to their full-precision counterparts while using significantly less memory and computational resources.

The low memory requirements are a revolution in themselves.

They can run on desktop GPUs without any expensive hardware requirements.

Where 175 GB was required, now only 1.2 GB is required!

This means that the transformer technology is now open to everyone, which is an incredible, unbelievable accomplishment.

Yup. Unbelievable.

Industry Sectors

1-bit transformer language models (LLMs), of which BitNet 1.58b (see HuggingFace.co website) is a primitive precursor, can be applied to various industry sectors accessible to the average user without specialized hardware.

This is a paid story—you can read the rest on this link:

Why 1-Bit Transformers Will Change the World | by Thomas Cherickal | Technology Hits | Medium